Making tools that aren't a hammer

Sometimes you need tools that conform to a specific problem, instead of trying to adapt the problems to the tools you have.

There’s a famous saying about hammers and problems. I think it goes like if all you have is a hammer, everything looks like a nail. That’s a common thing to happen on the devops/sysadmin trade and goes completely against the UNIX Philosophy of small tools getting together to achieve big things.

Sometimes we will essentially try to fit the problem to the tool, just because we have all this variety of do everything tools. That, in my opinion, is not worth it. We like to call us devops, right? So let’s throw some dev’ing and create tools tailored to the tasks.

For example, it is possible to load balance, cache and serve PHP using Nginx. That doesn’t mean you should do that unless it’s a small install or non-critic server (go ahead and do it on your personal site, I mean, even I do it), but if you want to keep flexible and scalable, spread out and try to find the best of it’s kind… or the close second. Cache on Varnish, serve PHP from a different box (maybe using Apache), load balance on HAProxy (or Varnish really it’s very good at it).

We put this thought in practice at Vox Media all the time. Not only because we agree with it, it also helps us increase the visibility of our services and do a better job monitoring everything.

Don’t consider it a not built here syndrome. As I said, use the best available or make your own, and don’t feel bad about having to change later when something better appears. Constant evolution!

For example, those are two tools that I made for very specific objectives:

  • slowqwatch: It watches a log and increases a StatsD counter when a regex is matched. Nothing great or fantastic, but allows us to measure rates and plot nice Graphite graphs of stuff like MySQL slow queries (hence the name).
  • ipcount: This may have been a little over-engineered but I wanted to do some stuff using Redis and Go. What this does is basically watching access logs and sending the IPs to Redis, and then a web page shows the top IPs. You know, to catch those spidering robots that hammer your serves like crazy.

I am sure a lot of people will point me to different tools that do the same thing or/and are better at doing those things. The point is, we had an itch and we scratched it, custom made. Of course we can make the same with Splunk, but not every company has the money for that…

TL;DR: stop hitting your problems with the same tools over and over and think about expanding your toolbox, even if that requires you to create the tools from scratch.


comments powered by Disqus