Tag Archives: Linux

DevOps: Using Fabric to fix the Leap Second bug

So today I had a fun problem:  How am I to correct the leap second bug on 700 systems?  Gosh, that’s a thing.  So I went looking in my systems administrator toolbox and found Fabric.  Turns out that you can automate things in pretty awesome ways with it.

Fabric is a python tool that lets you specify a runlist of things to do, and go execute them across a pile of hosts.  For me, this meant:$ cat fabfile.py

$ from fabric.api import run, env
def fix_date():
     run('date; date `date +"%m%d%H%M%C%y.%S"`; date')

$ fab --skip-bad-hosts -t 3 -H \
      `perl -e 'chomp(@l=<>); print join ",", @l;' < hosts.txt `\

And bam, 30 minutes later, the bug is done.


Building Cassandra-Cacti-M6 on Centos 5.5

Turns out this is hard to do.    I’m writing this here for my benefit, and everyone else’s, too.  I’ve got a client using Cassandra, and of course you monitor that stuff.  So I figure that using the cassandra-cacti-m6 stuff is a good plan.  That’s cool, it works with Cacti.  Cacti’s pretty snazzy.   My customer is using Centos 5.5 on the monitoring box.  Turns out there are a lot of hoops to jump through for that to work.  So here’s what I did.

  1. Install jpackage-utils
  2. Install the jpackage.repo into /etc/yum.repos.d, and enable the rhel5 targets
  3. Install JDK6 Update 3 (because of the following step)
  4. Install java-1.6.0-sun-compat from ftp://jpackage.hmdc.harvard.edu/JPackage/5.0/generic/RPMS.non-free/

Then I go to install ANT, but I discover something — trying to install ant.noarch complains about “Missing Dependency: /usr/bin/rebuild-security-providers is needed by package”.  This sucks.  So I do some googling, and figure out that someone solved this issue on Centos 5.x.  I do what they did (documented at http://plone.lucidsolutions.co.nz/linux/centos/jpackage-jpackage-utils-compatibility-for-centos-5.x ).  This works great.  The package is built, I install it out of the local build dir.

Then I install ant.noarch and ant-nodeps.noarch.  These do the trick.

The I build cassandra-cacti-m6 as documented in its source tree.  Woo.

It took me a while to dig up and get all this working, but for you, I hope it’s fast and easy.  Enjoy!