Of Tuning WordPress on Cherokee

So at work, I had a blog.  Not my blog, of course.  One I support.  So this blog was thrown together quickly to facilitate business goals.  Like you do.  And that’s great.  We met the deadline.  We got the product functional.  But performance kinda sucked.  A little backstory.  Here’s how it got set up to start.

We love virtualization here.  Everyone should.  It’s a fantastic way to take adventage of hardware that you’ve got laying around that’s being idle.  Idle hardware is lame. So we run KVM.  This means we get to manage machines more effectively, and can provision things faster.  That’s awesome.

So this blog.  It’s a cluster of boxes, made up of a pair of webservers, and a pair of DB backends.  The DB backends are a master, and a backup.  This is a small project, so these got set up with just 4GB of ram.  So that’s fine.  The web servers are each a VM with 1 vcpu, 4GB of ram, and some disk space.

So we set that all up, and it did fine.  But not great.  here’s what it did.

Incidentally, if you’ve got data you’ve collected for performance with Pylot, and need to CSV the averages to make a pretty graph, here’s my way of extracting the averages from the results HTML to feed into gnuplot.

for file in load-dir/*/results.html ; do cat $file  | egrep 'agents|avg' | head -3 | perl -e 'while(<>){s/<\/?t[dr]>//g; s/(\d+\.?\d*)/\t$1/g; chomp; @p=split /\t/,$_; push @r,$p[1]; }; printf "%s\n", join ",",@r; '; done

So what we can see then from these graphs is that a pair of single core boxes, tuned with the default cherokee+php installation don’t do all that great.  They can handle 1-2 simultaneous requests, but past that, the response time gets pretty bad. That’s where the project got left for a while, until the other day I got the request to make it handle 500 requests per second.  “Wow, shit.” I thought.  So I dove in to see what I could do to improve performance on the blog, and it turned out that there was a lot I could do.

  1. Single-core boxes don’t have a lot of performance available to them.
  2. Cherokee uses PHP in FastCGI mode, and does a terrible job of tuning it out of the box.  It defaults to a single PHP thread.
  3. WordPress is very hungry for CPU time.  It chews up a lot of resources doing its DB queries and wrangling them.

To address these points, I did the following.

The VMs themselves were restarted with more CPU cores — four cores per box.   This allowed me to dive into discovering that Cherokee wasn’t tuned well.  Under a single core, I saw 80% cpu utilization on PHP, with high system wait time.  That sucked.   But after I bumped up the core count, it still looked bad.  Still only one CPU @ 80%. WTF.  So then I turned to Cherokee, and I tuned PHP so that it would invoke 16 children, enough to handle the request volume.  This helped a lot, but there was still room to do better.  So I added APC, the Alternative PHP Cache, to the configuration.  This helped out a lot.  I then looked at wordpress specific caching engines, and settled on using W3-Total-Cache to help out.  This brought performance into the range of fulfilling the customer’s request.  I felt great about it.

I used pylot to graph performance at various points through this project so I could figure out how I was doing a better or worse job of tuning the boxes.

Here’s the performance data showing the improvements that caching at various layers added to the party:

So it turns out that these are some great ways to make sure your blog performs:

  • Make sure that there’s enough hardware supporting it
  • Cache in depth and breadth, early and often
  • Tune your PHP/WebServer to handle the project
  • Employ performance testing to measure your real improvements

With some hard work and performance tuning, you can turn an abysmal 7 requests/sec peak performing blog into one that can sustain 350 requests per second, and do it in a tiny fraction of the time.

-Gabriel

 

Be Sociable, Share!