Httperf

CS290F Fall 2006 - UCSB Computer Science - Thorsten von Eicken

Creating a workload

The key to using httperf is the --wsesslog workload generator. The first thing to do is to create a file with the workload. To start a little test of the aws-console I wrote the following workload file:

/
/sessions/new
  /javascripts/prototype.js?1160194386
  /javascripts/effects.js?1160194386
  /javascripts/dragdrop.js?1160194386
  /javascripts/controls.js?1160194386
  /javascripts/application.js?1154539549
  /javascripts/niftycube.js?1160194386
  /stylesheets/yui/reset.css?1157476643
  /stylesheets/yui/fonts.css?1157693397
  /stylesheets/yui/grids.css?1160194386
  /stylesheets/syslitics.css?1162714447
  /stylesheets/niftyCorners.css?1160194386
  /images/loading.gif?1154732591
/favicon.ico
/sessions method=POST contents='email=test@voneicken.com&password=test'
/
/ec2_instances
/ec2_images
/ec2_instances
/ec2_images
/ec2_instances
/ec2_images
/ec2_instances
/ec2_images

This file fetches "/", which redirects to "/sessions/new" and which is followed by all the javascripts and stylesheets referenced by the response. Then comes the favicon and that is followed by a POST of the login information. Then it fetches "/" (the login redirects there), and then a bunch of fetches of the instances and images pages.

The way I came up with this list is to start a fresh browser and navigate to the site, and then look at the server log in /var/log/httpd/access_log (or similar) and grab all the URIs listed there. I had to make-up the contents of the post myself, but it's simply a URL-encoded string of the various form fields.

Then I tested this using httperf with the following command line:

httperf --hog --server www.aws-console.com --wsesslog=1,0,aws-test1 \
--session-cookie --print-reply

This runs the test file one time and prints all the replies. Lots of stuff to scroll through, but it allowed me to check that the login and the other page fetches worked properly.

Applying some load

Then a quick test running 10 sessions starting 4 sessions per second:

httperf --hog --server www.aws-console.com --wsesslog=10,0,aws-test1 \
--session-cookie --rate 4

To produce a graph, I would vary --rate from about 0.1 to 5 (ramp starting a new session from one every 10 seconds up to 5 per second). Good numbers for your app will depend on the performance that you see.

Note that when interpreting the requests per second figures you really need to factor in the overall percentage of Rails requests (as opposed to image requests that get handled by apache). In my example only about half the requests are Rails requests as the /stylesheets/... and /javascripts/... URLs get served straight by apache.

Now I need to add more sessions to my description file so that I can exercise different "critical paths". I may also automatically generate variants of the workload file where I change the login id and perhaps some other post parameters.

The "-v" option to httperf is helpful in seeing progress. You can specify a very large number of sessions in --wsesslog and then watch httperf print out its requests/sec measurements every 2-3 seconds. Once you have one to two dozen measurements, hit ctrl-C to get the overall stats.

Figuring out where the time is going

So, why doesn't it do more requests per second than it does? First places to look are the production.log (/home/rails/project/current/log/production.log) to see where the time within each request is going. You can also see which specific requests take the longest. Then check the mysql slow query log (/var/lib/mysql/mysql-slow.log) for any queries that take a long time.

More to follow...