Home

CS290F Fall 2006 - UCSB Computer Science - Thorsten von Eicken

Welcome to CS290F - Scalable Internet Services

!!!!! THIS CLASS TOOK PLACE IN FALL 2006 !!!!!
Registration for this Wiki has been closed.
If you have comments/suggestions, please post them on my blog.
Thank you for visiting!

Lectures: Mon/Wed 9am-10:50am in TRAIL 932
TA Office Hours: Mon/Wed 2pm-3pm in Engr. 1, Rm. 2152A
Final project presentations: Thursday December 14th 9am-noon.

What was this course about?

Learn about all the technologies that go into a scalable internet service, specifically into dynamic web sites. This was a very hands-on, learn-by-doing course with a significant project component. The lectures provided background information to support the project and they explained technologies beyond the scope of the project.

The project consisted of building a transactional dynamic web site in Ruby on Rails and running on Amazon's Elastic Compute Cloud (EC2). Each site had to hold >100'000 database records that could be searched and explored, have user accounts, and include some form of transaction, such as a shopping cart check-out.

Each project then had to be deployed on multiple servers on EC2 and the groups had to use httperf to demonstrate that they could scale the performance of their site by running a front-end load balancer server, a database server, a memcached server, and up to 10 application servers. All this had to fit into a 10-week quarter, with none of the students knowing either Ruby or Rails at the outset!

Note that the emphasis of the course was on the scalability aspect of the sites and not on the web design or feature-set. Thus it was more important to understand the performance characteristics and optimize the core of the site than to have the most eye-candy. (Although eye-candy is always appreciated, of course...)

Featured projects

In the end the following projects stood out for their scalability and their exploration of the optimization space:

GameZone Roopa Kannan & Keshava P Subramanya Online PC game store with data from Yahoo!. Check out the scalability graph: X-axis is applied load, Y-axis is requests serviced/sec, the lines are from 1 app server to 8 app servers. (Go straight to the full explanation.)
BaDTunes Brett Stone-Gross & Dominic Metzger Music store with custom playlists for purchase, data from Rhapsody and Yahoo!. Not as linear, but still impressive scalability to 70 dynamic pages/sec on 10 servers! Again X-axis is load, Y-axis is requests serviced/sec, the lines are 1, 3, 5, 10 app servers. (Go straight to the full explanation.)
BukManiac Neha Mittal & Tushar Ranka Online book store with reviews and ratings. The graph shows applied load on the X-axis, requests serviced/sec on the Y-axis, and from 1 app server to 10 app servers. Doesn't get much cleaner than that and at over 200 dynamic page requests/sec at the peak it's pretty hot! (Go straight to the full explanation.)

Check out all 15 projects!

Lessons learned

• The Ruby on Rails framework has been excellent for this course! Students got a lot further than 5 and 6 years ago when I taught the same course using Java servlets and JSP pages!
• Ruby on Rails scales very nicely: add servers and faster it goes! But the single-server performance really needs a 5x boost before I'd be satisfied with the overall results. Let's hope work on Ruby 1.9 or other performance enhancements make good progress soon!
• Amazon EC2 was a pleasure to use for scalability experiments: we used over 80 machines when multiple groups ran experiments concurrently. Some additional automation would be needed to make it more convenient and reduce the urge to leave servers running 24x7.

Optimization Lessons

• Do triage before optimizing:
  • Figure out if the database or the application is the bottleneck (or something else altogether). Be sure to optimize the bottleneck, not the rest. The groups that spent the time to optimize on one server to make the app server the bottleneck were successful at scaling to multiple app servers. The other groups got just a little speed-up. Obvious, but sometimes needs to be remembered...
  • If adding servers doesn't give linear scaling (see the graphs for the featured applications), then the database is almost certainly the bottleneck.
• If the application is the bottleneck, just add more servers
  • This is easy and cheap on EC2: one day of programmer time is worth about half an instance-year on EC2!
• If the database is the bottleneck, here are some suggestions:
  • Use slow query logs to figure out which queries are a problem and change or eliminate them
  • A really common bottleneck are queries like "SELECT count(*) from X". Often these are generated by dumb pagination code displays the total item or page count. Try to just eliminate these queries, or consider keeping an updated counts table if the information is really essential.
  • Make appropriate use of indexes.
    • Any has_many relationship probably should have an index on the foreign key field.
    • Any field that is frequently queried for specific values should also have an index.
    • Limiting queries (using a limit SQL clause) is much more efficient if the order-by field has an index on it.
  • Use memcached to keep session state instead of storing it in the database.
  • Keep important tables small, specially keep the row-count low:
    • Use subsidiary tables to store uncommonly accessed information.
    • Consider keeping only "live" records in the main table. Retire old records to another table, and restore them to the active table when necessary.

Home page history

All course materials are online on this site, use the navigation box at the left!

The home page history contains all the news items that appeared on the home page during the course.

Visiting?

• Check out the student projects.
• Look at the overall course schedule and download lecture materials there.
• Learn yourself: try the projects on your own!

Thanks!

A big thank you to Amazon Web Services for making generous credits for the Simple Storage Service and the Elastic Compute Cloud available for the course projects!