Scaling Rails
Last Wednesday we hosted a Newzwag classroom for a group of 25 Ruby developers who came to eat delicious pizza, drink beer, and learn how we scaled Rails. I personally had a lot of fun putting together the presentation and hosting the guys that showed up.
I presented an overview talk about database gotchas, and how to load test a session-based app. Daniel talked about his unique implementation of memcached as a temporary data store and bulk write mechanism.
The presentations were informed by all of our hands-on experience adapting our write-heavy quiz from it's capacity of 200 simultaneous users all the way up to 30,000 simultaneous users (a 25x efficiency upgrade!). Rails does scale, and just like any other app, requires custom software and hardware solutions to match a project's needs. Throwing hardware at it is the first solution, but analyzing each and every bottleneck all adds up.
We fully intended to record the whole event, but alas we were video camera-challenged... sorry!
Here's a link to the talks on Slideshare, click to download the pdf file.
Links from the talks:
- Active Record Extensions
- Detailed DB Logging
- Ebb (lightweight web server)
- God (Monitoring)
- Gruff Graphs
- Jmeter
- Magic Multi Connections
- Seattle.rb's Memcache Client
- Thin (lightweight web server)
If you have any questions, post 'em up in the comments.
Quick question about slide #31 ("What was the problem?"), you mention posts from older browsers and then say your solution was switching from nginx to lighttpd. But what was the underlying problem and what was lacking in nginx? Does it have to do with the simplicity of the proxying algorithm in nginx? HAProxy can help here.
Cody Caughlan on November 27, 2008 7:07 AM
@Cody - Thanks for the HAProxy tip. The problem was related to how nginx passed the post request through to our backend (ebb), although I didn't evaluate exactly how the request was clobbered. The ebb process was effectively killed immediately by the malformed requests, which is quite serious. Bugs in code are one thing, but when it's a critical part of the server stack, it's crippling (and was less easy for us to debug and find).
Josh Schwartzman on December 1, 2008 2:53 PM