Chronicling My Ruby on Rails Journey

January 28, 2008

Making Twitter 10000 Percent Faster

Filed under: ruby rails,scalability,twitter,xmpp — Bob Ngu @ 5:19 pm
Tags:

Found a couple of other interesting links to this topic. One is Scaling Twitter: Making Twitter 10000 Percent Faster. It is from a site called High Scalability dedicated to describing (in high level) various scalable architectures like MySpace, Plentyoffish, eBay, Google, Flickr, etc. Definitely worth reading if you are interested in exploring the various architecture designs employed by these different high traffic sites.

The most interesting comment on scaling Twitter is this

Twitter started as a side project and blew up fast, going from 0 to millions of page views within a few terrifying months. Early design decisions that worked well in the small melted under the crush of new users chirping tweets to all their friends. Web darling Ruby on Rails was fingered early for the scaling problems, but Blaine Cook, Twitter’s lead architect, held Ruby blameless:

For us, it’s really about scaling horizontally – to that end, Rails and Ruby haven’t been stumbling blocks, compared to any other language or framework. The performance boosts associated with a “faster” language would give us a 10-20% improvement, but thanks to architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster than it was in January.

This is different than what Alex Payne, another developer at Twitter, said when he was interviewed here. Subsequent to Alex’s interview, Rails has been blamed incessantly for not being able to scale. Even open source drama follows mainstream media drama, i.e., take the sound bite and run with it, never mind the details.

It is as I suspected all along, the language / framework are all but bit players in the grand scheme of scalable architecture. IMO this also explains why other scripting languages like Perl, PHP, VBScript, Python, etc are not front and center when it comes to scalability. Don’t get me wrong, I think Ruby performance can be improved but the real challenges lie elsewhere in the backend. I guess all the initial hype about RoR productivity led to exuberant expectations and the inevitable crash. Truth be told, the scaffolding bits that allow you to put up a web page in 5 mins is useless when it comes to real development. Once the hood is lifted, the real work begins. Sites developed with other scripting languages/framework went through the same growing pains but without the glitz and media attention that RoR garnered. I believe J2EE went through the same growing pains early on in their adoption. These are expected growing pains, nothing more, nothing less.

Another interesting article is Could Instant Messaging (XMPP) Power the Future of Online Communication?. It talks about using push instead of pull. This is nothing new, this design has been around for years and is being re-discovered and re-use in a new domain. Something old becomes something new.

January 26, 2008

Pondering Ruby on Rails Scalability

Filed under: ruby rails,scalability — Bob Ngu @ 9:41 am
Tags:

It has been a while since I blogged. My site, JiggyMe has been chugging along and has picked up as much traffic as 50,000 hits a month, 3000+ hits a day. I am hosting my site on Slicehost 256MB, 10GB VPS and it has held up rather well so far. However, I keep hearing how Ruby on Rails can’t scale, courtesy of the constant downtime at Twitter.

I know that Rails 2.0 has recently been released but unfortunately I haven’t been paying much attention to it. I wouldn’t mind offering my site up for scalability experiments. Rails is fairly young and I believe that as more sites encounter scalability issues that Rails will be extended to address those issues. Scalability is a complex domain and there is no one correct solution. If the limitation of Rails not being able to connect to multiple databases is the scalability problem, then I believe it is a solvable problem.

How about it, Rails developers, shall we prove the naysayers wrong? Afterall, scripting languages were never designed to scale and yet they (Perl, Python, PHP) are scaling now, it goes to prove that it is just a matter of time to work out the scalability issues.

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.