By BoLOBOOLNE payday loans

Problems Scaling Ruby to Complex Systems

I’m pretty annoyed with Ruby right now.  At least I feel that way.  Looking a little deeper I realize the source of the annoyance is, like usual, my own shortcomings.  My friends and I embarked on a software project a while back.  I helped talked the group into using Ruby on Rails as the framework over choices like Java or .net because I was excited about it.  Many had reservations.  Today I’m annoyed at myself for not listening to them more.

The biggest problem with an uncompiled language is that there’s no compiler to tell you when you’ve screwed something up.  The incredible power and flexibility you get from Ruby’s loose dynamic typing and mixin inheritance style means that the IDE and compiler really have no idea what’s valid when you type it.  Compare this to Eclipse for Java or Visual Studio for .net where once you type ‘objectname-dot’ there’s a list of valid methods you can call and what kinds of parameters they take.  If you get it wrong, there’s a red squiggly underline saying something is wrong before you’re on to the next line of code. 

Yesterday 3 reasonably good software engineers took a solid 3 hours to figure out that we were passing the wrong class of argument into a method.  The problem was exacerbated by the behind-the-scenes magic that Rails does to try to make your life easier.  We were passing a Tmail object into ActionMailer.receive, which might seem to make sense since the receive method that everybody who uses ActionMailer writes expects a Tmail object as an input.  But we had forgotten that you’re not supposed to call this method.  In fact ActionMailer makes it impossible to directly call the method we wrote.  Instead you’re forced to call the class method, which we had forgotten expects a string as input that it parses into a Tmail object for you.  And like usual, the error message is useless.  Bad error messages are the single biggest flaw with Ruby on Rails IMHO.  (Maybe just the easiest to fix compared to the other problems.)  This was particularly frustrating because we spent an entire day tracking down something that any strongly typed language would have caught instantly.

For a long time, I’ve argued that people should use the highest level programming language they can afford to.  A huge advantage of pointerless languages is that you can’t make pointer mistakes.  It’s simply not possible to write that kind of bug in the higher level language.  Coding therefore become faster and more reliable than in a lower-level language.  The only cost is run-time performance, and if you’re writing server code as most of us do these days, then scalability is generally not limited by performance.  I had assumed that this analogy would continue from managed-code languages up to the hottest new scripting language that seems like it’s flexibility might fulfill OO’s promise of code re-use.  Today I’ve changed my mind.  Ruby is not a higher level language than C# or Java. 

Conceivably, a really good IDE could make up for a lot of this.  But due to the run-time binding in Ruby it would never be perfect.  When selecting a development system, generally I think the language itself is really unimportant compared to the quality of the tools and libraries available.  OO languages are generally about the same.  But IDE’s matter so much.  And libraries determine how much you have to write yourself vs. using what others have done before you.

A friend asked me if I was writing this as a warning or a cry for help.  It’s both.  It’s a warning about the scalability limits of Ruby.  Many people intuitively suspect that Ruby on Rails won’t scale well, but they confuse the different types of scalability.  The most common complaint about Ruby is that its runtime performance is too slow to scale well.  As Cal Henderson’s wonderful book on website scalability points out, raw performance does not matter if you can add more servers, which you can with RoR.  But as I learned yesterday, Ruby on Rails does not scale well in terms of complexity.  A friend once aptly pointed out that 43 things was the largest site anybody had managed to build in RoR to date.  The current state of tools and libraries and aspects of the language itself make it extremely difficult to write and maintain large complex projects with many developers.

It’s also a cry for help, but not for our particular project.  You’ll see it soon enough.  It’s a request that if you’re working on Ruby or Rails, please invest in the tools and the robustness of the libraries.  The error messages in Rails are total crap, as I’ve mentioned a couple times before.  And the IDE’s really need to improve if the framework is going to see major use beyond hobbyists.

  1. Nathan says:

    Sorry, as a follow-up to my last comment, the hosting servers I'm referring to here were for the management interface only – those 9 servers did not actually handle the hosted sites.

  2. Nathan says:

    Good post – I've encountered the exact same problems you're having. The "add more servers" answer is really stupid, too, given the number and cost of servers involved. I've worked on a large on-line payment platform written in Java which used exactly ONE server to cater to over 60 countries, but a hosting company I've worked for requires 9 servers running RoR to handle Canada and the USA.

    As for the question of tools – there will really never be an instance where an IDE can really help you. Part of the problem is that so many libraries, tools and framework rely on "method_missing" to handle functionality. It would be challenging if not down-right impossible for an IDE to fathom what would be an acceptable "method" name under these circumstances (as a point of reference, ActiveRecord makes heavy use of "method_missing")

  3. big says:

    I found another post which is worth looking at that discusses scaling the logger facility, much like twitter does it:

    http://www.dotrb.com/2007/8/11/scaling-rails-with-sysloglogger

  4. Owen says:

    I have to point out to most RUBY fanatics that even rubyonrails.com, 37signals.com, bascamp.com and backpack.com all run PHP on their frontpage just to be able to handle the bandwidth!! This has been KNOWN for a long time and the RUBY guys still don't move off PHP. Of course excuses are made and no one believes it, but it is a huge issue. It is a known fact that RUBY does not scale and 43things required 4 times the number of machines to get the job done and their migration was a nightmare.

  5. leodirac says:

    Back in 1996 I spent a long time working on a pretty large (tens of thousands of lines of code) perl system for generating dynamic web sites. It predated JSP, ASP and all that, but had much of the same functionality — sessions, users, security, database access — all built by hand. Yes, it is possible. But would anybody sane embark on such a project in perl today? I doubt it.

    Whether the language is compiled or interpreted I think is really immaterial. The issue is how much your IDE can help you. If your language is statically typed, your IDE should be able to do a lot in terms of figuring out what will work at coding time. If your language is dynamically typed, then it's not so obvious. But depending on how you write your code, your IDE might still be able to help a fair bit. If your framework creates many of the methods you use on the fly by doing string manipulation on the method name, like Rails does, your IDE is hosed. It won't be able to help at all.

    A side effect of this coding style is that even basic syntax errors won't get caught at compile time — in fact at run time the stack traces typically include 5-10 lines after your bad line of code where Rails is trying to figure out what you might have meant.

    All this points to a system where developer productivity is fabulously high when you get started, but is severely limited by the tools once things start to go wrong as they inevitably will with a big project.

    You're right — the language isn't the limitation. It's the coding style that is adopted/encouraged by the framework. It's totally possible to write maintainable code in perl or smalltalk or ruby. I've never worked with smalltalk, but I know almost nobody who writes maintainable perl, and I'm having my doubts about ruby especially on rails.

  6. Jason Dufair says:

    I'd be reluctant to group uncompiled or interpreted languages all in one bucket and suggest they don't scale. Perhaps you aren't saying that, but it was the impression I got. We have a large (10s of thousands of objects, dozens of servers, 10s of thousands of users) system in use at Purdue University, where I work, and it's done in Smalltalk. It scales very well, both in the traditional sense and in the sense of complexity. I am under the impression Ruby is pure OO, like Smalltalk, so perhaps it's an architecture thing rather than a language thing.

  7. Eric says:

    What was the useless error that you got?

  1. […] blogged about┬ámy experiences with rails and in particular my conclusion that at the time┬áRails was not ready for large, complex projects, partly because of a lack of good tools, libraries and sensible error messages, all of which can be […]