Speeding up Rails startup time | Hacker News Reader

Speeding up Rails startup time

rhnh.net

152 pointsddagradi31 comments

Couple things:

* Wow, an O(n^2) require algo. I guess this really drives home that we shouldn't make basic assumptions about the quality of the libs we use without actually looking at them. Great catch.

* For all of you waiting for this to be pulled into Ruby master. Why not just patch your copy in the meantime?

The patch isn't done yet, there's still significant bugs to fix.

As of now, there are none outstanding that I know of.

Wow, amazing find. I've been struggling with this issue for many months, and I just figured it was rails 3.0 fault for being so big and bloated. Now we find out it's ruby implementation. I really hope this patch makes it in.

I've patched my local copy. Feels like I have doubled my computer speed! Seriously though, I can't believe the fix was so easy, and the ruby core hasn't done something about this yet.

The last major external fix to ruby was with the REE people, and that still hasn't made it back into MRI. So I'm a bit worried this patch will get rejected and the problem won't be fixed.

Rails does have problems with 3.0 being slow.

See http://www.youtube.com/watch?v=kWOAHIpmLAI&t=34m9s for a recent talk on it by @tenderlove.

tenderlove is going to follow up on this. The issue is for metal-like responders, not for full-fledged apps. We've lost a couple of ms on the low-end due to the stack issue.

As a followup, while this patch works great, using it forces you onto ruby-head which breaks a ton of shit.

I'm back to shit rails bootup speeds for now. But at least I now have hope that it's possible that a fix may be on the horizon.

joevandyk posted a great example project here: https://github.com/joevandyk/slow-rails

20second bootup of an empty project = hair on fire omgwtf issue. the start of phpenvy? (no, not really)

I've always thought it would be nice if you could cause Ruby to dump its heap and symbol table to a file, which could then be loaded by other instances of Ruby. So, for example, you could run Ruby, load all of the Rails files, and then save everything, so the next time you want to run Rails, you just have to load one file instead of hitting the file system 2000 times.

You could probably also do this by running a Rails instance as a service, and then every time you want to run Rails, you tell the service to fork, and use the forked thread. I think this is how Passenger works actually?

You could probably also do this by running a Rails instance as a service, and then every time you want to run Rails, you tell the service to fork, and use the forked thread.

FWIW, this is how Spork works, a tool people have been using a lot more recently in order to have fast(er) tests on Rails 3. Spork boots up an environment then waits for RSpec (or whatever) to hit it over DRb and then it forks off for that run.

I've always thought it would be nice if you could cause Ruby to dump its heap and symbol table to a file...

Another full circle back to Lisp. Dumping a Lisp image file is much like you describe. (Just sayin'.)

This reminds me of an old joke:

One day in the early days of computing, General Electric had a problem with their computer. All of their engineers took a look at the problem. Although each was wise, they were unable to understand the complexity of the machinery and repair the error. A call was made to the retired engineer who had helped in the original set up of the machine.

The retired engineer walked around the machine for a few minutes, just looking it over, not touching anything. After a few minutes, He took out a piece of chalk, walked over and placed a large X on one particular part of the machine. He then said' "Tap it here with a hammer, just once."

After the one tap, the computer roared back to life and began working!

A few days later, GE received an invoice from the retired engineer for $10,000! This was a lot of money in those days, so they returned it to the engineer and asked that he itemize his invoice.

A few days later, they received an itemized bill which read:

Chalk for one X mark - $1.00

Knowing where to place the X - $9,999.00

Given the roots in Perl, which has the %INC hash, it's surprising that Ruby started with an array for loaded modules. Can anyone shed some light on this design decision? Surely there was a reason at some point -- and maybe there still is.

Makes me wonder how many more "simple" optimizations can be made to the Rails stack. Very exciting!

If you're not using Bundler and have a simple dependency chain, you can get further improvement from symlinking your gems into a single directory (see https://gist.github.com/975509 ).

This patch certainly improved my load time tremendously, but the core of the problem still lies with the way rubygems and bundler dump all directories of gems in the load path. The promise of $LOAD_PATH is that all the directories in it will be tried -- the most bang-for-buck optimization is thus minimizing its size.

rpg works by installing everything on a common directory: https://github.com/rtomayko/rpg

If rubygems did this, everything would be much better.

There is one problem though, and it is that some libs are bad citizens and install stuff outside of lib|bin and depend on it (haml's VERSION file is an example of this), and that sucks.

rpg has a 'shitlist' with fixes for specific cases: https://github.com/rtomayko/rpg/blob/master/rpg-shit-list.sh...

This patch seems to introduce a bug when loading files with extra periods in the name (I think).

My app which includes thinking-sphinx (which has a require '0.9.9'), breaks with it.

This has been fixed and the post updated.

Yep, I saw, thanks!

Does this problem exist in jruby and rubinius as well?

jruby big yes and they're aware of it and this fix. rubinius small yes, possibly a maybe (haven't investigated properly).

Rubinius uses the exact same technique as is detailed here, namely backing an Array with a Hash (LookupTable in the Rubinius case). We added this a few years ago to speed up require.

on the topic of rails, has anyone noticed images loading slowly on localhost with 3.1?

I haven't tested this myself, but I can guess that they will load slower in development. In development asset requests go through the asset pipeline now instead of being directly served up through the HTTP server.

is there a reason to dump your images into the assets vs public folder then?

Not really for application specific images. However, vendor, lib, and engine images will be copied into public in production. Keeps you from manually having to keep somebody elses asset dependencies updated in your app.

Wish I could vote for this to be in 1.9.3 as easily as I can vote for it on HN.

I think it has to be. The startup performance of 1.9.2 and 1.9.3 as is now is unacceptable.

Sorry for the snide comment, but if you want to really speed up your Rails, switch to Go, it will compile from scratch and start up your project faster than the ruby interpreter starts up.

I used rails in a few projects, and performance was so painful even on pretty good hardware that I said never again. Go on the other hand is pure pleasure.

This is a bit like saying, "If you want to get better gas mileage out of your car, walk." Switching to Go would be throwing the baby out with the bathwater. If you like Go, that's good, but people who are using Ruby and Rails presumably like those technologies. It's unlikely that somebody would choose Rails based solely on a mistaken assumption that it has the fastest possible runtime.

Reply on news.ycombinator.com