Edit:
This page explains the high memory usage and it seems more reasonable now: https://github.com/jruby/jruby/wiki/Troubleshooting-Memory-U...
http://www.techempower.com/benchmarks/#section=data-r8&hw=i7...
- The client connections are run without concurrency, serially, on the same machine: https://github.com/brianhempel/isrubyfastyet/blob/master/run...
- The JIT is only given 3 seconds to warm up: https://github.com/brianhempel/isrubyfastyet/blob/master/run... -- HotSpot kicks ass, but not after only 3 seconds of serially issued requests.
- By running the tests single-threaded, they're throwing away one of the major wins of the JVM -- low overhead concurrency.
Well, mutable state on the models seems pretty fair, unless I'm misunderstanding you. It's rather unforgivable on ApplicationController though.
Edit: and the slow startup time doing TDD with JRuby was my primary complaint when I developed in it. Although iirc you can use nailgun or something to keep the JVM active and mitigate that pain.
Alternatively, Nailgun should reduce a lot of that pain. Effectively you have a running server instance of JVM and run some clients on it. That gives you JVM speed advantages without the slow startup. JRuby natively supports it: https://github.com/jruby/jruby/wiki/JRubyWithNailgun