JRuby 1.7.0. Released | Hacker News Reader

JRuby 1.7.0. Released

jruby.org

124 pointssheff50 comments

> InvokeDynamic support

This is the big one, in my opinion. See here[1] for some benchmarks of JRuby + invokedynamic. We're talking orders of magnitude for some benchmarks.

[1] http://blog.headius.com/2011/08/invokedynamic-in-jruby-const...

some benchmarks from Wed, 10 Oct 2012

jruby-1.7.0.RC2 -Xcompile.invokedynamic=true

http://shootout.alioth.debian.org/u32/jruby.php

I guess I should have specified, I was thinking of the benchmarks vs non-invokedynamic JRuby. No intention of starting a benchmark war. Ruby 1.9 is fast too.

OK. Back to August 29th, JRuby 1.7.0 preview2, invoke-dynamic=true on the right, the last number is elapsed time in seconds (so 451.046 vs 258.643 with invoke-dynamic=true) --

http://anonscm.debian.org/viewvc/shootout/shootout/website/w...

I see that ViewVC link seems to have problems showing the diff at the moment, here are elapsed seconds (without invokedynamic and with invokedynamic)

jruby 1.7.0 (1.9.3p203) 2012-10-22 ff1ebbe on Java HotSpot(TM) Server VM 1.7.0_09-b05 [linux-i386]

    invoke-dynamic=	false	true
    binarytrees #1	290.08	252.55
    binarytrees #2	290.34	252.92
    binarytreesredux #2	289.64	262.04
    chameneosredux 31	39.85	32.73
    fannkuchredux #2	1720.70	1481.23
    fasta #6		372.59	418.07
    fasta #5		294.54	270.02
    fasta #1		278.98	260.14
    knucleotide #2	428.75	377.28
    knucleotide #1	470.40	461.46
    mandelbrot #3	1404.58	1149.08
    mandelbrot #1	2301.81	2167.50
    meteor #1		17.59	17.42
    meteor #2		13.19	13.37
    nbody #1		678.62	568.72
    pidigits #2		3.10	3.22
    pidigits #3		25.89	25.73
    pidigits #1		50.80	51.03
    regexdna #1		78.76	78.60
    regexdna #3		44.29	45.33
    revcomp #2		24.05	24.49
    spectralnorm #2	462.09	387.54
    spectralnorm #1	496.94	438.55

YMMV

So with about the same CPU time, JRuby will use up to 100x memory in worst case?

Anyway Ruby 2.0 is suppose to solve the Ruby is Slow problem. although i have yet to see any evidence that will the case. Sigh.

these benchmarks also include the time it takes the vm to start up and may not include full JIT performance. locally in jruby pidigits.rb 2000 takes 4.132s, running it twice it takes 4.965s. jruby is quite annoying for running small little scripts because of the startup time but it an be quite nice for full applications.

Yes, each Ruby method needs to get executed 50 times to JIT to JVM bytecode, and from there it takes an additional 10,000 calls for HotSpot to JIT that to machine code.

Any hotspots therefore need to be hit at least 10,050 times in order to be fully warmed up.

Just start the JVM with -XX:CompileThreshold=<some_number>

It only defaults to 10,000 when the JVM is running in server mode, its 1,500 when in client. Which mode a default install runs in is a bit wonky so I always explicitly set it (with -server as an argument to the JVM).

>> jruby pidigits.rb 2000 takes 4.132s <<

Why did you measure a workload 1/5th of the workload shown on the benchmarks game webpages?

Each program page shows measurements for 3 workloads, in the case of pi-digits 2,000 6,000 and 10,000.

http://shootout.alioth.debian.org/u32/program.php?test=pidig...

Possibly not. Those benchmarks (mandelbrot and n-body, pi-digits) are very low memory footprint benchmarks. So it might be a difference of going from 20K to 30meg constant memory usage.

Whereas the other benchmarks seem to have single digit multiples of memory usage. SO seemingly real world apps will use more memory, but not 100x in practice.

Also note that for Fixnum dependent benchmarks, JRuby has to allocate memory due to boxed integers, whereas on the de facto interpreter it's able to use tagged pointers which allow Fixnums to be treated as simple numeric values.

This may be going away in JDK9 as they claim they're eliminating primitive types in favor of an everything-is-an-object model.

> So with about the same CPU time, JRuby will use up to 100x memory in worst case?

The JVM often trades increased memory usage for increased performance. That said "100x memory" is total hyperbole, even in the context of a microbenchmark. If you want the JVM to use less RAM, you can turn down the maximum heap size. Otherwise the heap will grow until it reaches close to full before the GC does a stop-the-world style collection of the old generation.

> Anyway Ruby 2.0 is suppose to solve the Ruby is Slow problem

Absolutely not. There are presently no plans to add a JIT compiler to the de facto Ruby interpreter.

The JVM on the other hand has the most advanced JIT compiler in the world. InvokeDynamic on Java 7 allows JRuby to do method inlining, and on Java 8 it will be able to take advantage of the full suite of HotSpot optimizations, including escape analysis and lock elision when compiling Ruby code into machine code.

No other Ruby VM will have that level of optimization.

>> If you want the JVM to use less RAM, you can turn down the maximum heap size. <<

That's how the programs used to be run with JRuby 1.6.7 and with those same limits the n-body program failed to complete within 1 hour using JRuby 1.7 -- so the brittle RAM limits were abandoned.

> Otherwise the heap will grow until it reaches close to full before the GC does a stop-the-world style collection of the old generation.

Most real applications will be using ConcMarkSweepGC(or G1) and ParNewGC. With these collectors the JVM won't just burn through all the memory and halt the world unless you throw a workload at it which the GC can't handle with its normal GC cycles.

The GC makes small sweeps regularly but the duration of those sweeps is limited to ensure relatively consistent response time. If you were creating and throwing away so many objects which were promoted to the tenured generation that these small, time limited, GC sweeps couldn't keep up, then you would see the heap grow to its maximum and a 'stop the world' GC would take place. In practice that doesn't happen in the vast majority of workloads.

No, you're wrong, an extremely scant number of applications can completely avoid allocating memory in the old generation. Your best bet would probably be to use the Disruptor approach of preallocating everything then avoiding any subsequent allocations if that's really what you want to accomplish. However, if you honestly believe "most real world applications" can avoid allocating memory in the old generation, I'd sure love to see some graphs of some heap profiles of real world usage of these applications!

CMS does a stop the world collection of the old generation, however it will do an incremental collection of the eden generation. If you've ever looked at a graph of the JVM heap observing CMS running under VisualVM or YourKit, you'll notice a distinct sawtooth pattern to its collections. The diagonal parts of the sawtooth represent incremental collections in eden space. The vertical parts of the sawtooth represent stop-the-world collections of the old generation.

The "small regular sweeps" you're talking about affect the eden generation, not the old generation. Collecting the old generation is a stop the world operation in every JVM GC option except Azul's C4 collector. C4 resorts to all sorts of clever tricks to avoid stopping the world, most notably read barriers and eager pointer updates. On the Azul hardware, these tricks involved hardware transactional memory. On x86 they don't have HTM (at least until Transactional Synchronization Extensions ship on Haswell). What they do, however, is quickly set up GC invariants and a trap on a given page. Whatever thread accesses a page that contains an object which needs to be relocated relocates the object on the GC's behalf and gets the new pointer. All that said, C4 is only available on the Azul JVM, which is commercial.

CMS can't do this. It stops the world at a JVM safe point whenever it needs to collect the old generation.

CMS collector has two different types of collections it will do on the old generation. Both are "stop the world" but for drastically different amounts of time. During its 'normal' operation most of the sweep is done concurrently with two short "stop the world" points for each GC cycle. If it can't keep up, then it will do a full blow "stop the world for real" collection.

Ruby 1.9.? There is a significant difference between 1.9.1 and 1.9.3

http://shootout.alioth.debian.org/u32/ruby.php#about

http://shootout.alioth.debian.org/u32/program.php?test=nbody...

or ...

If you're confused about what InvokeDynamic is...

Previously languages like Ruby suffered on the JVM as the set of JVM instructions available were designed around the Java programming language. This meant that for invoking Ruby methods and other things like accessing instance variables could not take advantage of many of the JVM's features such as method inlining.

Since HotSpot is unaware of Ruby semantics, invoking Ruby methods involved a bunch of custom logic implemented in the form of bytecodes which were completely opaque to HotSpot and thus could not be optimized away.

InvokeDynamic allows JRuby to teach the JVM how to invoke Ruby methods in a way that plays friendly with the HotSpot inliner. Here's a contrived example of HotSpot inlining a Ruby method and optimizing it down to a single assembly instruction:

https://gist.github.com/3728777

I will be happy if someone corrects me but JDK 7 has some bugs with InvokedDynamics and it is turned off by default. These bugs have been fixed in JDK 8 (beta so far).

In the future, even if JRuby team does only bug fixes, JRuby will get faster and faster as JVM's support for dynamic languages improves through InvokedDynamics.

And can be turned on like this:

    /usr/local/src/jruby-1.7.0/bin/jruby -Xcompile.invokedynamic=true

Biggest things stopping me from using jruby:

1. Incomplete integration with postgresql. The ActiveRecord jdbc adapater doesn't have all the features that the standard pg adapter does.

2. Slow startup / test runs. At one point, loading large amounts of files (like during rails bootup) was much slower on jruby than MRI.

> Incomplete integration with postgresql.

I'm planning on using it with postgresql - do you have any links to whats not supported with the JDBC drivers ?

One of the issues I ran into when looking to migrate a multi-tenant Rails app to Torquebox was lack of schema support (eg. schema_search_path). I was able to workaround those issues using raw sql though.

The JDBC driver doesn't support EXPLAIN or COPY

I use it with Heroku's pg service via a JDBC driver, and I've never had a problem. Do you have any links? Can you elaborate?

Arrays, hstore, domains, etc. Not supported by the activerecord-jdbc-postgresql adapter.

> Slow startup / test runs. At one point, loading large amounts of files (like during rails bootup) was much slower on jruby than MRI.

You can develop on MRI and deploy on JRuby. Several people, including myself and Square, develop JRuby applications this way, and validate stability across multiple Rubies by doing matrix builds on a CI system.

Check out Xavier Shay's talk on the matter:

http://www.youtube.com/watch?v=hMpd4CzR1f8&t=16m

I know several people balk at this suggestion and claim development should be as close as possible to production... and do that despite the fact they develop on OS X and deploy on Linux, develop with a case insensitive filesystem and deploy to a case sensitive filesystem, develop in an environment without load and deploy to an environment with load, etc...

There is good reason to use the same environment. No one wants to be dealing with wacky version issues on remote servers, which is precisely what you would be signing up for.

If you are smart, you'll run CI on JRuby. Then your test environment matches production.

Then I assume your development workstation is an exact duplicate of a production server, and is serving a production load?

Even getting a staging environment to realistically resemble production is an incredibly difficult undertaking. Your development environment is a joke, sorry. If you're expecting it to resemble production in any way, I don't know what else to say except you're extremely confused about how a production environment actually works, or your production environment is a pathetic joke which serves a nonexistent userbase.

Based on this comment, I started looking into JRuby support for the pg gem. I think it's possible; I have started stubbing it out, and the Postgresl JDBC driver publishes additional interfaces that provide everything folks here have been asking for.

I'm looking for help filling it out. Here's the mostly stubbed extension:

https://github.com/headius/jruby-pg

And here's the docs for pgsql's hidden interfaces (cast the JDBC objects to these, and you're good to go):

http://jdbc.postgresql.org/documentation/publicapi/index.htm...

Great news, would be a welcome addition to JRuby!

Interested to do some testing as it comes together and see what I can help with.

This would be brilliant, as Postgres is one of the most commonly used databases with Rails (generally, and also because of Heroku) so having feature parity between the pg gem and JDBC drivers that you can use with, for example, Torquebox would be great, especially for people wanting to migrate to JRuby.

I did some basic benchmarking with one of the JRuby 1.7 preview releases and was impressed enough with the speed benefits to want to try it out with a small app on Torquebox when JRuby 1.7 was released.

Thanks to joevandyk and xiljin for pointing out the search_path and datatype support situation before I undoubtedly encountered them myself - and love the fact that headius has immediately leapt in to try and fix things :-)

May be it would be better to reimplement protocol, like postgres-pr does but using java ? Protocol is rather simple, and postgres-pr could be used as a reference (it implements meaningfull subset).

JRuby is really great. I use it every day for both work and personal projects.

Big thanks to all the contributors!

Looking forward to installing it and trying it out with Torquebox ( http://torquebox.org/ ) for a new project.

Not only does it default to 1.9 after one year, it catches up all the way to 1.9.3-p286, released only ten days ago.

Congratulations to all devs, committers, documentation maintainers, supporters, donators, secret lovers and overall alternative ruby enthusiasts.

Is JRuby worth using if JVM language integration isn't needed? E.g. A fairly standard Rails app.

I'd use jruby if you were going to use something like http://torquebox.org/

It's hard to say from the description of "fairly standard". Libraries with C extensions may not work with JRuby. A few Rails deployment options aren't available with JRuby, and the ones that do support it may treat it like a second-class citizen.

In short, there's potential downsides that may or may not be relevant to your usage. That said, even if you aren't using the Java ecosystem, the garbage collection, performance, and threading in JRuby may be worth your time. It's all up to what you're actually doing.

A lot of companies have operational requirements that make running "a fairly standard rails app" possible in their production environments with managed JVM apps.

My experience is that if you don't need JRuby (for Java interop or to satisfy pointy-haired strictures), you probably won't see much benefit from choosing it, at least for a Rails app.

On one hand, JRuby is technically a much better Ruby than Ruby, and if you want to improve bulk throughput you'll probably benefit. On the other, if bulk throughput were important, Ruby probably wouldn't be your first choice, and there are some downsides.

The biggest problem, especially for Rails, is the warm-up cost. Rails's start-up time is already painful, and adding to that hardly improves the test-develop cycle. I've not tested this latest release, but in 1.6.8, running a test suite was multiples slower than C Ruby. Test suites, which run a lot of different code, aren't really representative of real-world workloads, but if you're working on an application it really hurts.

You might also have trouble with gems that have compiled extensions. It takes a bit of Gemfile fiddling, but there are alternatives to most of those.

For applications other than Rails (which is really a bit of a pathological case), JRuby seems well worth a look. I've never found C Ruby's threading up to the job of making a responsive GUI application, but JRuby might work a lot better with its JVM-backed threads.

It's a great choice if your application needs real concurrency. JRuby does not have global interpreter lock like YARV, so threads can take advantage of multiple CPUs.

cool. "rvm install jruby-1.7.0" works :-)

I initially got an "Error: Your application used more stack memory than the safety cap" message running that.

This solved it: https://github.com/jruby/jruby/issues/346

tl;dr:

    rvm get stable

then reinstall jruby 1.7.0

"rvm get head; rvm install jruby" also works ;)

Best Ruby implementation so far.

Reply on news.ycombinator.com