Truffle uses Graal, a new JIT compiler for the JVM written in Java. As it's written in Java, it can be used like a library - directly calling methods to compile exactly what you want, exactly how you want. Truffle uses this to compile your Ruby program using partial evaluation, a technique which is really good at removing the kind of overhead that the JVM usually adds, like costs for allocation.

This blog post and video explain where a lot of the big speedups come from.

http://chrisseaton.com/rubytruffle/deoptimizing/

I wrote my PhD about this topic, so there is even a whole thesis if you really want to get the details on how it works.

http://chrisseaton.com/phd/

EDIT: The man himself made a post: https://news.ycombinator.com/item?id=13058481

I wish it could be summarised in a comment, but it's actually not "that simple," so I recommend you read Chris Seaton's posts about GraalVM and JRuby+Truffle which has some excellent high-level explanations [of why Graal is amazing]. :)

http://chrisseaton.com/rubytruffle/