Truffle uses Graal, a new JIT compiler for the JVM written in Java. As it's written in Java, it can be used like a library - directly calling methods to compile exactly what you want, exactly how you want. Truffle uses this to compile your Ruby program using partial evaluation, a technique which is really good at removing the kind of overhead that the JVM usually adds, like costs for allocation.
This blog post and video explain where a lot of the big speedups come from.
http://chrisseaton.com/rubytruffle/deoptimizing/
I wrote my PhD about this topic, so there is even a whole thesis if you really want to get the details on how it works.