There are quite a few open source ones, but they aren't as stable.
As always, Language != Implementation.
Excelsior JET is the most well known one.
Oracle actually also supports AOT but only on embedded systems, with the commercial JDK.
Avian is a JIT compiling JVM but one which is much smaller than HotSpot. It has a mode where it statically links your JAR into the binary itself, so you get a single self contained executable. With ProGuard and other optimisations like LZMA compression built in, such binaries can be remarkably small. Try their example:
https://readytalk.github.io/avian/#-xamples
It's a full blown GUI app that demonstrates the full range of widgets available, is cross platform, and yet is only about 1mb in size.
Usually this means iOS and the ARM processor type but RoboVM is also capable of generating code for Mac OS X and Linux running on x86 CPUs. [0]
There are several forks of the latest open-source version of RoboVM. This one in particular is targeting desktop/server usage:
https://github.com/ashleyj/aura
Language == Thought-space >= Implementation,
so you want to maximize your language's expressiveness.With static linking? I don't think so.
You can do dead code elimination and other forms of LTO using ProGuard. Just watch out for code that uses reflection. Java 9 will include a less aggressive version of the same thing which performs various link time optimisations like deleting modules (rather than individual methods/fields), statically pre-computing various tables, and converting from JAR into a more optimised (but platform specific) format.
That tool can also bundle a JRE in with your app, giving you an "tar xzvf and run" deployment model. It's not a single file, but it makes little difference in practice. The same tool can build DEBs, RPMs, Mac DMGs and Windows EXE/MSI installers with a bundled and stripped JRE too.
Also, I don't think there's much of a case to be made that bundling a JRE with your JAR is small, even though the tooling might be simple and it might resolve many deployment issues.