i think i would rather build a binary for each platform and run as fast as possible than to build once and run slow on all platforms.
But Java bytecode execution is definitely not slow and usually within 2x the execution time of a C implementation, which is faster than the vast majority of other language implementations.
[1] At least in the language benchmarks game. Yes, I know that microbenchmarks are not representative, etc. etc. etc. (Until your favorite language is faster ;).)