1) The long load time / hotspot JIT overhead when starting applications.
2) The large memory overhead of the Sun/Oracle GC's reserved heap.
To some degree, Dalvik solved some of these issues, but has also retained the (very necessary) escape-hatch-to-native JNI.
JavaScript is SO MUCH HARDER to optimize than JVM bytecode. Whereas you can even feasible AOT-compile JVM code, JavaScript's dynamism makes this all but impossible.
On top of that, JS is welded to the DOM and CSS, the combination of which incurs huge rendering pipeline CPU and memory overhead for anyone doing anything as complicated as a "native" UI view hierarchy.
Rather than enumerate all the failures, I'll note that it's not all doom and gloom. We've got a web bytecode in the works that's much saner than JS. Canvas and WebGL exist.
Progress on those fronts may create a situation like you're describing, but -- and this is no great loss in my book -- it won't be the web.