>The JVM itself has a certain amount of unavoidable overhead

That is not a "certain amount of overhead" but the inherent incompatility with the modern hardware. With Java writing cache-friendly code is extremely difficult: boxing and indirections are encouraged while primitive types are cumbersome and value types are possible only through the direct byte manipulation. Memory overhead is enourmous. A simple collection like a hashmap of short strings can have up to a 75% overhead.