It's not about performance (nothing memcache does or more precisely -- given all new features I am likely unaware about -- when it was first rewritten in C is particularly CPU intensive), it's about memory management and efficiency: a in-memory system serves to reduce latency by using large amounts of memory; tuning the JVM to handle managed heaps larger than ~24gb (with only half to 2/3 of that usable for caching data) is doable but pretty much a black art if you want to avoid huge latency spikes caused by garbage collection pauses (which defeats the purpose of an in-memory store). Even if pauses are rare enough as not to affect 95-pctile latency (or if you don't care about 95-th pctile latency) in a highly-available system they introduce one of the more nasty distributed systems failures -- node A talks to node B and goes into a GC pause; node B thinks node A is dead, node A thinks node B is dead -- because when it didn't receive a reply to its outstanding requests until after it emerged from a GC pause triggering a timeout (all in the mean while certain figures from RDBMS community insist this didn't actually occur because "partitions never happen in a single datacenter...")

While it's very much possible to directly allocate and manage memory yourself in Java -- and I have done that ( http://mail-archives.apache.org/mod_mbox/hbase-commits/20130... ) -- but without memory safety guarantees, without the great tooling, without the general idea that "if your code compiles, it will probably work correctly". In other words, you're programming a very verbose and ugly (see DirectByteBuffer interface) C. There's many cases where this approach ("off-heap memory") makes sense when building memory intensive apps in Java that clearly benefit everywhere else from being written in Java: JNI is unpleasant to work with, JNA adds additional performance penalties on top of those imposed by JNI calls that it makes under the cover -- which is fine in many cases, but remember that reading files or sockets are also JNI calls -- but less so when you're building a system that distinguishes itself by being in memory.

On the other hand, if you use C (or "C+" a.k.a. "C with objects", or -- as I prefer to say to avoid confusion with libraries like glib or apr or the Linux vfs layer, all successful object oriented systems written in regular C -- "C with templates") you have accesses to the existing works (advanced allocators like jemalloc and tcmalloc don't work very well with JVM), excellent memory-debugging tools, and so on...

I will say this: I think more software rather than less should be written in higher level languages. I absolutely love OCaml, Erlang, and Lisps (especially those -- like Typed Racket and Clojure -- that have started importing features from Haskell/ML family), like what I see in Go. I use Python (and formerly Perl) on a daily basis for "casual programming", to experiment with new ideas, and automation. I've written a great deal of software in Java, where many people blinked the idea that this category of software could be more than a toy in Java. I think garbage collection could be great improved and I see no theoretical reasons why, e.g., compilers can't be written in OCaml as opposed to C or C++ (I do see practical reasons having to do with runtime, lack of multi-core support but that's a separate and fixable issue). I find projects aimed at making systems programming in high-level languages fascinating (e.g., Mirage OS, Microsoft's experiments, Jikes RVM, etc...)

However, even if these theoretical strides are achieved, there's always going to be room for C -- in the end, you need systems that give you great deal of control and act in a very deterministic and predictable fashion . In the mean while, however, there's also need to build practical systems -- so while C and C++ may not be ideal in theory, they are often the only realistic option in practice.

[1] Note, however, I didn't say anything about performance -- OCaml, Java, Haskell, LuaJit etc.. do well in the Debian benchmarks; Lisps follow those languages closely. Likewise, many high-level languages can be AOT compiled. While when written by a strong programmer and/or assisted by today's excellent optimizing compilers C will still beat these fast high-level languages in most cases, it should be noted that today it's extremely difficult to write assembly code by hand that beats assembly code written by an optimizing C compiler or even the JVM. I predict that when a language comes about that has a better designed type system than C/C++ and yet still provides ability to control memory layout much as C does, eventually compilers for that language will emit assembly code that beats assembly emitted by C compilers for the same reason compiler-written assembly beats hand-written assembly -- compiler is able to leverage the type information (declared or -- in the case of dynamically typed languages with fast runtimes -- inferred) to aid optimization.