I would take the memory measurement with a grain of salt. A 200 MiB large JVM doesn't mean all the 200 MiB are used. They could have been reserved (preemptive memory allocation).
You might want to clarify that statement a bit. It almost sounds like you are implying that memory pressure is never an issue for long running web applications in java. Did you mean to say something else?
As for the startup time, MojoJolo summed it up.
That's ok. It's not a revelation to anybody here (I hope) the enormous cost in memory overhead you have to pay for acceptable performance from the JVM. That "top quality GC" basically requires a 2X overhead (on top of your actual cache payload) to perform with reliable low latency and high throughput.
The JVM (at least OpenJDK, probably not Azul) is quickly becoming untenable as server memory goes into the hundereds of GBs. I'm reluctantly moving back to C++ for that reason alone.
As a possible work around to the JVM limit, a distributed architecture with N JVMs running a portion of the task could solve the small memory space with minimal system overhead. What I mean by this let's say you need to have 64 GB of memory for your app. Given the comment above, Java would not do well with this. But you could have 4 16 GB VMs each handling 1/4 of the work. The GC would prevent fragmentation that you'd see in long running C++ apps and still provide you with operational capacity.
Many applications can distribute work among multiple processes because they don't need access to shared data or can use a database for that purpose. But for what I'm doing (in-memory analytics) that's not an option.
Another question about in-memory analytics is do you have to be in-memory? I'm currently working on an analytics project using Hadoop. With the help of Cascading [3] we're able to abstract the MR paradigm a lot. As a result we're doing analytics across 50 TB of data everyday once you count workspace data duplication.
1 - https://en.wikipedia.org/wiki/Tuple_space 2 - http://river.apache.org/index.html 3 - http://cascading.org
The obvious alternative would be to go with a traditional relational database, but my thinking is that the dataset is small enough to do everything in memory and avoid all serialization/copying to/from a database, cache or message queue. Tuple Spaces, as I understand it, is basically a hybrid of all those things.
Regardless of whether it is C, C++, or a JVM language you are going to be reusing data structures, directly accessing memory, and in the case of JVM systems using off heap memory. If you are doing this correctly your JVM can be quite small and never GC (or more usually, GC right after initialization/configuration).