Most popular JVM memory configurations
plumbr.eu
plumbr.eu
All applications are different and have different memory requirements. Too low of a setting and you are constantly thrashing or running out of memory. Too large and you have huge garbage collection times. It has to be tuned. Now, I've heard (as I haven't really looked into it yet) that the JVM in the JDK 8 distribution has gotten rid of PermGen setting. So, I'm curious to know what 'they' are doing now. Is the G1 garbage collector the new default in JDK 8? Does that not have the PermGen or is this something different?
Anyway, I don't know what this post is about yet. They probably should have just finished the thing instead of saying: "If you managed to get this far, then you are most likely interested to hear about our forthcoming posts. Stay tuned by subscribing to our Twitter feed." It was worthless.
And another aspect we wished to demonstrate was the sheer size of permanent generation on some of the applications. I have yet to face a leak-free app where permgen > 256MB seems justified ...
Perhaps the tool you install should be showing the number of classes loaded and the size of the code in general. But, again what's in a jar file can't tell you how many classes will be loaded because many common API such as hibernate and spring generate thousands and thousands of dynamic proxies that end up being stored in memory for the lifetime of that application.
The data also doesn't show if the higher heap users tested out running a lower PermGen/Heap and ran out of memory, hence their increased size. I'd rather allocate too much memory though than run out of memory at a peak time and have one of my hospitals deal with downtime because I saved $25 worth of RAM.
Anyway, I'll read the followup but I hope it has more info. The key is you have to profile your application, tune it and your done.
We're thinking of giving up on scaling above 16gb/jvm and going to something crazy like 16x2gb jvms. Treat them like the worlds biggest mongrel and let the loadbalancer sort it out.
Mind emailing me your config? You should be using a CMS collector. -XX:+UseConcMarkSweepGC. Also try reducing CMSInitiatingOccupancyFraction until things stabilize. I've never had to drop it below 70%. Finally, ALWAYS make sure Xmx and Xms have the same value so the JVM doesn't try to resize itself.
That (and a truckload of trial & error) led me to this config: http://pastie.org/pastes/7130367/text?key=s8jajyejyhgz7x303s...
You're probably right, I probably could just keep dialing up xms/xmx/xmn, but even then what exactly to watch for in the gc log and how to do it tools-wise is a dark art. Exactly what field in what line to grep for and be mindful of when is a constant source of debate/confusion, and once you get up to a few hundred req/s visualgc just can't keep up so the graphs stop being useful.
As you can see from my config link there I've tried throwing money at the problem but that didn't do it either. I'm at the point where I can hire an engineer to truly master this, or go bootleg parallel.
Understanding the gc logs can be difficult. gcviewer is pretty good though at figuring it out. Always remember that garbage collection is slowest whenever the JVM has to copy objects from one generation to another or between survivor spaces. The logs print the sizes of the young gen, old gen, and survivor spaces. Sometimes after a collection you can see the sizes of these change dramatically - that means a big object has moved around. You can either change your code to use big objects less often or hold onto them for a shorter time or try to increase your survivor space size or new gen size.
A major, major, major thing you are missing is -XX:+UseParNewGC. With the CMS collector, your new gen collection can be done in parallel, and the parallelization scales very well with the number of cores. Without ParNewGC, the service I work on would take over 1 second to garbage collect. With ParNewGC, it only takes 50ms.
Also, more concretely, this is the most frequent type of entry in my gc log: http://pastie.org/pastes/7142036/text?key=dkwvlvaaysk63rcajb...
I (perhaps wrongly?) interpret the ParNew there to imply that's the young gen collector in use.
[1]: http://www.azulsystems.com/products/zing/virtual-machine
"...whether it is a 32 or 62-bit infrastructure..."
and a bit later...
"And those two who think 258MB is better – are you just bad at calculating the powers of two..."
These metrics do not really reflect the heap sizes used in production, so I would not conclude that java does not need much memory. If anything, those that use Plumbr in evaluation mode do not run their JVMs with large heaps. You do not need a large heap to cause a OOM.
Still, it is interesting to know what the Xmx settings are on these boxes.
Many Java devs, when looking for object/memory/resource leakage, will turn to the well-known general purpose tools, such as JProbe, YourKit, Introscope, or even just JVM profiling output and verbose gc. Crazy people will start using DTrace or JMX hooks. The rest will just say they need a hardware refresh.
As you say, this is an analysis of metrics gathered from people looking for a 'free' solution. The reason for the 'small' mx settings could be that developers are finding ways around procurement to find leaks on the same machine running their IDE, email, browser, etc.
Everything is an object, generics need wrapper classes, padding everywhere. Those make HashMap<Integer>, ArrayList<Integer> or even "two dimensional arrays" a bad joke compared to C(++) equivalents. Projects like Lucene have to rewrite standard library eqiuvalents for specific cases since otherwise so much performance would be lost.
Examinating app server -Xmx settings is more handwaving than anything. Especially people that refrain from doing certain things in java due to memory issues, have a completely misleading influence on such a metric.