ElasticSearch pre-flight checklist

asquera.de

64 points·Xylakant··4 comments

Good advice in general, but 50% of total isn't a good rule of thumb for the heap. On boxes with more than 16GB of RAM, this will likely be counterproductive. The exception is if you're using cached filters (facets, filter queries) heavily on a very large index (~10MM+ rows).

I must admit that this is actually a piece of information I just proxied through as a starting value (as stated, these are basically reworked notes from a training). I should make it clearer that this parameter is subject to tuning. What would be your suggestion here?

Two things are important:

1. Your OS will use free memory as a filesystem cache. Ideally, your index is being served entirely from cache. Usually, your working dataset is maybe 1/3 of the size of the index, so if your index is 6GB on disk, you'll typically want to leave 2-6GB unallocated. iostat is your friend.

2. You need enough heap. More than enough heap hits diminishing returns quickly. I wouldn't trust ES with less than 2GB, and I've never seen it need more than 8GB. GCs will be less frequent with a larger heap (even up to 30GB), but that's really the only benefit. I'd make sure your index fits in RAM before expanding the heap.

More about GC: http://www.slideshare.net/aszegedi/everything-i-ever-learned...

Thanks for the post. Learned a few things that I was not aware of.

Reply on news.ycombinator.com