Hacker News Reader

Just upgraded an AWS cluster from Elasticsearch 5.5 to Opensearch 1.3 - which is just really is an ES 7.10 fork. Those are 4 years apart and many releases in between, I was expecting some expressive cost reduction, but not at all, it took the exact amount of VMs for the cluster to stay above water.

View on HN

Elasticsearch in later versions has done quite a lot in terms of memory reduction. Last section of https://www.elastic.co/blog/three-ways-improved-elasticsearc... but it has been an ongoing effort in multiple of the later 7.x and and 8.x releases.

I would have expected some memory savings due to the indexes being lighter on the heap. Perhaps your workloads are more cpu heavy than memory?

Quite likely yes. I can't share the graph, but memory pressure didn't alleviate at all as I'd hope it would. We'll be experimenting with different VM types next.

Shameless plug, but one of the other blog posts in the above series explain how we addressed high and unpredictable memory load in relation to wildcard searches.

https://underthehood.meltwater.com/blog/2022/11/25/how-we-up...

After those changes were made then we haven't had any major issues with heap spikes. But it depends on a lot of factors, how your data looks, how your queries look and what aggregations that are made so its very hard to give some always applicable advice.

Reply on news.ycombinator.com