Hacker News Reader

Comment by smittywerben | Hacker News Reader

ElasticSearch looks like a REST api with http/json schema to pipe directly to browsers. Why would you need to store the results of ElasticSearch?

View on HN

It's not about results, the data itself within ElasticSearch is not guaranteed safe as a primary datastore, so you should have a reliable database for the primary data that is then synced to ES.

As someone who has used ElasticSearch in production for years, as far as I know this is simply not true.

That's good for you, but it sounds like you're extrapolating it on just your experience rather than research? Perhaps you've been lucky? Do you really have the info to objectively state it as "simply not true"?

Read the jepsen tests (old but show core problems that aren't fixed):

https://aphyr.com/posts/317-call-me-maybe-elasticsearch

https://aphyr.com/posts/323-call-me-maybe-elasticsearch-1-5-...

Or just search for ES data loss:

https://news.ycombinator.com/item?id=9475620

https://stackoverflow.com/questions/29841348/how-reliable-is...

And if that's not enough, here's the official page listing several "ongoing" fixes for major durability issues:

https://www.elastic.co/guide/en/elasticsearch/resiliency/cur...

After reading the links you gave, I’d like to clarify that I meant that edge cases that can cause data loss are not common enough, in my experience, for one to be forced to treat ElasticSearch as an unreliable data store. Anecdotally, I’ve never seen it happen. You’re right, maybe I’m lucky; I would like to see a case study of some project where these issues were significant enough to treat ElasticSearch as an unreliable data store.

I don't understand the syncing. Why not write-only akin to UDP protocol?

What are you talking about?

Reply on news.ycombinator.com