Hacker News Reader

Comment by cduzz | Hacker News Reader

I've often wondered why someone doesn't reimplement elasticsearch using sqlite as the back-end datastore, instead of lucene.

Or did I just describe foundationDB?

I guess the real problem is that such a beast wouldn't have kibana, and people want kibana and put up with elasticsearch to get it.

View on HN

As far as I can tell, Lucene is the library that powers the main search-related features of Elastic Search i.e. inverted indices, stemming, tokenization, etc.

Elastic Search, Amazon Open Search, Solr, etc. just wrapped Lucene up in a much easier to use HTTP / Rest API and then added bells and whistles like a coherent DSL, scripting, authentication and authorization, jobs, bulk API, etc. Then they built additional tools like Logstah and Kibana, which integrate using these APIs.

GP did say 'reimplement' (not making it sound trivial, like 'add a sqlite backend plugin') in fairness - i.e. in your terms 'why not implement the ES API for sqlite'.

More like "make a thing that's a bit like elasticsearch, but exposing sqlite query language to a sharded event store" -- not "make elasticsearch but use lucene" because elasticsearch is lucene, and reimplementing it via sqlite wouldn't give elasticsearch.

I'd call it elasticite and then (correctly!) be sued by both elastic and the sqlite orgs...

Because Lucene’s index format is highly optimized for information retrieval and SQLite’s isn’t. I love SQLite but Lucene is the sine qua non heart of ElasticSearch.

Reply on news.ycombinator.com