Hacker News Reader

Comment by bomewish | Hacker News Reader

The big issue with tantivy I've found is that it only deals with immutable data. So it can't be used for anything you want to do CRUD on. This rules out a LOT of use cases. It's a real shame imo.

View on HN

I’m pretty sure that Lucene is exactly the same, the segments it creates are immutable and Elastic is what handles a “mutable” view of the data. Which makes sense because Tantivy is like Lucene, not ES.

https://lucene.apache.org/core/7_7_0/core/org/apache/lucene/...

It is indeed mostly designed for bulk indexing and static search. But it is not a strict limitation, frequent small inserts and updates are performant too. Deleting can be a bit awkward, you can only delete every document with a given term in a field, but if you use it on a unique id field it's just like a normal delete.

Tantivy is a low-level library to build your own search engine (Quickwit), like Lucene, it's not a search engine in itself. Kind of like how DBs are built on top of Key-Value Stores. But you can definitely build a CRUD abstraction on top of it.

Certainly one could build that on top of quickwit — which also doesn’t allow crud — but it’s not trivial. You need to insert a new record with changes, then delete the record you want to update. The docs instruct that the latter action is expensive and might take hours(!). Then one would need a separate process to ensure this all went down appropriately (say server crashes after insert of updated record but before delete succeeds). Meanwhile you’ve got almost identical records able to be searched. Just not very nice for anything involving CRUD.

Please do advise if I’ve missed something here. I was really excited about using quickwit for a project but have gone with Meilisearch precisely for these reasons. Otherwise it would be quickwit all the way.

Reply on news.ycombinator.com