Unfortunately, I think it's a large reason why it was made a design choice that indexed documents are immutable (IIRC), which doesn't work for my use case.
Unfortunately, I think it's a large reason why it was made a design choice that indexed documents are immutable (IIRC), which doesn't work for my use case.
It's possible to add updates/deletes to Quickwit, but this is a lot of work, and for now, we have not prioritized this development.
Do you mind sharing your use case?
Some web pages update frequently, so I would need something that can handle that.
I do understand that you seem to be targeting the log use-case of Elasticsearch moreso than the "Apache Solr" use-case.
I would be in passing curious of what about the GDPR makes infrequent deletes a design choice. My understanding of GDPR is that the "right to be forgotten" aspects would if anything require deletions to be inexpensive.
The idea then would be to "reindex" the world. It might seem ludicrous, but to give you an idea, indexing CommonCrawl takes about a day with 8vCPUs.
I suppose their are ways to structure things to minimize this, though.