Loki author here: got some ideas? We’re all ears!
That, and retention policies. I just had a disk fill up because Loki keeps saving data.
That said, I’ve been using Loki for several months and I love it. Keep up the great work!
I'm also looking at sonic, which is a new full-text search engine in rust with less overhead than Elasticsearch. It's lacking a gui focused on log search.
Could they work together somehow, maybe as an alternative backend to you distributed grep?
Sonic does not store the original content, so it could store a reference into your compressed chunk.
Thanks!
Even a simple label match would be a good start.