This looks similar to Graylog (https://www.graylog.org/)?
From Loki's readme, It still stores the full log (but compressed), and indexes values in each logline.
From Loki's readme, It still stores the full log (but compressed), and indexes values in each logline.
Inside kubernetes the metatadata would be podname, namespace, deployement name, container name, etc...
We batch together sets of lines into compressed "chunks", and only add index entries per chunk. In practices, chunks have 1000s of lines in them - so the index tends to be many orders of magnitude smaller than in other systems.