[1] https://github.com/y-scope/clp
EDIT: See also[2][3].
[2] https://www.uber.com/blog/reducing-logging-cost-by-two-order...
[3] https://www.uber.com/blog/modernizing-logging-with-clp-ii/
[1] https://github.com/y-scope/clp
EDIT: See also[2][3].
[2] https://www.uber.com/blog/reducing-logging-cost-by-two-order...
[3] https://www.uber.com/blog/modernizing-logging-with-clp-ii/
If the query infrastructure operating on these compressed data is itself able to store intermediate results, then we've killed two birds with one stone because we've also gotten rid of the restrictive query language. That's how cascading mapreduce jobs (or Spark) does it, allowing users to perform complex analyses that are entirely off the table if they're restricted to the lucene query language. Imagine a world where your SQL database was one giant table and only allowed you to query it with SELECT. That's pretty limiting, right?
So as a technology demonstration of Quickwit this seems really cool--it can clearly scale!--but it's kind of also an indictment of Binance (and all the other companies doing ELKish things out there).