Hacker News Reader

Comment by storyinmemo | Hacker News Reader

The paper on the algorithm is here: http://www.vldb.org/pvldb/vol8/p1816-teller.pdf

Somebody implemented the algorithm in go based on the paper here: https://github.com/dgryski/go-tsz

A short answer that may work for your question: the bits that are set in RAM are xor values relative to previous values. To provide an answer as to what the value is, a series of read|xor operations are performed.

View on HN

They refer to it as "delta of delta", which implies the encoding handles things like acceleration and momentum. I'm guessing the healthcare data has much more of this sort of behavior than your typical server event time series.

Not quite. Delta of deltas means that periodic timestamps turn into a long run of zeros (since the difference between each point is constant), which is very easy to compress. In the Gorilla paper, 96% of timestamps compress down to zero, which is encoded by a single zero bit.

For series without periodic sampling, delta-of-deltas performs similarly to normal deltas.

"delta of delta" is only used for timestamps, not for values. In any standard monitoring setup, the acceleration of timestamps will be 0 for a big portion of measurements, and very small for a big portion of the rest, so it makes compression extremely easy and efficient.

> I'm guessing the healthcare data has much more of this sort of behavior than your typical server event time series.

I thought they were talking about healthcare data too when I first read it but seeing as this is Facebook I realized they are talking about server health not human health.

Haha, I thought jdonaldson was making a joke :P

Reply on news.ycombinator.com