Hacker News Reader

Old data historians use swinging-door compression algorithms (a lossy compression). Would something like TimescaleDB make this unnecessary for IoT?

https://docs.aveva.com/bundle/pi-server-s-da-admin/page/1022...

View on HN

If I understand your question correctly: For storage, mostly yes. Swinging-door compression went lossy because byte compression on raw floats used to be useless, but with new method like Gorilla you can compress it lossless, so you can keep every sample and still afford it. But there is another point that this Aveva historian adds filter that filters some points so I don't know how it impact ingest.

But in general now trend is to use normal databases from IT world in OT world to overcome some legacy solutions.

>But in general now trend is to use normal databases from IT world in OT world to overcome some legacy solutions.

Someone tell my company this please ;-;

Based on us keeping a replica of our AVEVA data archive in delta tables which are just parquets; and only compressing the parquets in LZ4 and getting a fuck ton of more space savings. I would say yes. Especially when you're onprem with a dedupe storage appliance.

I think swinging door made sense when storage was slow and scrace, but nowadays storage is cheap (even with these prices)

Reply on news.ycombinator.com