Hacker News Reader

Comment by nickphx | Hacker News Reader

that's pretty neat. I need to figure out a way to migrate ~10tb of data on a 5 year old postgres RDS instance that continues to balloon in disk usage. D:

View on HN

(Timescale co-founder)

Let us know how we can help :-)

Also just curious: What's your application?

Web push notification service. :|

Thanks! Happy to put you in touch with someone to help you migrate. Or at least, to help you calculate how much cost you could save :-) ajay (at) timescale (dot) com

If you load that into a column store you’ll likely see that drop to 1TB or smaller.

Very much agree.

Even without this new capability of S3-based storage, Timescale's native columnar compression often gets like 95% storage reduction, even while staying fully in the PostgreSQL ecosystem.

(We often see our users amazed about how much less storage/disk they require after migrating from RDS.)

https://docs.timescale.com/timescaledb/latest/how-to-guides/...

But isn't that the "wrong" store to use (as far as performance is concerned) when you want random, row-level access?

Columnstores can do row level access by trading off a bit in terms of compression. If you organize the columnstore files as an LSM tree and use incremental compression schemes (so you don't have to compress too many more rows then the one your after) it can get close to the performance of a B-Tree for point reads - it depends on the specific table schema.

[1] https://dl.acm.org/doi/abs/10.1145/3514221.3526055

Reply on news.ycombinator.com