Hacker News Reader

Comment by theomega | Hacker News Reader

I have used TimescaleDB for several purposes. As it is built on top of Postgres, all the existing tools, libraries and processes work out of the box. This is a huge advantage if you are operating Postgres anyway: Your existing backup tools will work, as does your user Managment.

Scaling out is a more complex story: Read Scaleout works seamless using read replicas (another existing Postgres mechanism). Replicas can also help you for high availability, Write scaleout is difficult with TimescaleDB (same for Postgres): Sharding can be an option, or buying bigger machines.

View on HN

(Co-founder TimescaleDB)

Write scale-out is something we have been working on for nearly a year now.

Here are some preliminary benchmarks we shared at Postgres Conf earlier this year (benchmarks have continued to improve since then): https://twitter.com/acoustik/status/1108813512321708032

We'll have more to share regarding write scale-out later this month :)

We ingest a lot of time series (IoT) data and use Postgres for other data so Timescale works quite well for us. One thing Timescale treats as second class citizen though is updates to existing data points which are 1-2 areas of magnitude slower than inserts. Granted this is also the case for all other TSDB solutions out there which are for obvious reasons optimized for inserts and reads aggregated along the time dimension. Still would be amazing if you could add to the already existing differentiation of allowing fast updates for cases like ours where we dont store events relating to a singular point in time but rather time-spans so new incoming data points might be "merged" into existing time-spans.

Same domain. I've come to the conclusion that these systems are dualistic in nature and the architecture needs to reflect that. I think the more general expression of this would be a time-series -> document-store pipeline. The concern then, if the time-series processing step requires domain data, is serving up fresh views of the evolving state in toto. That epsilon of temporal lag of 'current state' in the premier stage is a product issue. For some use cases it can be a very tight constraint, but I think quite a lot of others will be perfectly fine with snapshot isolation with generous requirement on freshness.

From what you're writing it seems TimescaleDB doesn't really solve large timeseries problems. So it looks like more a good transitory solution than a definitive solution.

Is that correct?

In my opinion you're correct, it doesn't solve the problem of querying large sets of timeseries data over a long time span.

It doesn't pre-aggregate your data so if you're trying to get aggregates for a large data set, it's not fast.

(Co-founder TimescaleDB) Pre-aggregation via continuous aggregations is in our next release, which I believe is slated for Monday.

We also have more in the works for large datasets: eg scaling out storage across multiple machines as well as data tiering for lower storage costs.

Feel free to email me if you'd like more info on either: ajay (at) timescale (dot com)

Hi! Engineer at Timescale who’s just been working on this feature, it’s coming in our next release and going to continue to be developed over the next few releases! Would love to have you try it out and get your feedback :)

God I love you folks. I can't wait for timescale to be available on rds, the more I read about it the more I love it and its authors.

Thanks for the feedback.

How is TimescaleDB doing with compression by the way? Meaning if you compare your raw data and the disk usage, what's the ratio?

You can get 6-8x compression by running on TimescaleDB on ZFS, but compression in general is also an area where we are investing a fair amount of R&D.

Also worth noting that TimescaleDB is far more memory-efficient than other time-series databases (eg InfluxDB [1]), and for some folks the memory cost savings is significant (b/c memory cost $$$ >> disk cost $).

[1] https://blog.timescale.com/timescaledb-vs-influxdb-for-time-...

Will it work on top of RDS?

No. https://forums.aws.amazon.com/thread.jspa?threadID=261623

The theory is they are not supporting it in favor of promoting their new time series DB.

(Co-founder Timescale) This is good feedback. Do you need something on RDS specifically or just managed on AWS?

Being a small team, we try to have managed hosting as much as possible. One less thing to think/worry about.

Then our stack is primarily in AWS. So when it comes to DBs, I'd prefer to keep it all within our VPC for security / latency reasons - for sensitive data at least.

Great info. Thanks for sharing.

Reply on news.ycombinator.com