Ask HN: Why do we need Time series database? | Hacker News Reader

Ask HN: Why do we need Time series database?

4 pointsrmitra7 comments

A lot of new time series databases are coming up. Why postgresql or mongo or other existing databases are not enough?

If you've ever had to do time oriented joins, or even just store a bunch of time oriented data, you would know the answer to this question. Timestamp keys are important. Storing the data in a structure that makes sense for a time series is important. The ability to construct verbs which make sense is important.

Is Kerf still under active development?

When Time Series databases peak your interest you might want to take a look at a new kid on the block; SiriDB. (https://siridb.net)

Full disclosure, SiriDB is created by my team and is released as a full open source solution including cluster mode support which allows to easily spread and safeguard your time series data across multiple nodes when data grows. SiriDB is fast, has no dependencies and is capable of handling large volumes of time series data.

A time series database is an abstraction where a timestamp [1] is the key in the first order tables. Often the abstraction is built on top of some well known database because getting all the bits to write and read from storage is a hard problem where existing solutions exist. For example, Kairosis built on Cassandra [2] and OpenTSDB uses Hbase. [3]. It's not the only way to do it, InfluxDB uses its own back end. [4]

[1] or a tuple of timestamp and some other values.

[2] https://kairosdb.github.io/

[3] http://opentsdb.net/

[4] https://www.influxdata.com/

I am not an expert in this, but I would say they are domain-specific. Time series databases I hear about are Prometheus[0] and InfluxDB's offerings[1], both meant specifically for monitoring something over time, hence the name.

[0] https://prometheus.io/docs/introduction/overview/ [1] https://www.influxdata.com/products/

Can you list some of these new ones?

Here is a solid overview of some of the systems out there: https://docs.google.com/spreadsheets/d/1sMQe9oOKhMhIVw9WmuCE....

Facebook also recently open-sourced their internal TSDB, Gorilla (http://www.vldb.org/pvldb/vol8/p1816-teller.pdf), as Berengei.

The concept of a database built for time-series data specifically is in vogue lately (if you couldn't tell). Most of the TSDBs in the spreadsheet above, for instance, are NoSQL data stores designed for high ingest.

Full disclosure, we're also developing a new time-series database (http://www.timescaledb.com/) because we found the ones above achieved ingest scale by sacrificing query performance (and SQL). We needed something that had good ingest/query performance at scale (and we wanted to use pure SQL).

If you're curious, here's our technical paper: http://www.timescaledb.com/papers/timescaledb.pdf.

Reply on news.ycombinator.com