Hacker News Reader

Comment by dschiptsov | Hacker News Reader

So, we need a new database engine instead of mere using tables with partitioning in PostgreSQL with a timestamp as a primary key?

And it is Java - waste two times more resources than the data you have ™

View on HN

Not sure if your Java comment is warranted, a lot of Apache projects are either written in Java or run on JVM. Yes, HBase's memstore can be optimized, in particular for small key values, but that doesn't negate a need for time-series databases with proven scaling. HDFS offers that. How you were planning to design your tables in PostreSQL if you have 1m+ unique series inserting at 100K+/sec. Assuming you use timestamp as a PK does it translate into 1m+ tables?

We have already seen how Cassandra clone in C++ perform.

I think that adding a new indexing strategy to a decent engine could be a good idea.

Informix did this.

Mapr has an HBase compatible stack in C++, I would love to see TSDB numbers on that. TSDB is pretty efficient itself and since metrics collection is such a batchy operation JVM isn't really that much of a problem. You can run some dedicated TSDs for front end/UIs to ameliorate JVM pain there too.

Do you mean Informix Time-Series Extension? It's been around for a while so maybe there are existing things that came out of IBM recently. Last time I checked its schema it was basically storing serie as a blob not unlike OPC historians like Pi.

It depends how much time you have to burn, and how much traffic you handle. If you have trivial scaling needs or want to spend 10x the human time managing that setup, then go ahead. You use something like opentsdb when you have hundreds of thousands or millions of ops per second and don't have the time or desire to deal with a database that you have to build your own partitioning + resharding + failover + recovery systems on top of.

B+ trees are not optimized for the kinds of write-heavy workloads that you typically stick in HBase (LSM-tree).

Reply on news.ycombinator.com