Hacker News Reader

Comment by ddorian43 | Hacker News Reader

They did the mongodb approach. Build a shitty db and then build a normal one and claim 100X faster.

And then add synchronized disk commits (like postgresql has always done) and performance goes puff (meaning lower than pg) (example: elasticsearch, mongodb).

View on HN

fsyncs happen on every write. They're durable. This release may be 100x faster than our previous release, but it's also a competitor with other solutions.

We haven't run any comparisons vs. other solutions, but we will do that soon and I expect that we'll be competitive and better in some cases.

Very tasteful way to handle the parent comment :)

Does Todd Persen talk about durability testing in the video or is more about performance benchmarking? I haven't watched yet.

https://vimeo.com/153802549

Thanks :). He talks about doing kill nines on the process while doing a bunch of writes, trying to corrupt the DB, and perf testing. In addition to a little bit of talk about what came before and some lessons learned.

Have you had a chance to throw Influx through the paces of Kyle Kingsbury's Jepsen distributed data store tester?

Not yet, but it's on our TODO list. However, we have to finish the clustering implementation before we bother going through that testing. At this point we're telling people it's experimental and not meant for production use.

This is not the first time I have seen an anti-InfluxDB post that also subtly pushes PostgreSQL in its place. Now I totally agree and I am not happy with InfluxDB's performance, and maybe I'm just being paranoid, but it's interesting I've seen this a handful of times.

I would like to see actual comparison of both. I'm going pg way myself but due to millisecond precision of timestamps required for handling scientific data.

InfluxDB supports timestamps down to the nanosecond. We recommend that you only use the precision you need, since you'll get better compression that way.

Good to know, thanks!

Have a look at this release. The write performance is massively better than it was. We did tests writing up to 100B points against a single server.

The next release will focus on improving query performance, but this one still works for many cases.

What is the query performance like today? And, what are your goals for query performance after the next release?

"Massively better" doesn't mean deployable to production either. Especially not if you've only tested with a single point of failure with a single sized payload. But I'm willing to give it a third (and final) trial:

1) Does "DELETE * FROM foo" still cause the system to lockup, freeze, and require a restart to free memory? Or are there other conditions/queries that cause the system to become unstable?

2) There's no README/CHANGELOG/dependencies info on the download page. Which is the preferred version of Go to install on my servers for Influxdb - 1.4 or 1.5?

I'd guess go version doesn't matter, you should install the binary which is statically linked, no?

Massive performance improvements like that tend to come down to overall changes in the data structure and/or algorithm. This is like, "oh yeah, maybe we should do LSM?" or "maybe compressed bitmaps?" kind of stuff. What what the magic?

New storage engine.

Introduction from last October: https://influxdata.com/blog/new-storage-engine-time-structur...

Oh right, I remember when that blog post came out. It was amusing because I'd built an in-house metric store much like InfluxDB (using a LevelDB engine), and reached much the same inflection point and decision. I so should have open sourced that so you poor saps didn't have to experience my pain.

Nothing against influxdb. But if you have 100X, you probably were doing something shitty before.

I use postgresql as comparison because antirez did in http://oldblog.antirez.com/post/redis-persistence-demystifie... (and ~none else has done something similar).

Reply on news.ycombinator.com