InfluxDB v0.9.0 released with developer and production support | Hacker News Reader

InfluxDB v0.9.0 released with developer and production support

influxdb.com

66 pointsArtemis237 comments

Are there any plans to run @aphyr's jepsen CAP testing system on influx? It doesn't give me a ton of confidence that you're redoing your clustering pretty much from scratch a 3rd time in a pre-1.0 piece of software.

It would also be excellent if you could post more on the not really CA and not really AP aspect of your new clustering design, and why no one has really done that before (from what I can tell). Why that is the best path forward.

Thanks for releasing such awesome OSS software.

We will be doing that testing in the future. I've already talked to Kyle about our design. We'll iterate on his and other smart people's feedback. That's one of the reasons the clustering alpha right now :)

This seems reasonable, thanks. Keep the blog posts coming, they're great!

I never nail it the first time around. The fact that they have explored the problem three times and revised their solution in pre 1.0 piece of software actually comforts me. pre 1.0 is a great time to make big revisions like this. It may have been close to good enough the first time, but the third iteration will undoubtedly be better. From @aphyr's blog it seems that other projects have issues with clustering, so the fact that influx's clustering design is unique isn't necessarily a detracting factor. It isn't not re-inventing the wheel if there is no tried and true solution they could be using.

For people curious where the clustering support stands:

"From the InfluxDB clustering design doc we’ve implemented the write path to replicate data to other nodes. We’ve implemented the per write request consistency levels and hinted handoff, ensuring that server reboots and short outages are quickly recovered from and consistency is restored.

What’s missing are anti-entropy, distributed queries, joining new servers to a cluster, and tools for replacing servers in a cluster. We’ll start development on some of these features in the 0.9.1 release cycle and more in the 0.9.2 cycle. They will be merged into releases once they’re stable and tested."

Super excited about this. Congratulations on the release!

I've seen Prometheus bash a little on InfluxDB regarding storage requirements [1]. Is this something that has been addressed? Is it even something worth worrying about?

1. http://prometheus.io/docs/introduction/comparison/#data-mode...

We will be starting work on a new storage engine or an optimized version of what we have now that will add compression. Tentatively, that will start in the 0.9.3 release cycle.

First rule of software: make it work, then make it efficient

I hope you guys collaborate with the RocksDB team. Column families [0] seem like a great fit for time series.

[0] https://github.com/facebook/rocksdb/wiki/Column-Families

Last I checked that still held true. Here's a more up-to-date version of those benchmarks: https://groups.google.com/forum/#!searchin/influxdb/promethe...

For Prometheus's long-term storage I'd basically like to have something like InfluxDB (Go, simple to operate, distributed storage), but better suited towards purely numeric time series rather than arbitrary log records. Unfortunately as it stands, InfluxDB takes roughly 14x more disk space for exactly the same data (without replication) for a typical Prometheus use case. I hope that improves in the future, but InfluxDB's use case is also different: they enable you to store arbitrary key=value metadata with every single record, rather than defining a time series once and then only appending numeric samples. So the comparison is not really fair, it's just maybe not the right tool for Prometheus-style monitoring.

Blog post with more useful information: http://influxdb.com/blog/2015/06/11/InfluxDB-v0_9_0-released...

The lack of an upgrade path from 0.8.x makes me feel pretty stupid for trying out that version. Then again, they don't have upgraded client libraries for Python either, so we wouldn't be ready to upgrade anyways...

Thanks! We updated the link from https://github.com/influxdb/influxdb/releases/tag/v0.9.0.

Finally! congrats @pauldix & co. Go checking that influx 0.9 wasn't out yet was the only thing that made me feel better about missing deadlines at work, looks like I'll have to find something else :)

Sorry to ruin a perfectly good excuse :)

Lies, damn lies, and delivery estimates

How can you release a product with such an outdated doc? Seriously, I literally copy-pasted the example of data insertion from the github README and I get 'Time Serie Name is required.'

The Getting started mentions the line protocol, and the link to 'learn more about it' talks about the POST body being JSON (a malformed JSON cause I get the same error in the UI when trying to insert it). Which one is it?

We'll update the readme. Overlooked in the push to release. The docs on InfluxDB.com should be accurate

The libraries also weren't updated :) -> see https://github.com/influxdb/influxdb-python/issues/193

The issue behind is that "name" has been renamed to "meassurement" - https://github.com/influxdb/influxdb/issues/2564. They implemented that change on 2015-05-21 in rc31

I wonder when they are planning to replace their shitty DEB and RPM packages with something built properly and with proper post-install and init scripts. Currently they're using ridiculous interactive script that clamps together fetching code, compiling it, packaging artifacts (in fact, it contains some artifacts (sic!)), and shipping packages to Amazon.

Have a recommendation for what we should be doing? More than happy to update how we package to make things easier or more idiomatic

1. Split all the above-mentioned steps to separate scripts or makefile targets. It's ridiculous to mix them; how are people supposed to build InfluxDB inside networks that don't have access to internet?

2. Learn how to build RPMs and DEBs the proper way (Debian source package, SRPM) instead of cutting corners with FPM. It fails to include dependencies, and full description, for instance, or to mark files as configuration.

And yes, your binaries do have dependencies.

Also, your DEB metadata is broken, according to lintian.

3. Learn where to put which files in Linux. LSB wasn't agreed on for nothing.

4. Rewrite your post-install script to something that fits your target distributions' conventions. This most probably means two scripts, for Red Hat and for Debian. For instance, under Debian your packages create service user that doesn't fit the rest of the system (UID above 500, home under /home, usable shell).

I've wanted to create (deb) packages in the past, but found a large number of conflicting instructions on the "right way". Is there an approach / howto-doc you would recommend?

The right way is to provide a debian directory with the appropriate packaging metadata.

Use git-buildpackage and associated tools (git-dch etc) to build the packages natively (or allow others to build for their Debian based distros).

Thanks, but it's still not obvious to me what to do. Looking at this: https://wiki.debian.org/PackagingWithGit, git-buildpackage documentation seems to suggest either creating a separate branch, or packaging tarballs, both of which seem different to what you're suggesting where I just create a debian directory.

Do you have a different document or an example repo with the debian/ directory? This is why I've resorted to FPM in the past - the deb tools seem so simple, and I'm sure they are simple for people that have done it before, but the documentation for FPM is much more direct and thus people (such as myself) use it, even though we know it's not "right".

Just like the large number of conflicting instructions for building iptables firewall, none of which got it how to write a set of rules and instead focuses on how to write a set of commands. One just needs to apply common sense instead of blindly follow somebody's recipe.

But why not just use the introduction from the official documentation?

https://wiki.debian.org/IntroDebianPackaging

I find that introduction is intended for people packaging software that other people have written, where upstream creates a tarball.

What FPM nails is the use-case where a developer wants to package their own software. Obviously I could take on the role of upstream, publish .tar.gz files, and then take on the role of packager and consume those, but I think you can understand at that point why FPM looks so attractive!

You obviously know this area well: a link to software you have written and packaged would be greatly appreciated.

It'd also be nice to have official systemd service files. With Debian 8 out there its basically necessary now.

Has anyone been adventurous enough to use the 0.9 release candidates or previous versions to collect and analyze application & server metrics with statsd, Grafana, etc.? How has it worked for you? Any recommendations?

I've been waiting for 0.9 to hit, so I guess I'll find out myself soon enough.

My server ran out of disk and now I can't 'list series' in InfluxDB and my graphs are empty. Need to investigate...

(0.8.8 + statsd + grafana)

I've been running infxludb 0.9 + heka 0.9.x + grafana 2.x stack in a production environment for over a month. Mostly successful, although I find the grafana tooling for building queries to be a bit weak. I think there are improvements coming soon to address this, though. Overall, influx performance seems to be pretty good although we aren't ingesting enough data to say anything really concrete. We also make use of continuous queries to aggregate our time series, which of course makes large-interval queries more efficient.

We've been running Influxdb 0.9 (pre-rc32), Heka 0.9 and Grafana 2 as well. It's been going OK, though we really needed to add stats collecting and batch write in Heka layer to not spike the CPU on writes to Influxdb.

The data write API changes so late in the 0.9 release candidates phase was an unexpected surprise. Also, needing to throw away all data written using pre-rc32 versions sucks. No way to migrate db versions? Boo!

Minor typo in http://influxdb.com/support/

"Purcahses" for InfluxDB development support provide...

Also, their support options don't seem to have prices? Why publish a list of different tiers if you have to ask for a quote anyways?

Go Paul and Todd! Go!

Homebrew still has version 0.88. Will 0.9 be coming soon?

yep, should be getting updated in the next day or so. We're on it ;)

The download link is currently throwing access denied :(

looking at it, should be fixed in a few minutes

wouldn't pay for a developer plan when the docs are so out of date. Update the docs, and I'll see what I really need paid support for.

Reply on news.ycombinator.com