As a consequence of being two different things, storing and analysing them efficiently need different solutions. I've seen people stuff everything in Elastic Search and it's certainly possible if you really want to.
As a consequence of being two different things, storing and analysing them efficiently need different solutions. I've seen people stuff everything in Elastic Search and it's certainly possible if you really want to.
I think the thing is that the primary use-case for a lot of people combines the two - eg they want to log a whole key:value map where values can be numeric or string, and then dynamically generate time series. We can do this with SQL, like if I want a graph of traffic per domain name, "SELECT time_bucket, domain_name, sum(bytes) FROM access_log GROUP BY time_bucket, domain_name" - but all SQL servers are totally non-optimised for this use case.
Surely there must be some software out there which can do this efficiently?
The more pertinent difference is event logging vs. metrics.
For the former the ELK stack is popular, and Prometheus.io is suitable for the latter.
I found that by trying to see if it could be used as a metrics DB but stumbled upon a few issues like configuring retention times per target. See for example https://github.com/prometheus/prometheus/issues/1381
Prometheus has lots of potential but it's not a metrics DB at this point or in the near future. I just wish they'd made that a bit clearer on the page.
"We make design decisions that presume that Promtheus data is ephemeral, and can be lost/blown away with no impact."
That viewpoint pretty much limits them to be an ops tool. I wish they'd reconsider this point.