What's worse (and I've seen this trend a lot) is to use Elasticsearch for metrics. My god.
Try to replicate the freedom (no-schema, cardinality, adding a new node to cluster) of Elasticsearch with InfluxDB or others and you will hit cardinality problems real fast.
Open to hearing other opinions though.
The result is a dashboard that takes ages to load just to show a trend in values.
I think it's using the wrong tool for the job, but maybe it's just me.
Immutable time series data like logs and metrics are a great fit for Elasticsearch due to the way Lucene stores data. Documents in Lucene are immutable so an update in Elasticsearch is creates a new document and places a tombstone marker on the old one. Immutable data means you don't have to tolerate those inefficiencies.
Dashboards don't load the entire dataset by default. I can't remember what the exact default time range is but I think it's ~15 minutes or so. They're fairly quick to render in Kibana.
Elasticsearch is a great tool for observability data (logs, metrics, and APM data). Elastic's tooling makes a lot of this really easy in most cases.
One of the main strengths of Elasticsearch is that you can use it for searching and aggregating in a single query. But you need to ensure you are searching on fields that are indexed for search and sorting/aggregating on fields that are indexed for that.
any idea why? Seems like an odd approach to me.
It seems to commonly be perceived as a panacea of databases.
If you're not doing free-text search, and your data will fit in memory in a big relational database VM for the foreseeable future, why Elasticsearch?
It’s generally considered too expensive to derive metrics from logs in this way, but it’d be a killer debugging experience.