There's so much marketingspeak in that piece, it's nauseating. I use and maintain an ELK stack and I'm not even entirely sure what most of that nonsense means.
"Data Streams sounds cool! It could help alleviate some of the management overhead that I was accomplishing with ingest pipelines."
clicks into the docs
`X-Pack only`
I get that it's complementary to their ILM functionality, which, too, is an X-Pack feature, but still, clarity would be helpful.
e.g. "Data Streams" is covered under the Basic license so you can use it for free (I think? As far as I can tell at least). You can find the feature on their subscriptions page: https://www.elastic.co/subscriptions
I find the whole situation with X-Pack, Subscriptions and OSS vs Basic super confusing and the subscription page seems very bad at explaining it all.
But the executive summary for an ELK user like you:
- Kibana just got faster (finally). I know some people in that team and they completed some internal javascript refactoring that gets rid of the annoyingly long page loads. So, this is a relatively big improvement for a 7.9.
- Elastic Agent is a new thing that can get a lot of data into dashboards quickly. That might be relevant for you if you have a (no doubt) lovingly handcrafted logstash ingestion pipeline and matching dashboards. Also, setting up half a dozen elastic different beat agents to achieve the same was definitely a bit of a devops burden. Sounds like something I might give a try next time I set up new infrastructure.
- You can now self host Elastic Cloud in your own AWS infrastructure. This may be relevant for you if you were self hosting some home grown setup on premise (or in aws) or are using the fully managed cloud solution. Kind of a big deal for more advanced setups as a lot of companies get themselves into trouble with getting their homegrown devops for this right.
- A lot of updates to Elastic products neither you nor me use. The workplace search thing sounds interesting though. Wasn't really on my horizon so far.
I feel attacked, somehow. I have made so many logstash pipelines. Some of them I love. Others I don't.
Elasticsearch:
* Data streams -- an abstraction that eases management of a set of indices containing time series data.
* Wildcard data type -- makes searching for partial strings much more efficient
* EQL -- a new query language that's popular in the SIEM (security info and event mgmt) world
* Miscellaneous new aggregations
* Tableau connector for Elasticsearch
Other parts of the stack:
* free version of Workplace Search (index all your stuff from multiple sources such as Gmail, Google Drive, MS Sharepoint, Salesforce, etc.)
* free version of endpoint security from Elastic's acquisition of Endgame
* Ingest Manager/Elastic Agent -- centrally manage and configure Beats-based endpoint agents that push metrics, monitoring, etc. to Elasticsearch
* new rendering engine in Kibana