otisg

For once smaller is better.

How would you compare Vespa to ES when it comes to CPU/memory requirements and performance for something like log indexing and search use case?

That's interesting to hear. Would love to read about how Vespa compares to Solr and ES.

This may be of interest to you: https://sematext.com/opensee/report/project/trend?q=ElasticS...

Would you happen to know how Vespa compares to ES in terms of memory or CPU footprint? Have you done apples to apples comparison by any chance?

> so tired of Elastic just randomly killing their clusters out of nowhere and having to spend basically triple to deal with it that they're bringing it all in house.

That is because Elastic Cloud is not a fully managed Elasticsearch. People often don't get that with Elastic Cloud you are still responsible for your ES cluster. That's one of the differentiators that e.g. Sematext has (disclaimer: founder).

> AWS's hosted solution is poor, Logz.io and ElasticCloud are expensive.

Right. Have a look at https://sematext.com/logsene pricing. People say Sematext compares favorably to Logz.io and Elastic Cloud.

At Sematext we help companies with Apache Solr and Elasticsearch. ES/ELK is definitely used mode for timeseries sort of data. Solr community puts more focus on full-text search (email search, product search, database search, etc.). Elasticsearch can do that, too, and we regularly help companies who use ES for that, but Solr seems more focused on that use case.

People were constantly confused about OT vs OC differences. Great to see this merge!

Do you think AWS will "donate" the distro to ASF or CNCF or Eclipse?

Great to see Java 12 (and the shorter release cycles) but how quickly do people actually move to newer Java? Here's a poll with some results: https://twitter.com/otisg/status/1108093046397247488 (still open, please vote so there is more data)

I'd love to hear more. Could you describe your experience and elaborate on how and why it's next-gen compared to ES?

You would think so, but that is not how it works with Elastic and lots of it customers. I have heard that from N of their (ex-)customers. I get why this happens for them. There is a certain vision and goal, and anything that doesn't align is a nuisance. This is part business part philosophy, culture, and brand you want to build in and around your business.

There are other companies that offer enterprise support for Elasticsearch (e.g. https://sematext.com/support/elasticsearch-production-suppor...) as well as training an such. No loss for ES users there.

Yeah. I think the motivation there was the same. Judging by https://github.com/corretto/corretto-8/graphs/contributors and https://github.com/corretto/corretto-11/graphs/contributors there is some adoption... Not a lot yet.

WDYT: Would it have been better for AWS to initiate Open Distro for ES as part of e.g. Apache Software Foundation, whose mantra is "community over code"?

(or if not ASF for some reason, then Eclipse or CNCF for example)

Then, if that happens, we will see the "Open Open Distro for ES" ;)

My Twitter followers include a lot of Elasticsearch and Solr users. It will be interesting to see what they think of this from their POV as users... https://twitter.com/otisg/status/1105146138921979905

Oh, interesting. I see this as a pure attack/defense move, not just marketing themselves in the OSS world. We'll see :)

Here is one good comparison from Sematext (Sematext provides both Solr and Elasticsearch consulting, support, and training, so we are familiar with both Solr and ES and have no horse in the race):

https://sematext.com/blog/solr-vs-elasticsearch-differences/

> This is a very surprising move from AWS.

I wonder why you say that? It's kind of obvious that somebody was going to do it sooner or later to protect themselves, no? Perhaps fearing the license change?

> So you effectively get a fork.

Yup, I assume that will happen, too.

Any modern monitoring service will provide you with volume discounts if you have hundreds or thousands of servers.

Since you mention Papertrail specifically in the context of costs - Papertrail is actually a bit pricey relative to the competition. For example, compare https://papertrailapp.com/plans to https://sematext.com/logsene/#plans-and-pricing . I think Sematext Logsene is 2-3 times cheaper than Papertrail.

Lastly, I was at a Cloud Native Conference in Berlin last week. A lot of people have the same setup as you - ELK for logs + Prometheus for metrics. We're running Sematext Cloud where we ship both our metrics and our logs, so we can easily switch between metrics and logs much more easily, correlate, and troubleshoot faster. Seems a bit simpler than ELK+Prometheus...

Both metrics and logs are needed for the full picture (plus alerts and a few other things, like RUM and error/exception capture, both on front-end and backend).

Saying that logs should never be sent to the cloud is overly absolute. Some logs should indeed stay behind firewall, but lots of organizations have logs that can be shipped out to services whose features derive all kinds of interesting insights from logs.

Unfortunately, the post doesn't share things like: how much infra is needed and how much does it cost, how much time it took to set up, how much maintenance it needs, how long upgrades of the setup take, how much time future hacking of missing features will take, and so on. After that sort of stuff is truthfully taken into account I suspect most if not all savings would be lost.

We've used Grafana with Sematext Logsene (which exposes Elasticsearch API, so it's like having Grafana talk to ES).

Here's a short howto + video: https://sematext.com/blog/2015/12/14/using-grafana-with-elas...

Did you keep the beard?

Is that list of events exhaustive? Actually, a better Q: are there any Docker events SPM doesn't capture?

VERY thorough and juicy, thanks.

interesting, but Q: "journalctl -o short -f | ncat --ssl logsene-receiver-syslog.sematext.com 10514", what happens after journalctl dies/restarts?

+1, 30 pages of pure Elasticsearch metrics/monitoring goodness!

We use:

* SPM for monitoring and alerting for our ElasticSearch, Solr, Hadoop, and HBase clusters, as well as Java (web)apps + the usual server metrics + custom business and performance metrics

* Sensu for server metrics monitoring + alerting

URLs:

* http://sematext.com/spm

* https://github.com/sensu