Hacker News Reader

Comment by jonaf | Hacker News Reader

Something else people should know: AWS ES is on the Internet. You can't deploy it to a vpc yet, and you can only lock it down using IAM, which may or may not be good enough for your use-case.

For those that prefer the ease provided by AWS ES Service, consider Elastic Cloud, which affords most of the same capabilities but is run by Elastic themselves (it was previously known as Found, which Elastic purchased a few years ago). There's also an Enterprise offering. If you're looking for a hosted Elasticsearch solution, it's probably better than what AWS is offering. Side note: they update about as often as elastic releases, whereas AWS ES is consistently behind.

View on HN

As Daniel Parker (acloud.guru) mentioned in the article comments, they went with Algolia [1] despite being AWS experts. He cited the uncertainty and complexity around AWS ES as the problem.

[1] https://www.algolia.com/

An attacker is only ever RCE on one server away from being on your VPC subnet. You're going to have to set up authentication for internal applications anyway, although I suppose vulnerabilities in the login process are harder to exploit if you can't even get to it.

I'm curious about how the pricing compares. I'm not very satisfied with AWS ES, but managing ES manually doesn't seem like the most fun either. (In fairness it looks like there's not too many knobs to turn, but it's still another concern to have to personally deal with.)

Elastic cloud doesn't require any more management than AWS. You just need to click some buttons to add capacity https://www.elastic.co/cloud

The IAM authentication is really annoying. It's not supported by many client libraries, nor have I found an easy way to make arbitrary HTTP calls with signature v4.

The only other options are completely public or IP-based whitelist, the latter which is untenable in most cloud environments.

You can also use a signing proxy.

I wasn't aware of that option. I'll look into it.

A simple solution in this vein is to white list your the EIP addresses of your NAT. This would give access to all resources in a private subnet (this is useful for Lambda's running in subnets).

>nor have I found an easy way to make arbitrary HTTP calls with signature v4.

https://github.com/okigan/awscurl

Yep, that's precisely why I made awscurl "easy way to make calls to AWS".

I can be easily tested with AWS Elasticsearch.

It's a great tool man, I use it tonnes, thanks for making it!

This sounds good. Any feedback on cost? How is plugin support / security? Integration with IAM?

(Disclaimer: I work on Elastic's Cloud team)

While AWS ES can be cheaper in some configurations, Elastic Cloud is actually quite competitive in pricing for larger clusters when compared to AWS' ES-service. This post compares the two services, and there's an example price comparison at the end of the post: https://www.elastic.co/blog/hosted-elasticsearch-services-ro...

We support most official plugins, and if you get a gold or platinum subscription you can upload your own plugins. Elastic's X-Pack is included in every cluster, which includes security features like role based access control.

It's not possible for external service providers to integrate with IAM at this point.

One issue I've found with Elastic Cloud is that there doesn't seem to be a horizontal scale-out option other than multi-DC or getting bigger boxes. Is horizontal scaling in the works? Easy horizontal scaling seems like one of the better benefits of ES.

Or, alternatively, am I mistaken about how configuration works?

Per availability zone, Elastic Cloud currently scales vertically (in power-of-two increments) until a cluster hits 64GiB memory, at which point multiple 64GiB-nodes are added. While you can run Elasticsearch with e.g. two 8GiB nodes per zone, we prefer a single 16GiB node as there's fewer things that can go wrong. (If you want the second 8GiB node for redundancy, then that's exactly what our multi-zone HA configurations are for, and we encourage HA setups by making them less than twice (or thrice) the price, throwing in additional master-only nodes for free)

(A bit of history: When Found (the company Elastic acquired and which is now Elastic Cloud) was in private beta in early 2012, we actually did allow custom cluster topologies. We ultimately disabled that as it was overwhelmingly used to make sub-optimal cluster configurations, such as 5 x 1GiB memory nodes)

Any idea if/when Elastic.co will support multiple user accounts and 2FA in the management portal? Not having those was a deal-breaker for us when we evaluated it awhile back, and was the sole reason we went with the less stable AWS service.

This is (understandably!) a common request, and both are on our roadmap.

My understanding is that this is going to change and soon. Amazon has been poaching key elastic search employees presumably with the idea to improve the service to on par or better.

Elastic employee here, and this is the first I'm hearing of that. I don't know a single person that has left for Amazon, from any team. Certainly no "key elastic search employees".

Got a source?

Reply on news.ycombinator.com