Hacker News Reader

Comment by airhead969 | Hacker News Reader

IIRC, ZK would be more modern and cloud-friendly if it could self-assemble with a preshared passphrase alone. It's good technology otherwise, it's just a PITA to deploy, configure, and support.

View on HN

I really love a lot of the software under the Hadoop umbrella, but so much of it assumes a static deployment on bare metal hosts, it's a struggle to use it in "modern" setups (HBase, for example; I miss my old friend).

https://azure.microsoft.com/en-us/services/databricks

Yeah. They should have written it twenty years ago in a datacenter with public cloud in mind

Hadoop launched in 2006, the same year as AWS' cloud portfolio. HBase showed up in 2008.

Many of the hiccups with running Hadoop and friends in containers or on cloud VM's boils down to how hostnames are resolved and advertised; not any significant design issue.

> Hadoop launched in 2006, the same year as AWS' cloud portfolio.

Which makes it all the less reasonable to assume it would be in any way cloud native, when "the cloud" was at best a nascent idea at that point.

And how many years did it take AWS to get any serious traction after launch?

I haven't assumed anything of the sort, I said I loved the software and wished it was easier to use in what is considered a "modern" environment.

The irony of the name Cloudera has not been lost on keen observers of this space.

It makes you wonder how many people think "clouds" run on something besides servers.

It seems weird to me that cloud providers do not offer distributed coordination primitives "as a service." I understand there are KV stores but not with watches, locks, etc. in the way that etcd and ZK have them.

Agreed. We need highly available & distributed

1. Locking service

2. Id generator

3. Bloom filter etc.,

Azures blob service supports the first one. Its strongly consistent and has a lease/lock API. Ive seen leader election and coordination stuff built on top of it.

Why the id generator? Aren't GUID's enough? The chance of a collision is basically 0.

A few different points of view:

https://tomharrisonjr.com/uuid-or-guid-as-primary-keys-be-ca...

https://news.ycombinator.com/item?id=14523523 (the hacker news comments for the above article are really insightful)

https://www.percona.com/blog/2019/11/22/uuids-are-popular-bu...

https://blog.codinghorror.com/primary-keys-ids-versus-guids/

For most of the use cases GUID works great. There are some cases where we need a numerical id which is sequential.

Are you looking for something like this ? https://azure.microsoft.com/en-us/updates/azure-cosmos-db-ap...

Yes! Exactly. Are there analogues in GCP and AWS?

The dynamic reconfigure API they added is also terrible, imo. The only thing that made ZK tolerable to use for me was Exhibitor but that appears to be long dead.

Reply on news.ycombinator.com