Hacker News Reader

Comment by vegabook | Hacker News Reader

just lean on the shoulders of giants who have thought all these points out before, and use Erlang (or Elixir). That includes back pressure.

http://elixir-lang.org/blog/2016/07/14/announcing-genstage/?...

View on HN

Erlang doesn't have great inter-node backpressure. Messages sent from one node to another in a cluster will suspend the sender if the output queue is too big, which is a global value independent of the queue length of the receiver. Also, the inter-node protocol is based on TCP, which has the usual TCP problems (head of line blocking, etc.).

Erlang's distribution story is great when you're on a single machine; pretty good if you're on a very high reliability, high bandwidth LAN; and not great if you're going across any network not under your direct control (e.g. AWS, the public internet).

k but a) back pressure as per my original post with Elixir (or Apache Storm, or Flink, amongst many others), and b) if you need latency-aware cross-timezone you should probably be letting another piece of distributed infrastructure, explicitly designed for this, handle the issue, for example local/cluster/rack/data-center aware Cassandra. There is never any reason to re-invent something which big, multi-national corpos have worried about for you for a long time already. It doesn't strike me that anything in the OP post has not already been solved for 99% of us, using the right, freely available and well-documented tools. Distributed computing is hardly a new technology.

+1. Just use Elixir.

It's not quite that easy. While Elixir/Erlang provide a lot of great tools for distribution, it still takes a lot of knowledge and effort. It's not quite a case of "Just use Elixir".

Reply on news.ycombinator.com