Hacker News Reader

Comment by lod723 | Hacker News Reader

I've read that blog post and some of your other posts on HN. I get why the JVM, C/C++, and Go were not fits. However, I have not seen a lucid explanation of why you didn't go with Erlang or Elixir.

View on HN

That's a complicated topic. The short version is:

We looked at Erlang. Several of us are friends with folks who worked at Basho on Riak and we talked with them about our performance goals. They were very skeptical that we could meet them using Erlang. Based on that, we moved on from Erlang.

It's worth posting the long version. I've been following the various Pony blog posts with interest (I'm a both language geek and a distributed systems geek), but I always come away with the notion "Huh, kinda cool, but why didn't they just use Erlang, it'd be a great fit for this".

So either:

a) Erlang is not a good fit, and I'm wrong. Then I'd really like to know why I'm wrong!

b) Your friends at Basho led you astray. Would also be interesting to know what happened in this case!

Either way, without knowing more details, the short version you just posted is inconsistent with the claim that you guys did serious research into existing language ecosystems before going your own way.

I too would love to hear a long-form answer to this. Erlang seems like a good fit for this problem, besides maybe packaging up the client in an easily usable fashion.

"this problem" is very broad and there are aspects of it that Erlang is indeed a very good fit for. There are aspects where it is less so.

I think this is a rather in-depth conversation where HN comments aren't the most productive mechanism. If either or both of you are interested in chatting more on this, my email is sean@wallaroolabs.com. Drop me an email and we can arrange a time to chat.

Erlang, while having many virtues, is simply slow. Once, I reimplemented in Elixir a toy data science tool I had previously built in node. Idiomatic node, idiomatic Elixir, both written for readability. The Elixir was approximately 100 times slower than the node version.

Now Erlang often feels fast, because of the architectures it allows, but when you get down to shuffling bytes around or doing low level math it is currently slow, slow, slow.

Given Wallaroo's speed goals, I would have been really surprised had they used Erlang:

http://benchmarksgame.alioth.debian.org/u64q/nbody.html

I mean really though you shouldn't be using BEAM languages for scientific and computational tasks. For starters, there isn't a native array type (everything is lists). That's fine, because lists can give you flexibility while preserving immutability guarantees when passing across functions.

If you're doing an n-body simulation, then this benchmark is a good benchmark for deciding whether or not to use erlang/elixir. If you're doing a server which is mostly parsing JSON inputs over HTTP and spitting out more JSON with HTTP, and needs to handle thousands or millions of parallel connections without hiccuping, is an nbody simulation the right thing to use as your benchmark reference?

It's well known Erlang is not suited for data science and number crunching. I'm curious why you even bothered, near every introductory guide I've read makes this clear.

So that's not a good example of Erlang/Elixirs performance which is hardly known to be 'slow'. The language and process/actor model is far faster than many other languages particularly in the web space.

The author's also mentioned heavy dependency on the actor model as a performance optimizing strategy and optimal code structure which is why it likely is worth fully exploring for the OPs problem.

Reply on news.ycombinator.com