A response to "Erlang – overhyped or underestimated" (2010) | Hacker News Reader

A response to "Erlang – overhyped or underestimated" (2010)

jlouisramblings.blogspot.com

64 pointsTomte63 comments

I think the blog post is guilty of the same thing we're all sometimes guilty of when discussing something we like, but few other people seem to appreciate. It's this "no, it's great, you just don't understand it" mentality. We get it, they don't, and if we explain the genius of it one more time, the world is going to see the light.

Erlang has been around for a looong time (it's about as old as C++). It never rose to any real prominence, nor has it dominated any well-defined niche. Sometimes, outcomes like that are just a matter of bad timing. But sometimes, people just try your thing, don't like it, and the best response isn't "actually, you just don't get it" - it's to learn and iterate.

I thought it dominated, or was at least popular within, the niche it was developed for, telecom hardware. It is the "Ericsson Language" after all

Is that not the case?

Not really - at least in my days in that industry, the bulk of telco equipment was running C/C++ or Java code. This included Ericsson gear. Ericsson and one or two other telco companies were definitely using it in production, but it wasn't dominant.

No. In fact, Erlang was open-sourced because Ericsson gave up on it. From wikipedia:

> In February 1998, Ericsson Radio Systems banned the in-house use of Erlang for new products, citing a preference for non-proprietary languages. The ban caused Armstrong and others to make plans to leave Ericsson. In March 1998 Ericsson announced the AXD301 switch, containing over a million lines of Erlang and reported to achieve a high availability of nine "9"s. In December 1998, the implementation of Erlang was open-sourced and most of the Erlang team resigned to form a new company Bluetail AB. Ericsson eventually relaxed the ban and re-hired Armstrong in 2004.

I know this seems like a black mark on Erlang. I think it just demonstrates the difficulty of carving a new niche. Training developers in a new language and paradigm is hard.

Or someone could read it as the Ericsson management not really understanding the capabilities of their own internal tools, and must've read something up on Java or .NET in an HBR article.

I didn't like Erlang when I first tried it.

Then later, I found myself doing things in other languages … and I realized what I was trying was significantly less effort in Erlang and I was poorly reinventing ideas from OTP. That was when I pivoted my career to seek out teams that work with Elixir.

Some years later, when I finally got around to learning and using Node.js, I realized just how poor the async/await/promise design Node.js was (error-prone by design). But I highly doubt I would convince anyone to really give Elixir or Erlang a go.

In my case, it was certainly a case of "it's great but I just didn't understand it".

> Some years later, when I finally got around to learning and using Node.js, I realized just how poor the async/await/promise design Node.js was (error-prone by design). But I highly doubt I would convince anyone to really give Elixir or Erlang a go.

IIRC Elixir got a pretty big boost early on thanks to folks making the jump from Rails to Phoenix, so it could kinda be the case that most of the folks who *would* give it a shot *have already* given it a shot.

That being said, I have the impression that you'd have more success with folks using Node.js than you might think. I expect there would be more pushback from folks using, say, Golang and Java... if only because the wider dev community doesn't look down its collective nose at those languages quite so much. (Unless you're talking about generics or null values, respectively!)

I have tried with people using Nodejs. I didn’t really get much anywhere.

The wall I run into starts with, “what do you think of Typescript?”

I don’t try very hard though. At most, I put out feelers and see where it goes, because it is impractical for my team to switch to Nodejs. Due to the nature of the work, Python is a better choice for the team even if I think Elixir can still work better for me.

Golang/Java/C++/Javascript developer here. We do look down our collective nose. I've tried to get into Erlang and I'm a big fan of Joe Armstrong.

Erlang incorporates great ideas, but using it for a project seems a bit risky, since it is quite different from other mainstream languages.

Go is more conservative, but less risky. It's just a C for today's world. I've had lots of success using it, even though I'm aware of its warts (especially when it comes to concurrency).

That said, I'm kind of excited about Gleam. Let's see how that works out.

I completely disagree with you that Go is a C for todays world. Rust/Zig/Carbon is a C for today's world, Go is much closer to the new Java/C#

It's also good to remember Go itself wasn't always risk free though, every new tech has growing pains.

https://www.kickstarter.com/projects/2066438441/haunts-the-m...

The warts with respect to concurrency? That's the one thing Go does well! The real warts are associated with error handling.

> I realized just how poor the async/await/promise design Node.js was (error-prone by design)

It's so weird to me when people make statements like this and it makes me wonder how many times I've been swayed by someone's opinion online who doesn't understand the fundamentals of what they're talking about (or they do, and they just have horrible taste).

Node.js's asynchronicity is quite literally one of the, if not the, best part of Node.js.

> who doesn't understand the fundamentals of what they're talking about (or they do, and they just have horrible taste).

You have no idea what the person you're talking about knows or doesn't know, so your comment is ironic.

> Node.js's asynchronicity is quite literally one of the, if not the, best part of Node.js.

But enough about your emotional life. Do you have any interesting reason for saying this?

I'm surprised you think it's emotional. Seems pretty rational to me. I thought Node.js's selling point is the event loop and its non-blocking nature, and how convenient it is to be able to write both your backend and frontend code in a shared runtime. The arguments against that paradigm, mostly around callback hell and promise chaining, have gone away with async/await. So I stand by my assessment that these parts are probably the best parts of Node.js.

There are many other parts of Node.js which need work, but it's weird when the thing to call out is its asynchronicity when that part of the language is usually the thing people like.

Asynchronicity is indeed part of what makes Node.js useful, but that doesn't mean it's perfect.

It's very easy to write non-deterministic bugs with async/await. Bugs that might result in flaky tests and incorrect code getting checked in.

The best part of a terrible language and ecosystem is a pretty low bar.

“Node.js’s asynchronicity is quite literally one of the, if not the best part of Node.js”

That’s fine if that is your opinion. Here is my perspective on it:

- Node.js async model is not unique nor innovative. There has been several other platforms (such as Tcl/TK or 90s Windows programming). It runs into the same problem with any cooperative concurrency control, where a block of execution can freeze out everything else. In contrast, the Erlang/Elixir has a preemptive scheduler

- Nodejs’s async/await has an implicit queue of execution blocks. Erlang/Elixir, on the other hand, queues _messages_ recieved by a process. This allows not only transmission of data, but also control signals (such as suspend, stop, restart), which you cannot do with Nodejs promises alone. You have to use event emitters, whereas every Erlang/Elixir lightweight process can receive event messages.

- Nodejs promises can get lost. If it is not held and tracked somewhere, you can’t get to it. Erlang and Elixir has a first-class, PID literal. You can pull up any running process because the scheduler tracks them, which makes live debugging or mitigations possible in prod

- Because Nodejs queues execution instead of messages, it has to use callbacks. The problem with callbacks are not necessarily callback hell so much as tracking rejections and errors. Every single async call requires it, whereas in Erlang and Elixir, a running process is a natural error isolation boundary.

- Nodejs only occupies a single core of a processor, unless you use workers or fork. The BEAM runtime (these days) will use up as many of the cores as available unless you tell it not to. I can run a single BEAM os process, but have to jump extra hoops to take advantage of that for Nodejs.

Those are the main reasons why, from my perspective, Nodejs is error prone by design.

These are some great points. I'm glad I left that message :) It's always interesting to hear the other side

Whatsapp had 35 engineers and 450 million users when they sold to Facebook for $21.8 billion. Whatsapp, of course, was developed in Erlang.

There are times when the masses are just wrong. The Whatsapp story probably isn't possible with some other technology stack.

Of course, maybe it's just that very few applications need to scale in the way Erlang enables.

Ironically I think the original Facebook messaging did something similar, but they changed tech due to their inability to hire developers competent in the language.

So they hired "smart" people who couldn't learn Erlang, an easy language to learn?

I don't think hiring was the issue. See these Quora posts: https://qr.ae/ps8h6J

Discord was (still is?) built on Elixir, which was a descendant of Erlang.

It seems Erlang is comfortable in the niche of "real-time messaging platforms"

They've had some recent-ish blog posts about it

https://discord.com/blog/maxjourney-pushing-discords-limits-...

In my experience, lots of stuff needs to scale massively, but (outside of the better tech companies I guess?) mainstream tools/stacks/approaches are hangovers from n-tier arch days and not really right for it, so the majority of distributed systems are tyre fires.

> But sometimes, people just try your thing, don't like it, and the best response isn't "actually, you just don't get it" - it's to learn and iterate.

Elixir is Erlang with more sugar coating. The language and many of its associated projects (e.g. Phoenix) are regularly touted as some of the most productive development tools. You still don't see people flock toward them.

I used to think that large groups of people can't be wrong. Time and personal experience taught me otherwise. Now I know that sometimes it's possible that they actually just don't get it. But that's also fine.

Billions of people believe theres an invisible man in the sky, not metaphorically, but literally a dude up there with a beard.

It’s popular enough; Simply making something massively popular rarely makes it better.

I think functional programming in general still has a lot to teach most programmers that are writing code in the most popular languages today. That doesn’t mean I think they should be using different languages primarily. It means I think they should make their own environments better by learning what others have figured out before them.

Erlang has seen plenty of success IMO.

Erlang wasn't opened sourced or released to the public until 1998.

The problem with not using Erlang and Elixir is that people are continually inventing a worse wheel. If you need a scalable system controlling many things robustly and reliably, anything built to solve that problem will be an approximation of the solution provided by Erlang/Elixir.

> *90% of all internet traffic goes through Erlang controlled nodes (Cisco)

(source: https://erlangforums.com/ )

Seems like a pretty good outcome to me?

> But sometimes, people just try your thing, don't like it

People don't just try things. And they judge things they havent tried. They stick with what they know. They all got taught java in school, so thats what they use.

> In Erlang, the mantra is “abstraction is had by introducing another process”.

This is probably "not a good idea", as processes introduce bottlenecks. Really, if you want to abstract in erlang, you should, in 95% of cases really just write a behaviour.

Of course if you need to abstract over something stateful, yes, please write a process. The benefit being that you can do some pretty painless dependency injection (or at a minor readability cost compile time pick between modules) and inject the state in your tests.

I really have to finally take a look at Erlang, it's been on my list for years

For me, no language has been more fascinating as unlike most languages Erlang is very focused on distributed systems as a first class citizen and sells that, rather than the more common sales pitch of being exactly like other languages but with subjectively better syntax/semantics.

If you prefer a more fun syntax I would consider Elixir instead of Erlang. It's transpired to Erlang so you get the whole ecosystem's benefits. One of my favorite features of Elixir is the |> operator. I find myself desiring it in all other languages.

To be specific, Elixir isn't transpiled to Erlang the language. It's compiled to the Erlang Abstract Format.

https://elixirforum.com/t/getting-each-stage-of-elixirs-comp...

https://www.erlang.org/doc/apps/erts/absform

(In Elixir) I've most recently found myself lamenting the indirection of processes as a ton of third party dependencies spin off processes to manage their state, and when something goes wrong (usually some argument I'm incorrectly passing in to a function call or something) the stack trace I get doesn't lead back to my code so I have to try and figure out the closest point to where I'm invoking it. Probably still a good tradeoff for immutability and the actor model, but an unfortunate downside that has pained me.

I haven't found most 3rd party dependencies doing that unless they're making, for example, a request pool.

Should we be shaming those libraries?

Maybe so, although there's usually at least some justification for it, whether it's performance related or just as simple as "the API can get a lot better when it can track some state so you don't have to pass every initialization value in every time." I particularly don't agree with the latter most of the time, but I can see how they arrive at the gen server conclusion.

What libraries? You might be able to do something with the process dict depending on how the libraries are set up. Basically you create a process id that you use to track what's going on:

    @type trace_id :: nonempty_binary()

    @spec put_trace_id(trace_id()) :: term() | nil
    def put_trace_id(id), do: Process.put({:my_app_name, :trace_id}, id)

    @spec get_trace_id :: trace_id() | nil
    def get_trace_id, do: Process.get({:my_app_name, :trace_id})

    @spec make_trace_id :: trace_id()
    def make_trace_id, do: Base.hex_encode32(:crypto.strong_rand_bytes(16), padding: false)

It does require that, along the call path, you set that trace id whenever a new process is started. It gives you some visibility into what's going on, especially if you also do something like Logger.metadata(trace_id: get_trace_id()) after you call put_trace_id/1

That said, I think any library that does start processes like that should have some mechanism that lets us identify the provenance of the crash. "Crashed when trying to fribulate. Trace id xxxxxx"

Libraries in a single process are tricky because so many things are dependent on messages. So a piece of code running inside another process has to rely on the "hosting" process to deliver all of the messages it needs. So if it monitors another process it needs to have its DOWNs delivered to it. If it wants to use a timer it needs the timer messages delivered to it. If it wants to make synchronous GenServer.calls, the parent process needs to be aware of that and plan around it (i.e. realizing that it may be stuck while waiting on that call to complete).

A separate process can own its responsibilities.

A Library can just be a bundle of code. Suppose you're parsing xml. There is no fucking reason for that code to be in its own process, and it does not need to do anything with messages.

In this example, I could see a library parsing xml spin off a process so it can track state. I could also see a performance argument for not wanting to do potentially computationally expensives (or even I/O expensive) work on the main process blocking other things.

I'm not enough of a parser expert to know whether stateless processing is possible, but that's how I can see a simple xml parser ending up in a separate process.

"parsing xml spin off a process so it can track state"

Don't do that. Use state tracking data type.

"the main process"

That's not a thing in the BEAM.

For something pure compute (i.e. no need for connection pools), if your user really needs that, then they can run the xml parsing in their own spawned process. Even if it's I/O bound, Erlang VM already makes great decisions about yielding, you will almost certainly not do better. You should not make concurrency decision on the users behalf.

I drank the Erlang Kool-Aid around the same time this was published. In 2013 I worked for a company that had a few Erlang services (as well as some JVM services, a mix of Scala and to a lesser degree Java).

One thing I was tasked with was replacing the ingress data collector. One of the limitations of Erlang at the time was that all SSL termination was funneled through a single core. Once the Java replacement was deployed, we saw a massive decrease in latency, the p95s and p99s especially, and all the weird operational overhead of trying to understand what the Erlang VM was doing at any given moment.

Say what you will about Java and the JVM, but it's a fantastic platform for reasonably high performance servers. Erlang might have a lot of claims for high concurrency and scalability, but practically I've had considerably more success with the JVM.

I haven't touched Erlang since 2013, so in the intervening 11 years I can only hope that it has gotten better. Though I have zero interest in trying it again.

What is an ingress data collector and how is it different from a server?

It is indeed a server, one that supported IPFIX: https://en.wikipedia.org/wiki/IP_Flow_Information_Export

Well, now the JVM has lightweight processes. We may eventually see OTP reimagined on the JVM.

A response to "Erlang - overhyped or underestimated" - https://news.ycombinator.com/item?id=2039180 - Dec 2010 (33 comments)

Erlang - overhyped or underestimated? - https://news.ycombinator.com/item?id=2038392 - Dec 2010 (38 comments)

I wrote this 13-14 years ago. Time flies I guess.

Keep in mind, it's frozen temporally at that point in time, and a ton of stuff has changed in computer ecosystems since then, and so has some of my viewpoints.

Anything jump out that you would change?

Discussion about things like erlang tend to go around in circles because people don't separate out the two distinct parts, the VM and the actual language. The language is niche and archaic meaning that libraries, tools, ecosystem and even syntax is a huge problem.

The VM is what people really want, but for some reason no one ever talks about using the VM with another language or cloning the VM for another language instead of trying to force the part that people don't want (language) with the part that works.

Without meaning to detract from your point, "no one" is a bit hyperbolic. Elixir, Gleam, Caramel, and LFE all compile to BEAM, and Akka looks like a decent attempt at porting the VM semantics to JVM and .NET.

Isn't Elixir meant as a solution to exactly that problem?

Have you ever used Elixir? It is still a niche language with niche features and a niche ecosystem. "Immutability" might be a nice buzzword but in the real world computers don't work like that and trying to shove a square peg in a round hole results in complexity and performance heartbreak.

Erlang devs wish it was as popular as Javascript when it used XMLHttpRequest. It's less popular than Lua today, which has no package manager. That means something is very wrong.

> The main objection is familiarity: “It doesn’t look like Java!” I think the point is somewhat moot.

It doesn't matter what the zealots think. It's not moot, because there are people born every day that will think it. This will be a response to the language, every day, until we're both long-dead. It matters enough that it drove people to dedicate time to develop entirely new syntax (eg elixir), as mentioned. Given the myriad of languages that use a functional style, Erlang is just not special, excepting the unique syntax which makes it unfamiliar and elixir that just makes a bit more user friendly. Maybe, just maybe, it is partly about the syntax.

The Erlang tooling is primitive. I don't want to develop another toolchain and have to do with the gaps, thx. Yes it's a chicken-vs-egg thing, but without support for a new language, you wont see a lot of adoption.

The semantics are less efficient. Now the context needs to be carried around or in another set of terms, or worse another event queue, or worse overloaded fns? When a for-loop has to be broken up into 2 fns, you've made things worse at the smallest scale. I then have to communicate the naming and choices to future developers? This makes debugging, design and code reviews harder/more divisive.

I can read through a couple functions and understand what a c-like program does, even if it's in Python or PHP or Java or C (most of the time). Saying they are different in details, is missing the point. I don't have to know how data is split up because everything is either a structure or a scalar, serially processed and in static locations for reading...until you hit a DB. Hitting a single async store is complicated enough that it has caused developers headaches for decades. Introducing easy queues as a solution, is handing out footguns. The complexity introduced into event-driven applications, limits the practical complexity of application development in practice, by orders of magnitude (bottlenecks and race conditions).

Sloppy programming is jr friendly. Raising the developer performance expectations higher, is worse for adoption. Perl died on this hill.

Years after I last used Erlang to replace a Java chat client from 5000 lines to single page Erlang module, then got a job writing it (2009), I would not recommend it to anyone. I did try cracking open another person's project in 2012ish and that was not fun. At least we have Kafka.

This is my take as well (adding erlang to my list of things along with python/kotlin/go/lua).

Some things are just a little *too* different. There's no for-loop, it's just recursion. Pattern matching on the function signatures are nice, but then I have to dig around a lot more to find what I need.

That being said, I do want to try it and learn about it more; the promises it makes about concurrency and event driven programming looks nice, i do really want to try it.

I haven't looked at Erlang much but I've been using Elixir for a couple of years now and it's been the most enjoyable language I've used to date.

I don't miss having a for-loop, recursion works but the Enum module is great and easy to use. When done right, pattern matching on function signatures is nice and clean. If you find it's actually more difficult to find what you're looking for it's a sign somethings probably off.

> It's less popular than Lua today, which has no package manager.

https://luarocks.org/

They wish. More like a few things in Luarocks but mostly github.

This is a pretty outdated take considering all of the experience you've referenced is 10+ years old now.

I find this statement, unconvincing. Elixir and the improvements to the VM were the last big leaps. None of these things affected the issues that keep Erlang from being relevant.

Erlang is some inexplicably sacred cow on HN, among the few other forums who still discuss it. Making sweeping changes is going to be required, to get the adoption that some are seeking...specifically the authors of these kinds of posts. Waiting another couple decades for the killer app to catapult it, will never happen.

Nah, this take would be as bad in 2009 as it is today. I mean, it's evidently clear they can't wrap their head around functional programming, which is absolutely fine, until you jump to generalizing conclusions like this.

This elitist attitude, is another mainstay of the Erlang community that hasn't helped in the slightest.

What is the reason it's not popular again? My reasoning is not "a bad take", as much as it's my take from over 30 years programming. I would say it's my own twist on common understandings across the west coast of the US.

You probably have your own, equally wrong reasoning. Would be great to hear them.

Reply on news.ycombinator.com