what might explain this? I mean hypes and infatuations are common but tangible advantages usually win the day because in the end people will pick the right tool for the right job.
is it that distributed systems are not actually as big a domain so it does not get much press?
No, they don't. They pick the familiar and the easy-to-hire-for. I've worked on systems (written in C++, Java, others) that would have been perfect for Erlang, but at most I was able to present a prototype. No one is interested in a better solution that they can't hire for or have to retrain their staff for. (The familiar and the easy-to-hire-for may, by some metric, be the right tool for the job, but it isn't necessarily the right tool by technical metrics.)
Not too long ago I interviewed with a company that only uses Erlang, not Elixir, and they rejected me because I am not an Erlang expert. There wasn't anything I could say that would change their minds. I even volunteered to do a coding challenge of some type & they still wouldn't consider me further.
Sometimes a talent shortage is a state of mind.
Wow... was this an actual hiring manager or someone, or like an HR person with a checklist?
I've interviewed for one or two Elixir positions where the opposite thing happened: I talked with someone who had a checklist. I haven't done Elixir professionally even though I have extensive Erlang experience and that was that.
There is a pretty stark difference between "has architected an application with OTP components" and "can write linear code in Erlang or Elixir", but even then you'd probably be able to fit someone in unless you're hiring for that kind of senior/architect position.
> No, they don't.
I'm in strong agreement. People pick the option that has a good story associated with it. Having a good story protects you in the event of a failure. And it protects your non-technical boss (or your boss' non-technical boss if your boss is technical).
I assert the following:
Scenario A: I have a literal mathematical proof that erlang is the best choice. I implement the solution, but due to the market changing (or an earthquake or ninjas literally destroy the server room) the overall project fails.
* Me: But, I had math proving everything out. Ultimate Boss: I don't believe in math, you're fired.
* Boss: Well, it failed, but look at this incomprehensible math that I don't understand. Ultimate Boss: You've obviously been hoodwinked, you're fired.
Scenario B: I have a really good story that includes phrases like: "best practices" and "industry standard", and is doing exactly the same thing as everyone else. The project fails because the tool isn't great.
* Me: It failed, but here's a good story. Ultimate Boss: Well, it looks exactly like everything else, so I'll look for a different nail that's standing out from the crowd to blame for this so that the muggles don't get nervous about taking non-chances in the future.
* Boss: Here's this great story that I totally understand and can embellish upon because it operates on storytelling and not category theory. Ultimate Boss: That's a really good story. I'll have to steal it myself to give to the board to handle some unrelated problems of my own. Better luck next time.
Even the scenario of master craftsman using erlang because it's the right tool for the job is more about a story about an uber expert who is off doing wizard stuff than it is about an actual merited set of reasoning.
The "nobody ever got fired for buying IBM" defense. Which has since been updated to Oracle, Microsoft, Java, C# (for a large portion of the enterprise world), Python, and Javascript.
Best-in-class that isn't mainstream is a hard sell to employers if they are strongly risk averse (which many people, if not most, are).
So, Elixir/Erlang combo seems like a perfect fit. Instead, we are having all remaining Elixir and Erlang systems ripped out and re-written in Java.
The reason is simple - readability and hire-ability. The cherries on the cake - Java's "good enough" for what we do, the tooling, the quality and availability of reference resources and subject matter experts - are just that, cherries. But the ability to hire and onboard new engineers is THE thing - it took us 3 years to fully hire out the headcount we were allocated for just our little team and that is in Java. Hiring for Elixir and Erlang has been such a pain that we are having to pay contractors while we re-write the current Elixir and Erlang systems into Java. Our domain is somewhat complicated - most people who have not worked in food retail generally would not imagine it to be so - so onboarding people to the domain knowledge takes 3-6 months and that is without "This is what? Erlang?" barrier. That is where readability of Java and non-FP vs Elixir/Erlang and FP comes in as well.
I guess what I am trying to say is that, yes, the right tool for the job and all that but good enough tool with a significantly higher hiring pool is proving so much better.
Elixir’s runtime, unlike many others, uses preemptive lightweight processes. Unlike async-only systems, a runaway process won’t block out other processes. That’s just the start of the kind of advantages in writing with Elixir/Erlang/OTP
It’s been battle tested for years. I think many of the hype cycles for distributed systems reexplore or reinvent what Erlang and Elixir has to offer.
A lot of it is unfamiliarity with the tool and unfamiliarity with the paradigm. And of course, you can do distributed systems with other tools that people tend to be more familiar with, so go with what you know. Erlang is adaptable, but was built for a Telcom setting with a lot more permanence than today's popular cloud model with nodes coming and going.
On the other hand, an awful lot of large scale chat systems start as ejabberd and adjust from there; sometimes rewritten in another language, sometimes rebuilt in company specific Erlang while the service is running, sometimes without muvh adjustment.
For example, you can effectively architect with microservices without actually creating more deployable artifacts. This is because each GenServer is already a microservice. You build it from the ground up reasoning through failure modes and coordination. Those are easier to separate out into its own deployable artifact when the time comes, if it ever does.
People hear about code hot reloainding (where you deploy new code without shutting down the production environment) and think that’s the killer feature of the BEAM vm. Code hot reloading is a narrow use—case of a more general capability — you can directly write or patch new code in a running system in a live environment through the REPL. That has been very useful for improving the system, to stop the bleeding or identifying bottlenecks and trying out fixes, even when deployed as ephemeral pods in Kubernetes.
I understand why Elixir was born.
I think Typescript would be "easier" for a larger group of developers to hook on
But the hard part of Erlang is OTP / understanding the concepts or rather unlearning the concepts from other programming paradigms.
Then again, languages lose adoption pitches based on whitespace and tabs significance.
If one can get past the initial impression though, they'll find it's actually a very simple, nice language. Much simpler than Scala/C++ and probably even Java these days. Also simpler than other functional languages like Haskell. I picked it up in about 1-2 weeks. There's very few gotchas in the language or hidden complexity. It flows well.
I'll also add the language suits the usage pattern. Pattern matching works very well with the gen_server approach. And the binary syntax is so nice if you need to deal with over the wire protocols, as is common in distributed systems.