Hacker News Reader

Comment by xcambar | Hacker News Reader

The Elixir ecosystem is growing incredibly well. I am impressed both from an engineering and a product perspective.

It shows that they are playing the long game.

I think they are achieving what I wished happen to the Clojure ecosystem: productive, well designed, respected and popular. Clojure missed the last step, unfortunately.

View on HN

It seems making a Lisp popular is an impossible task.

That said, I'm not sure if the data agrees with you. I think Clojure is more popular and widely used at this time. Not sure, but I think from what I remember of the few rankings, and just the fact I don't know an equivalent success story to NuBank for Elixir, I think maybe Clojure is at the moment more popular in practice.

But with the amazing learning material Elixir is putting out, maybe it won't last.

In terms of major companies using Elixir, Discord and Whatsapp are built with it. Whatsapp is slowly chipping away at its Elixir for infra homogeneity reasons with the rest of facebook but discord is still all aboard the Elixir train.

WhatsApp uses mostly erlang if I recall correctly.

Neat, I didn't know about Discord. Whatsapp I thought it was all Erlang, are you sure about Elixir being prevalent there?

I wonder if it's only a few teams handling major events at Discord, or if that's truly their default backend service language all their backend teams uses. If so, maybe I should look at their job listings :p

I think they use Go too.

They moved from Go to Rust.

https://discord.com/blog/why-discord-is-switching-from-go-to...

From the discord blog posts it seems that elixir powers the chat system, with rust and python as the other two main languages in their stack.

As for whatsapp, they are mainly a erlang shop and yesterday they open sourced a type checker for erlang:

https://github.com/WhatsApp/eqwalizer

Since Elixir just compiles into BEAM byte code, like Erlang does, can this type checker be used on Elixir code too?

I would think that NuBank would want to market and promote Clojure to ensure that there would be ample numbers of Clojure developers (since NuBank depends on a lot of Clojure code). Perhaps they do market Clojure in Brazil, but I don't see it elsewhere...

With 213 million people, they do have a ton of devs and a ton of ESL speakers. There’s lots of places like that, like Turkey or SE Asia, where word doesn’t reach NA/wEU.

Do you use pagerduty? They're an elixir shop.

I agree, and I think their move into ML is a genius move that is going to make elixir mainstream. I'm super excited for the future.

I've wondered whether it's easier to add data analyst stuff to Elixir that Python seems to have, or add features to Python that Erlang (and by extension Elixir) provides out of the box.

By what I can see, if you want multiprocessing on Python in an easier way (let's say running async), you have to use something like ray core[0], then if you want multiple machines you need redis(?). Elixir/Erlang supports this out of the box.

Explorer[1] is an interesting approach, where it uses Rust via Rustler (Elixir library to call Rust code) and uses Polars as its dataframe library. I think Rustler needs to be reworked for this usecase, as it can be slow to return data. I made initial improvements which drastically improves encoding (https://github.com/elixir-nx/explorer/pull/282 and https://github.com/elixir-nx/explorer/pull/286, tldr 20+ seconds down to 3).

[0] https://github.com/ray-project/ray [1] https://github.com/elixir-nx/explorer

Definitely easier to add Pandas to Elixir than preemptively scheduled green threads to Python.

Plus sane failure domains: go has had preemptively scheduled green threads from day one but failure domains are really not a thing in go.

Hm, I think I've read somewhere that both Go and Elixir and kinda cooperative. A process in Elixir can yield control after certain number of reductions (function calls I think) and in Go a goroutine can only yield control on function calls, so if you have an infinite loop just adding numbers it will run uninterrupted. Both of them are "less cooperative" than Python with explicit yield statement. Do I get this right? I started digging into concurrency not that long ago.

Yes technically they are not fully preemptive in the sense that an os thread is (the os sends an interrupt which halts the processing at the CPU level), but in both go and elixir the programmer has no control over when the context switching happens, and "function calls" which are the yield boundaries happen all over the place, so it's "effectively preemptive".

Elixir is in practice more preemptive than go (last I checked with go) because you cannot infinitely loop and lock cpu in elixir -- a loop requires you to tail-call in elixir, so that's a yield boundary, and I'm certain that in earlier go that wasn't the case if you `while true {}`

Yes, unless you're in a BIF or NIF every call is a reduction: https://blog.stenmans.org/theBeamBook/#_reductions so if you call out to native code or are using a VM builtin you can lock it up indefinitely but other than that it's not possible.

[I have no experience with Elixir.] Would that be possible to have elixir intelligently manage multiprocessing of python scripts? I would be especially interested in being able to have the scripts talk to each other, but have no idea how it could work, besides perhaps having them communicate by writing to / reading from the same files, which seems risky.

There is actually a case study on elixir-lang.org of exactly this!

https://elixir-lang.org/blog/2021/01/13/orchestrating-comput...

Very informative - thanks!

This is exactly the solution I was thinking of as well.

I think you can have the scripts talk to each other through Elixir via the BEAM/OTP approach: for each script you will have a GenServer module that manages it in a separate process. Call this module MyApp.ScriptServer and put "use GenServer" at the top of the file. This GenServer process will have a client API, which you can write, which will be how you interact with your script. In the BEAM programming model, processes communicate by sending messages to each other's mailbox, so you set up some logic for this and maybe have another module called MyApp.ScriptCoordinationServer that manages some global state.

This is just my amateur guess -- I'd be happy to hear an Elixir/BEAM/OTP expert chime in.

Thanks for the explanation!

> Clojure missed the last step

Oddly, it seems to me that Clojure (the established people in its community) don't really care if anyone is attracted to it.

It seems like the people who come to Clojure do so because they caught a glimpse of it somewhere, did some digging and searching, and decided to give it a try. Not exactly the same as Sun spending $500 million to market Java.

That's not to suggest the Clojure community is unwelcoming or silent, but they generally fly under the radar.

Several years ago I went to a few of the Amsterdam Clojure meetup group meetings, and at that time there were 10 to 20 people there.

Meanwhile, the Elixir meetup would have 20-40! And Elixir was still comparatively young. Of course the Ruby meetup group was even larger, but that's no surprise given the prevalence of Rails in business.

Because of the Java ecosystem, in Clojure you are in much more advantageous position than with Elixir.

Elixir also opens you up to the entire erlang and Beam ecosystems. Not as common as Java, I know, but still decades of knowledge and tooling around it cutting across many industries.

Seems like Beam is a more specialized tool than JVM but if you need that sort of functionality it's phenomenal. With things like built-in performant persistent key value stores and cluster management I'm kind of surprised more people haven't pushed through the "this is useful but lacks broad adoption" barrier.

Well… yes and no. A lot of things built in are pretty spartan and not always appropriate (Mnesia …cough…).

This said it’s the only environment I’m aware of that is built around the idea of cluster, not just a local process that may do some rpc to other processes.

You can find libraries for many things but they are not the same quality and breadth as the big Java ones.

I've heard this, but honestly to me it just seems "specialized for writing and running reliable software".

Seemingly reliable for a specific type of problem, though. Most reliable software doesn't require a distributed network to be reliable, and I don't think binary execution on BEAM is any more inherently reliable than the JVM, right?

The BEAM is designed for a software architecture approach called OTP in the Erlang community -- it is designed around trees of process supervision. So there is an entire robust framework for reliability really built in as a primary concern for the BEAM.

https://www.erlang.org/doc/design_principles/des_princ.html

If you need high concurrency, heavy traffic, real-time system with high resilience, there’s probably nothing out there that comes even close to BEAM. It is definitely possible to write such a system in Java or Rust, but most likely at huge cost for something that you can get out of the box elsewhere. BEAM has narrower scope, but in that scope it’s probably far ahead of anything else.

Except using Java ecosystem as-is would not be idiomatic in Clojure, which means you need an ecosystem of wrapper libraries. I recall someone making similar argument on some other post, on how it adds more space for bugs, performance issues, and still making it hard to work with. The context was in comparison to Kotlin, where using Java ecosystem is much nicer experience.

I tend think the opposite. Developing for Android I found the Java build process and maintaining a solid development environment a headache. If that follows on to Clojure I can see why it wouldn’t be as popular even if the language itself is overall better.

My experience would make me agree with you. Developer experience around tooling in Clojure makes a lot of assumption around prior knowledge of Java ecosystem.

Elixir, on the other hand, welcomes the newcomer better.

I see it the other way around -- because of the BEAM ecosystem and programming model you are in a much more advantageous position than with Clojure.

Reply on news.ycombinator.com