Hacker News Reader

Comment by jeremy_k | Hacker News Reader

Exciting to see that heavy numerical computation might be available to a language running on BEAM. It has always seemed to me like an area that people would point to where Elixir couldn't stand up to Python.

I've always dreamt that someday you might be able to reach for Elixir and get everything you needed for an entire stack of a company and it seems to be getting closer and closer. The building blocks for a web app have been around with Phoenix and Ecto for quite some time. LiveView was introduced to make more user friendly frontends. GenStage, Flow, and Broadway introduced ways to run data processing pipelines. And now we're seeing the initial steps to potentially have ML capabilities.

View on HN

I am almost convinced to go all in with the elixir ecosystem. Having been a victim of a similar type of thinking, however, I am not sure this is the best thing to do so I am skeptical.

Remember, Java and the JVM essentially have this today: front end frameworks, back end frameworks, data pipelines, big data processing, entire databases, etc.

In your opinion, what will make the Elixir flavor of this situation different?

I think Elixir has the potential of being like the JVM but for functional distributed programming. The JVM is fantastic and I do most of my work on it, but having an entire environment of consistent actors, fault tolerance, hot reloading, etc is a tempting proposition. Especially with the easy interop of languages & functional aspects. I don't think it will surpass the JVM, but I think it has a lot of potential in its own sphere of influence. Especially if the library support & technical contributions keep coming.

Code elegance and development joy, maybe? I know Java has a mature ecosystem, but I wouldn't touch it with a stick

Right I get that elixir as a language is more enjoyable than Java. The question I wonder if it is wise to go all in on a single platform for the sake of being uniform.

Why would it not be wise? Python did it, Java did it, Elixir can do it too. Uniformity is good for productivity, as long as the ecosystem stays healthy & maintained.

Well for example imagine your stack is in Go and enjoy the speed and maintainability and expertise, but you want to use spAcy and all the great data science and ML libs in python. Or imagine you are all in on Java and want to use PyTorch. Or you are struggling finding an api-gateway like solution and want to utilize Zuul. Or you want to use Edge Lambdas on AWS but have to write them in node.js. etc.

Not being statically typed could mean that there aren't long compile times for large projects.

More importantly, the functional style used with Elixir will likely mean shallow usage patterns that are more composable than the deep layers upon layers of 'architecture' that plagues long lived class-based OO designs.

My completely biased answer is I haven't written Java since I was a freshman in college (2009) and that was like a few months. I basically know nothing about the ecosystem.

I've written Rails professionally for almost 10 years and have been following / playing with Elixir since 2016. While Ruby / Rails are amazing and the ecosystem is well suited getting a project off the ground extremely quickly, I'm just a big Elixir fan and hope to work with it professionally one day. I'll parrot reasoning from some of the sister comments.

"having an entire environment of consistent actors, fault tolerance, hot reloading, etc is a tempting proposition." - Very tempting indeed!

"Code elegance and development joy, maybe?" - Coming from Ruby / Rails, I agree 100%

"Not being statically typed" - Again, I've worked mainly in dynamically typed languages. I've experienced typing in Javascript and wasn't the biggest fan. But don't get me wrong, Rails can be awful when trying to dig multiple objects deep to understand what the data structure of the return value is so its not completely off the hook.

If you need a library for something in Java, it probably exists. There's several languages on the JVM these days that give you full Java interop - Clojure being my personal choice.

I think Elixir's big selling point is getting the BEAM runtime wrapped in a more modern language with more modern tooling. Lots of the positives people list about Elixir are due to BEAM, which you can also get in Erlang.

> I haven't written Java since I was a freshman in college (2009) and that was like a few months. I basically know nothing about the ecosystem.

I was in love with Ruby and hated Java around 2009, but the latter has gotten so much better with IntelliJ IDEA, to the point where I have to admit that working with JavaFX from IntelliJ was likely my most serene programming job ever. To me, the whole ecosystem is only worthwhile because of JetBrains.

Robust IDE support for Elixir is probably a must-have requirement for many nowadays. I am really tempted by the design of Elixir and find Java/Go/C++ aesthetically repulsive, but I'll happily forget about that if I can keep my refactoring muscle memory.

Thankfully JetBrains isn't the only option.

Fair enough, all the OG IDEs from 2009 are still around. And on the lightweight end of the spectrum, VS Code has excellent tooling support for some languages like Dart/Flutter.

What's interesting is my company hired a bunch of ex-ruby/rails people who transitioned over to Python. A year later, we tried to explore elixir and all those ex-ruby devs were adamantly against adopting it - they figured async python would be better. Very odd, since I figure it would be easier to adopt if you come from Ruby... but perhaps the Ruby community isn't that into functional programming?

It was very interesting experience being a long-time Pythonista having to be like "Isn't this awesome language with a Ruby-like syntax great???" and being told nah, let's just use async python by people who had spent years writing Ruby code.

I guess it depends on the programmer. I've fallen in love with more functional style of coding. The idea of data in and data out makes me feel like it's so much easier to follow what is going on compared to having objects with side effects and mix ins and instance variables. I try to write as much of my Ruby / Rails avoiding the things I mentioned, but its pretty hard to avoid in a larger codebase.

I think it's important to note that as much really good work as TJ Holowaychuk did for the early NodeJS community, much of the speed of that work is that he was replicating the functionality of Ruby gems. All the lessons learned by the original authors come cheap, and you can do a good deal of head-down coding that way.

This also helps with recruiting, as the tools work similarly. I didn't spend a lot of time in Rails, and decided not to go that way again, but I did appreciate that I got to recycle some of that new thinking when I started writing Node.

Phoenix also seems to be designed to recruit Rails developers. You could do worse than porting missing tools and crates from Ruby, and as far as I'm aware nobody has nominated themselves as the TJ of Elixir.

(assuming you confused CJ with TJ) This just is not true Having coworkers who personally knew TJ, he was a brilliant mind in his own right, and even if you think that is true, reimplementing functionality in another language but sticking with that languages current design patterns, working within the confines of that languages quirks to provide a cohesive, fluent, and extensible API is no small task.

What the parent posted isn't a criticism, they never said it was a small task or that TJ Holowaychuck's work in the nascent Node ecosystem wasn't brilliant. Just that there is low-hanging fruit available when a new technology of the NodeJS type arrives. If someone has the drive and skill to build high-quality [re]implementations of a large number of key libraries, based on what is available in other (much more established) platform/s, then all being well, that person can do it quite rapidly as the problem space has already been mapped (which then in turn has a huge catalytic effect, particularly as all the APIs they create will follow common patterns).

I like Erlang and Elixir quite a bit. However, I do think a bit of caution is warranted. With languages like Java, Ruby or Python, you can probably do an ok job at any particular task. Maybe not a great job, but good enough to get something up and running. I think BEAM has some sweet spots where it really is the best, but they're not huge spaces, and it's good to be wary of people jumping into BEAM without solid reasons why.

In terms of what it's good at, we very successfully employed Erlang as the command and control language for some semi-embedded system medical devices. The runtime is predictable in terms of performance and memory, and fast enough for a lot of things. Where it wasn't, it's easy enough to talk to C. All the fault-tolerant bits are nice for a system like that where you want it to take note of a failure and restart the subsystem that failed. Lots of good logging, introspection and diagnostic tools exist.

For what tasks is the BEAM not "good enough to get something up and running"?

AFAIK, any modernish language is good enough to get something up and running regardless of the task.

In my experience the difficulty of using BEAM is directly related to how well your problem fits into the actor+deep copy model of the environment. For certain problems BEAM will be a terrible solution because you're going to spend a lot of time working around this. I also think this is what makes it easily the best choice for other problems that fit this model well.

In the middle you will probably have a large number of problems that it's ok-ish at. But not really sure on what the distribution is here.

>For certain problems

Can you give an examples? Respectfully, you were asked for tasks and instead gave a description of the category.

I can only think of real-time, or real performance limited software (example: rocket guidance software, aviation, etc). With these every modicum of performance per millisecond matters; but these constraints only apply in a few (relatively) small fields of software development.

I'm an Erlang enthusiast, but find myself writing C for (small) things where I need to interface with struct heavy library calls, and nobody else has built up an BEAMy interface. I could build it, but it would take about the same work to just build my project in C.

Of course, maybe there's a generalized BEAM to C call library I'm not aware of? (I didn't think to search for one until now)

Also, honestly, it's a little easier to have a C daemon than an erlang daemon. Just call daemon() before the service loop vs setting up an app file. Of course, you have to build hotload yourself from dlopen and friends, and you lose out on a lot of features.

Probably the best you're going to get for a generalized c call library is rustler or zigler (I maintain zigler)

Yeah, so those look like nicer ways to call C than writing a NIF, but they don't look like point to a C include and get the constants and a way to call the functions in the include (and use the types in the include. If I'm reading correctly.

I'm looking for the equivalent of Perl's h2xs, or something like that. (which incidentally, maybe I should be writing these things in perl ;)

challenge accepted: https://github.com/ityonemo/zigler/issues/240

Two ideas that might help:

Elixir releases are built in and easy to use. You don't have to write an app file directly for instance.

I usually use ports for C interface (rather than NIFs). I have a little boilerplate C and Elixir that I recycle to do the ground work and then it's just your specific implementation detail.

Yeah, so I'd write just as much (or likely more) C code to interface with the library and BEAM as just writing the implementation in C.

My particular problem was a back end for a medium-level multiplayer RPG (500ish people connected at once per world). The only real way I found to get it performant was to lean heavily into processes. But then you have a race condition problem and need to DIY your own method of synchronization because you're no longer using the process mailbox for that purpose.

In practice I haven't found the immutability of Elixir that useful for concurrency because data is deep copied when it goes across processes anyways. This is in comparison to Clojure where you can safely and perfomantly share immutable data across multiple threads.

I would have suggested ets tables + entity system.

From my testing, the performance characteristics of ets isn't that great - data still gets copied around when you access stuff in it. I tried using it in my exploratory tests and certain worst case, player-focused actions would take around 20ms, when using a regular GenServer with a synchronization system on top would take around 5us.

To be clear, for NPCs it worked fine. But players are hoarders and in an RPG that can result in a ridiculous amount of data being attached to them.

There are things where the Ruby/Java version will be faster to use, more polished, or simply exist in the first place. I mean, yes, you can probably kinda sorta get something working with BEAM languages, but I'd occasionally hit something where Ruby had a gem, and Erlang didn't have much.

> AFAIK, any modernish language is good enough to get something up and running regardless of the task.

A lot of languages that have more limited existing use have more gaps in their ecosystems. That means that to get something going quickly, you run into cases where you spend a lot more time building (or validating and tweaking existing but relatively rough) support infrastructure that you would just grab a well-tested, established library for in a language with a richer community and ecosystem. This is an area where the top tier ecosystems (Python, .NET, JVM) really shine over even second-tier popular ones like Ruby, and where both of those tower over things like BEAM, which has the additional problem that it has a fairly high impedance for incorporating C libraries, where many other platforms (some also with stronger ecosystems of their own) also more easily consume C libraries.

This extends both to broad application domains (e.g., scientific libraries) and to things like probability of having an idiomatic, language specific, well-supported client for a new service you want to tie in and incorporate.

I've been porting code across platforms and most languages cannot embed themselves into arbitrary runtimes, for technical or political reasons. Had to go with rust rather than higher level languages.

If you're running a server—sure, BEAM me up.

What logging tools are you using? We have a couple of Elixir stack and their logging functionality leaves much to be desired at the moment, compared to the Python standard logging library. I'd like to improve it but haven't found much guidance so far.

I agree with that, we had to replace the whole logging module that Phoenix gives you out of the box to provide for logging messages to one line, JSON format, logging both to stdout and file. This is an area that would be easy to improve. Also distributed tracing is not easy to get working or at least the libraries I've found.

I've worked in Elixir every day for the past 5 years.

Things I like using it for:

- long running services

- doing anything w/ concurrency

- setting up cron-like processes that run every so often

- actor model anything

- stateful services

- web development using Phoenix & Ecto

- parsing

- encoding/decoding

Things I don't like are primarily related to it being a small enough community. That means libraries for third party services aren't always the best, and using it in the enterprise means that the smaller talent pool is less attractive when the company's priority is to use proven/boring tech for which you can hire tons of developers. I also don't think Elixir has a clear value proposition when you're writing small, single purpose functions for a serverless architecture (even though I really like writing it), or when you're company decides to go all in on Kafka for communication between micro services.

Just my two cents.

I think those are all pretty good fits.

The single unix process that BEAM uses is more conducive to some kinds of systems than some of the hackier-feeling solutions for the Rails ecosystem to tie different pieces together.

> or when you're company decides to go all in on Kafka for communication between micro services.

What's the issue with Kafka and Elixir?

There isn't an issue with Kafka and Elixir, it's the inherent complication around using Kafka when a lighter weight solution (Rabbit, PG) would work perfectly fine.

I hear the 3rd party library complaint a lot, but as someone who has been writing elixir professionally for about as long as you have I just haven't seen that. Speaking of kafka, the clients aren't as far along as the official clients, but the same can be said for almost all languages.

Being able to utilize erlang libraries means there are a ton of heavily production tested libraries you can easily drop into an elixir project.

My current job uses a mix of Java & Elixir. I definitely feel the pain of using the Elixir client when the Java client offers so much.

I also encountered some pain with the AWS client for Elixir when our company implemented SSO. There was a lack of support for a certain flavor of environment variable if I recall correctly. It's not horrible, just a pain.

sure, but which languages would be capable of dodging the "not enough libraries" complaint if java is the baseline? Maybe 2 or 3?

> C#/.NET, Go, JavaScript/Node, Ruby, Python, C++, and PHP probably all do way better than Elixir in third party support.

hard disagree on all but c++, .net and maybe python

I think people forget erlang has been around a long long time and I've written ruby for years and can speak with great confidence that the equivalent ruby libraries in elixir are typically much higher quality

Reply on news.ycombinator.com