Languages on BEAM, the Erlang virtual machine
github.com
github.com
It has a lot of advantages:
- it is an easy to use, well-designed functional programming language, that happens to boost productivity and be (a lot of!) fun at the same time
- it is modern, with modern tooling around the language (e.g. mix, edeliver)
- it has Phoenix, a modular web framework, ideal for APIs and traditional MVC, very modular
- it is rock-solid for a relatively young language
- stack traces are clean, readable, and point you in the right direction most times
- performance is decent on a single-core, but since it's basically Erlang, it scales better on multicore than most solutions out there
- it has a world-class concurrency model powered by OTP, but more approachable to the average programmer, lots of high-level concurrency abstractions built-in
- Umbrella projects for those who need or want a truly modular application without going all-in with microservices
- it is the only language besides Erlang proper, that runs on the ErlangVM that seems to be getting a lot of traction (or at least a lot more compared to the alternatives)
- the community is very welcoming and open, lead by José Valim and Chris McCord who are smart, pragmatic and very nice people
> it is modern, with modern tooling around the language (e.g. mix, edeliver)
Sort of. Mix is a nice package manager. Edeliver is a series of hacks on deliver, a five-year-old Capistrano clone. But don't expect Elixir to fit into 2015-class deployment systems just yet.
> stack traces are clean, readable, and point you in the right direction most times
Other times, the error happened in another process, all you see is a timeout, and it takes some spelunking. Not uncommon in my experience.
But I'll also add a couple of points:
- the docs system is fantastic, in particular the automatic generation of handsome docs websites, and the `doctest` system which allows example code in your docs to double as test code
- I'm not sure there is anything better than supervisors for isolating and recovering from errors. Defensive programming is one thing, but sometimes your RAM stick gets hit by a cosmic ray. Shit happens; have a plan.
Now that I have it figured out it's totally painless to upgrade, especially because of the Kubernetes tools. Simply make a new release, build a new container, push it to your docker registry, and then do a rolling update with kubernetes using the new version of the container. You can also add/remove scale on the fly, libcluster handles it all.
[1]https://github.com/bbhoss/k8sexamplephx
[2]https://www.youtube.com/watch?v=SNll1r6ZtO0&feature=youtu.be...
So far I've preferred to stick to the Erlang way of doing things because it let me be laziest and it hasn't failed me, but someday I would like to see tools that allow the Erlang way and the e.g., K8s way to work together.
On a side note, if you work with Ansible, HashNuke's project is worth a closer look even if do not end up using it. It's well-structured and shows how an Ansible role can do both server configuration management and deployment. The pattern works really well; the only downside to this particular implementation is that it has the server pull your code from a Git repository, meaning the server must have access to it, instead of having your development machine copy the code to the server. I'd prefer the latter for security reasons and to match how Ansible works. If that's relevant to your interests, I wrote a hacky Ansible task that shows what that might look like with the same deployment pattern: https://github.com/adhokku/adhokku/blob/master/tasks/deploy-....
Edit: Added a note about the single host.
With what you've described -- a package manager that works -- it would seem that it fits pretty well. By 2015-class I'm thinking those systems, like buildpacks, that leverage dependency metadata to build a deployable tarball. What are the parts that seem missing to you?
This is sort of my big reason for constantly wanting to sit down with erlang/otp/beam. No matter how good your Ada is sometimes you want software that can fall over elegantly.
http://spectrum.ieee.org/computing/hardware/how-to-kill-a-su....
Whether this is frequent enough to warrant consideration depends on your application, but there are definitely people here who care enough about how their systems recover after errors of this sort that it's not an absurd suggestion.
My issue is that the original commenter listed "resistance" to bit errors as an advantage to using Elixir vs. other languages for web dev.
The question then becomes: who the hell needs bit flip resistance when building a web app?
Elixir made a decision early on to not deliberately wrap Erlang libraries that were perfectly sane as they stood. For example, to get a simple queue construct in Elixir you just call the Erlang queue module.
Once you're up and running with Elixir then there is nothing stopping you delving deeper into Erlang and OTP.
Also, I personally vote for the term for "Elixir user/enthusiast" to be "alchemist".
I have been using Go almost exclusively on the backend for the last 18 months or so, and it's a huge safety net. Go is nothing new, of course (it's basically Borland Pascal/Delphi -- which I also used for many years -- with GC), but it brings to the table a strictness that is very much needed (not to mention very good single-core performance).
Almost all the problems I encounter in large-scale Ruby programs have to do with wrong types or wrong shapes of data being passed around. It's an antipattern, but we have a lot of code lying around that uses Ruby hashes (dictionaries) instead of classes, because it's easy to do; but even if you create classes to wrap you data and validate the type of every single piece of data you pass around, the surface area is too large to cover perfectly. Not to mention that Ruby suffers from its own icky form of null pointer problem, the nil value.
And of course, static typing has many benefits -- not least allowing programmatic introspection (autocompletion, automatic refactoring etc.) in ways dynamic languages will never permit.
I know Elixir mitigates the lack of static typing with pattern matching, guards and the possibility of using @spec together with Dialyzer, but I'm not at all convinced that this is sufficient.
Using supervisors to recover by retrying allows you to correct for instability (network faults, oversaturation, locking and such) where a retry might actually fix things. What I am describing is something that cannot be retried in the first place; the next attempt would try to feed the exact same piece of wrongly-typed data back into the supervised process that would then fail again, and so on.
Ruby doesn't have anything like OTP, but these are two different problem spaces. Elixir, as far as I can tell, is open to malformed data/types at runtime in exactly the same way that Ruby is.
Using supervisors to retry operations is going to make your system unstable. Supervision tree is not about handling semi-expected faults, it's about starting things at boot time and restarting them in a predictable manner (along with any dependencies) should any of them crash unexpectedly.
I believe static typing is just a must for current high-reliability software and going with a dynamic language just seems short-sighted.
The remaining languages are mostly JavaScript and Python, but the former survives, as JavaScript is mostly used for UI code and people expect less quality from GUIs and the ladder as it is status-quo for scientific computing and a swiss army knife, as we have so many libraries across so many domains.
Dynamic languages are excellent for prototyping (where you keep your constraints in you human head) and errors are not critical. There once was an argument for dynamic languages as they allowed for succinct code, but nowadays modern languages can be correct and succinct code, as we learned to exploit theorems about computation and type theory.
[/static hat off]
[1] http://web.cs.ucdavis.edu/~filkov/papers/lang_github.pdf
That is not true, there are real benefits and classes of bugs, that can be avoided before runtime. Historically there was the drawback that inferring these bugs has been computational infeasible for some programs, but nowadays the situation is much better.
IMHO the only positive aspect of dynamic languages is their syntactic beauty and existing large eco systems.
I admire Elixir's and Erlang's approach to concurrency and think this model (do not share data and engineer for failures that will happen in distributed systems) is great, but could be combined with static typing.
Another way to state the advantage of expressive languages is that the less code you have to right, the less bugs you will have. And the less code written, the less code needs to be maintained, the less needs to be read to understand what's going on.
Of course this all depends on writing clean code, not obfuscated hacks or undocumented messes with needless complexity. But that goes for all programming.
In particular, modern type inference allows you to practically eliminate type declarations and approximate the terseness and expressiveness of dynamic languages. Recent languages such as Crystal and Nim have demonstrated that this is entirely feasible. (I mentioned Go earlier, but Go is, if anything, regressive when it comes to language theory, and not a good example of how to do things right.)
The Lisp example is only convincing if you've not seen what a really impressive type system can do; Haskell and the ML family come to mind, and Rust and Swift both seem promising in that regard, as does Crystal.
"Less code means less bugs" is a powerful meme, but it's a misleading metric in the context of type systems. Dynamic languages necessarily require less code since they eliminate information that otherwise allows you to fully reason at compile time about the program's future execution, thereby eliminating the ability to guarantee its safety; and the opposite is also true: "less information means more bugs".
Ultimately, it's less logic we want — less decisions that can go wrong. In Lisp, or any other dynamically typed language, you can pass a variable of the wrong type to a function and won't know it's wrong until you run the program — how did fewer lines of code help you? The equivalent program in a statically typed language might be slightly longer (though it could also be the same size), but the additional type declarations have made your program less buggy, not more.
They're talking about a combination of using battle tested code and writing at a higher level of abstraction.
That's a big problem, indeed, if your goal is to ship code that was never executed.
I wrote about this in another comment, in the context of writing tests.
Or the property 'this variable is never used after a free' in a language with manual memory management (Rust encodes this) or 'this program is free from data races' (also Rust)?
No one is arguing that strong static type systems make it impossible to write programs with bugs, but eliminating whole classes of bugs at compile time certainly makes it easier.
That's cute, but no. There's a lot of code that only gets executed once in a blue moon and if my experience[1] is anything to go by, this is where types really shine. (Before you say "tests": The amount of code bases I've experienced in dynamically checked languages with no meaningful test coverage is shocking. That, and tests can only show the presence of bugs.)
Types are also hugely valuable as compiler-checked API documentation, especially if side effects can be document as in e.g. Haskell or Idris.
(I'm obviously assuming non-trivial type inference as in e.g. Haskell or O'Caml. If we're talking anemic type systems like Java or C#, then the trade-off becomes a lot harder to justify, at least for me. In these cases you can indeed remove a non-trivial amount of "ceremonial" code and the type systems indeed have very few useful guarantees like non-nullness and such.)
[1] We're all trading anecdotes here, let's be honest about that.
Dynamically typed languages.
> In particular, modern type inference allows you to practically eliminate type declarations and approximate the terseness and expressiveness of dynamic languages.
Not of dynamic languages.
> require less code since they eliminate information that otherwise allows you to fully reason
The idea of Lisp is to have a lot of information at runtime/development time. One develops running software, not a program as text. The running software gives the feedback.
These discussions are thirty years old now. Since then everyone saw the type systems of the day as advanced and modern. Haskell is actually also quite old now...
Lisp and Haskell are simply used in completely different ways to develop software. I would also bet that there are much larger Lisp programs than Haskell programs in use. A large Lisp program can be several million lines of code, like some Lisp-based CAD applications.
I tend to find it significantly harder in dynamic languages. Yeah, there is way more flexibility in small decisions (like right now, is it easier to return a string or an integer or whatever) but it's not free. Your little decision interacts, directly or otherwise, with potentially the entire rest of the program that might ever exist. There is probably a best choice, and it's hard not to try to find it every single time. Doing the easiest thing right now might mean you need to go and adjust many other things. In order to know what is overall easier you need to keep the rest of the program in mind.
I find dynamic languages kind of exhausting to write anything but one-off scripts in. I can't make good choices without holding far more of the program in my head than I would in a static language. Sometimes I don't even realize I'm making such a decision because I don't have a correct or full view of the rest of the program. To make matters worse those errors won't even show up as being a problem at the decision-site.
I feel like I express myself much easier in a language with a strong static type system. I can write down clearly exactly what the model is, and then the compiler or types let me know when I try to stray from it. From there I can decide if it's better to adjust what I'm doing in the small or the large.
The entire history of computing is abstraction away from the machine, and offloading as much of the work to the machine, so that humans can focus on solving problems instead of worrying about lower level details. Dynamic languages tend to be better at that.
Now if you Haskell is the comparison, then maybe not. But Haskell has an advanced type system with excellent composability, so it's quite capable of expressing high level abstractions and domain specific code.
That's the general idea, but clearly not everyone agrees. Or perhaps, other concerns are considered more important.
4.1 will officially support .net core as well. I hope to be able to use it for work some day.
It's also commonly accepted — and, in my opinion, true — that dynamically typed languages move the burden of verifying your program to tests. I waste a lot more time in Ruby tests throwing bad data at my code than in Go, whereas in Go I can concentrate on real use cases, because static typing eliminates a whole class of abuse. In other words, the language forces me to do my own type validation at runtime and in tests.
Moreover, one big reason why static languages are more robust is that static typing changes one's entire approach to handling data. For example, in Ruby it's common to just do JSON.parse(raw_string) and then pluck out whatever it parsed (and again, the burden is on the programmer to verify that the parsed data structure is the correct one). The equivalent in Go is to declare a schema in the form a compile-time struct annotated with the field names that you expect as the input; the unmarshaler will catch type errors and blow up if something goes wrong. In other words, there's a completely different methodology: When Go has parsed your data, you know all the types are satisfied, whereas Ruby is stuck with duck typing. Maybe someone clever has written a strict JSON library for Ruby, but I don't know of one. Ruby libraries tend to be very un-strict.
The dark side of duck typing is that anything you don't explicitly recognize can get ignored. If you have a method with "def foo(options = {})", and someone does "foo(non_existent: 42)", nobody will typically complain. This sort of thing is why ActiveSupport has "assert_valid_keys", and why Ruby now has proper keyword arguments (which only solves the problem of valid keys, not valid data). This stuff is hard to catch and leads to code rot, which is a big maintenance problem that exacerbated by the fact that if you change or remove an argument like this, you won't know if it fails until you run the code, and "grep" is the only tool you can use to find out who actually uses the code.
Pretty convenient for input validation in runtime or output validation for tests.
Bonus point, go scalability is limited to your server. In elixir you can remove server boundaries.
For a new product, I will definitly bet on elixir.
That way, you get the expressiveness of Ruby (mostly) with the benefits of static compilation. It also has macros to make up for some of the missing metaprogramming.
Its great but may not quite be ready for production unless you don't mind fixing the occasional breaking changes. Its supposed to hit 1.0 this year.
Of course if you're looking for functional on Beam, then neither Go or Crystal are good comparisons.
Elixir actually has things like behaviours and protocols instead of Ruby-style reliance on duck typing.
Go type system, like its predecessor Java, C and company do not help for that. These are not sound type system.
At best they are help for the compiler to decide how to allocate memory or to try to get polymorphism.
I really have not found this to be the case. You'll get obscure unmatched function clause errors coming from the opposite side of your program, that can eat up most of the day trying to track down, ending up having to carefully look at the function guards deep in library code to attempt to figure out where things broke. And not to mention the parse errors - completely unhelpful coming from languages like Elm and Rust. And then there is the huge overuse of metaprogramming - just use functions dammit! And how you have to go trawling through the source code to figure out what `__using__` is going to dump into your module. And cargo-culted Ruby syntax that's pretty hostile to FP...
- good marketing
In another thread, their CTO claimed they have more users than Slack.
I understand that it runs on BEAM, much like I understand that IronPython runs on the CLR.
However, IronPython's performance is slower than C#'s performance due to the dynamic nature of the language, even with the special builtin support that MS built in.
What I was asking is whether or not Elixir's performance characteristics were significantly different from Erlangs.
Those are different cases. Your question is more like "what is faster: C or Fortran?" than "what is faster: C or Python?".
> However, IronPython's performance is slower than C#'s performance due to the dynamic nature of the language
No. It's slower because IronPython is an interpreter, while C# compiles to byte code, not because one is statically typed and the other is dynamically typed.
It dynamically generates IL which is then JIT'd.
P.S.: If you're learning a new language, as a deliberate practice exercise, learning that language should be hard to you, or it won't be as effective in making you better.
This is not a hard language to get. That's not it's purpose. It's practical if you want to adhere to the actor system, functional everywhere and so on.
Most of the action around Erlang (conferences, etc.) seems rooted in nostalgia not in finding ways to connect to new users or new use cases. Looking at the speaker list for Erlang Factory 2017 I'm trying to figure out why there's nobody from Whatsapp, MZ, Facebook, Amazon, Visa, Goldman Sachs, etc., and why so many of the speakers are repeats of years past (in some cases multiple times over multiple years past).
Despite having a ~15 year head start, and some really high-profile use cases, it seems like the Erlang powers-that-be haven't figured out how to capitalize on much of that, and it's a real bummer for people in my situation (and I'd argue the industry as a whole is worse off for it too). I'd really like to have my team use Erlang, but I'm feeling increasingly irresponsible suggesting they invest in it instead of Elixir.
But it needs new users and new use cases to stay relevant and to be defensible as an investment of time, money, and people in companies and enterprises that are willing to depart from the traditional strongholds of Java or whatever the latest thing out of Google is unfortunately. To say nothing of the more risk averse institutions, which are vast and numerous.
Given how great it is I'm often at a loss for why it's ecosystem feels stagnant and much of its evangelism (what little there is to begin with) appears unable to build much excitement. :-(
Anyway, back to Erlanging and hoping more people discover how pleasant, simple, and robust it can be.
If a company gets its major apps onto BEAM one could imagine a fairly straightforward auto-scaling story. As the number of scheduled BEAM processes grows for a given app, the need for CPU and memory could be expected to grow more or less linearly with it.
Languages with great support for these hardware features will be the languages of the future and today.
Elixir is definitely a great bet, you'll max all your cores and not break a mental sweat about it.
Really compelling stuff and great developer UX.
Does anyone have experience with this?
It should be pretty far down in the list of things that influence your choice, imo, since it's basically a solved problem.
I can honestly say I've not been this excited about a language in quite a while. Working with Go is pretty straightforward... but it's a complete bore. The more I dig into Elixir the more I desire to use it full-time.
If you need any specific help, feel free to contact me!
Disclaimer: I'm the author of exrm and distillery, so I'm probably biased.
I found that besides hot-code updates, there is not as much magic as one could think, and once you "grasp" the idea of a build machine, it should be pretty straigt-forward. Throw in an HAProxy for rolling deploys and you have a sweet zero-downtime solution!
Feel free to contact me if you need someone to help out (from some free general tips and tricks to supporting your team as a consultant).
It is getting better but right now the few things I can recommend the most are:
• learn what an "OTP release" actually is and what it means. This alone will save you days of your life.
• do rolling restarts, doing live upgrades is difficult and not necessary for the vast majority of applications.
• read the prod.exs config, there's a commented out setting you'll need to enable. I forget the name of it, but it's on my github "tehprofessor" with the repo "flying with Phoenix" (its probably outdated in many ways but that setting is essential).
• remember the application is compiled and then run, so you really won't get any runtime configuration (likes Rails' environments).
• Docker can alleviate build pain a bit, as it stabilizes your environment-- I don't use it myself but it makes building a bit more pleasant from what others have told me.
• you'll very likely need to include all the applications, including ones listed in your dependencies (it's a pain in the ass), in your mix.exs applications list if you're using exrm.
... it's been a couple months as I've been on a coding hiatus for health reasons (and a broken arm) but I believe most of the above should still be very true.
That said and even with all the bitching I've had about it, I still love Elixir and Erlang. Though I've been moving more to pure Erlang as time goes on (FWIW).
Edit: clarity and note about loving me some elixir/erlang.
I think the second part is still possible, since if it can be done for Erlang it can be done for Elixir.
-- Original post --
I haven't found production deployments to be particularly painful, I'd be interested to hear what kind of troubles you've run into.
Runtime configuration is definitely possible - you can deploy configuration files with your application and determine which one you want to load. It's not built-in, but it's a few lines of code to do it manually. You can also always use environment variables, which is arguably a better approach.
0: https://github.com/HashNuke/heroku-buildpack-elixir 1: https://gist.github.com/joakimk/48ed80f1a7adb5f5ea27
The main benefit of Go, I think, is that it can cross-compile builds. Erlang and Elixir don't have any way to do that, so if you develop on Mac and deploy on Linux you'll need to build your release in a VM or on a build server running Linux.
And it looks like you can cross compile BEAM anyway. http://erlang.org/faq/implementations.html. It's just a C program isn't it so why wouldn't you be able to?
The goal of a release is to bundle together everything you need -- your program, the VM that runs it, your config files, and any instructions necessary to upgrade/downgrade from another version of your code.
The beams themselves are cross-platform bytecode, but in practice beam files are not distributed on their own (except for patching). Typically you'll ship them as part of the released package which includes ERTS, your application, and all required dependencies (system libraries, other apps/libraries, and any native dependencies you've introduced). It excludes anything you don't declare as necessary from the final release package
In practice, people will usually use either the official release tooling or third party tooling to create platform-specific releases.
[1] http://erlang.org/doc/installation_guide/INSTALL-CROSS.html [2] http://www2.erlangcentral.org/wiki/?title=Cross_compiling
A deployment that can autoscale is harder to accomplish, but still not impossible. AFAICT, most people doing this are building releases with Distillery, wrapping those releases in a Docker image, then using the deployment and orchestration tools of their choice. Monica Hirst gave a talk about doing this with AWS CodeDeploy last year at Empire City Elixir as well[4].
[1] https://github.com/bitwalker/distillery
[2] https://github.com/boldpoker/edeliver
Flynn is amazingly easy to deploy. It provisions DO/AWS/Azure then bootstraps a single node or cluster literally at the push of a button. Then you just add it to your repo and push (provided the buildpack is already set up)
http://marianoguerra.org/talks/beamba-buenos-aires-meetup/#/...
follow the commits on this repo from the beginning to se how a fairly complete language is built
For some things: Rust. For other things: BEAM/OTP
Joxa and CSCM are on my to-do list now (especially the latter, since it's supposedly a proper R7RS on BEAM) for that reason, but I haven't gotten around to giving them a serious go yet.
https://notamonadtutorial.com/interview-with-robert-virding-...
I went to a couple of meetups at OpenX. It's majority older programmers.
The topic of the day was how to attract new programmers to Erlang.
I've stated perhaps we need to emulate Ruby and have some killer framework such as Rails. Their community is very vibrant.
The response was no, stay the course, we don't need to do anything. What's the point of this talk of attracting new programmers to Erlang if the consensus was stay the course and do nothing?
This way back before Elixir was on the radar.
Stopped going to that Erlang meetup, the programmers were out of touch.
I am mighty glad that Elixir and Phoenix is taking off. Shows them that I was right. Either Erlang community change their mindset or stay the same.