Lovely Week with Elixir
ramblingcode.dev
ramblingcode.dev
But there are some great language features, like guards and pattern matching, that are hard to give up when you go back to other languages.
Plus it's great to have OTP goodies like GenServer at your fingertips if you run into performance bottlenecks (which you may not!). The OTP APIs are a bit weird coming from other languages but not too bad.
Other things I've liked:
1. Ecto is simply the best DB library I've encountered in any language. I'd almost recommend learning Elixir just to be able to use Ecto.
2. Plug provides great HTTP ergonomics highly reminiscent of Go's context-based middleware approach. Having direct access to the full request/response lifecycle is a win.
3. Phoenix is nice because it's essentially just Plug with some convenient helpers on top. Strikes a really nice balance between configuration and convention by letting you use only what you need. Haven't tried LiveView as I'd prefer to handcraft my own JS but probably worth a shot.
4. Absinthe is the best GraphQL framework I've encountered after many others in other languages left me completely cold.
It's interesting, people always talk about Phoenix, which is nice, but Ecto is really special. I often recommend it to anyone who is feeling burned by Active Record implementations.
(assoc-in changeset [:children 1 :title] "New title")Beginners will most certainly will have highly inefficient start with it.
There are others in Ruby world which needs more praise such as Sequel and ROM(which is what Hanami uses also).
With Rails having the monopoly and most gems having AR as dependency, it's hard to see where we will go regarding ORMs in Ruby world.
However, that "boilerplate" makes data access and return values super clear. It also allows you to segregate Ecto queries by domain, meaning in a User context you may only have access to certain fields, whereas in a data processing context other fields may be available. This is great for coarse table access control, and you get it almost for free. It also allows you to very easily validate data at the edges of the system.
As for the query language, I can say for sure that there's a learning curve but it's close enough that your 1st guess is usually correct in my experience, and it removes a lot of foot guns like n+1 queries by default.
TL;DR there's some learning curve and extra LOC to write, but you get a lot in return.
It might not be a plus if you're writing a web-app, or some other kind of easily-restarted "shared-nothing business layer" system. But get deep-enough into absorbing the Zen of Erlang (i.e. by trying to write a big-deal telecom system; or by working on a distributable Erlang DBMS like CouchDB), and you'll come to appreciate it. Specifically, you'll realize that it'd actually be impossible for the Erlang runtime system to support hot-code-reload plus automatic distribution plus static typing, all at once. One of them has to give. (I think this is one of those universal "choose only two properties" triangles, like CAP, though I don't think this one has a name.)
If you have a "living" distributed system—one that you don't bring down entirely for each upgrade—then inevitably you'll hit a situation where a node with the fresh new version of a module (V3, let's say), tries to use that module to send a V3 message to a node that is still only running V2 of the module. In a distributed system with static typing, that'll inevitably crash the V2 node—unless you did a whole extra "V2.5" rollout step to first teach V2 about the wire-format of V3 and how to handle it.
Much of the point of Erlang/Elixir's user-facing "data architecture" design—e.g. using partial destructuring pattern-matching (unwrapping tuples one layer at a time) instead of uniquely tagging messages with message-type UUIDs like COM/CORBA—is to "automatically" cope with this, allowing your "living" distributed system to interoperate in the face of cross-node module-version heterogeneity, without having to write explicitly backcompat "also recognize the previous/next version of the message" code during the rollout period.
Every Erlang term sent in a message is, in some sense, like a little extensible file-format; and the tools you're given to "parse" the term—when you use them idiomatically—give you forward-compatible "parsing." As long as you don't validate that a term has a specific "deep" structure corresponding to some static type, then it doesn't matter if your module-V2 actor has actually received—and is now holding onto—a module-V3 parameter in its state. It won't know what to do with it, but nor will that cause any problems as long as it doesn't poke too hard at it. It can even keep hold of it, storing it away for later in its state generically, without understanding exactly what it's received. It'll be there in the state for when the module upgrades to V3, and suddenly cares about that property.
(OTP's supervision system also comes into play here, ensuring that any unhandled edge-cases of this term "parsing" just become temporary restarts of the leaf-actors that receive them, without affecting the stability of the node as a whole. The network upgrade completes; the affected actors come back up running the new module version; the system continues.)
It would be up to the developer to account for different versions of the data when writing out type definitions, and the worst case is that you try to access a property that doesn’t exist or is the wrong type, and it crashes and causes the process to get restarted like what already happens in Erlang.
So it wouldn’t be a guarantee of correctness just like Typescript isn’t, but it could still offer a lot of safety assuming you get the type definition a right.
It's static typing that Erlang can't complete-the-triangle on; not typing generally.
The only problem is that "offline" type-checking like this does nothing to solve one of the main use-cases/pain-points where Erlangers want types (or, at least, think they want types): in the messages that actors receive. You can't make any sort of a type assertion about what other actors in the system are allowed to send this actor, and get that validated; because "other actors" in a distributed system necessarily include ones that aren't even part of your present codebase!
I have a philosophy about this—not sure where I picked it up, but I think it hews to the Zen of Erlang quite well:
If you already know the type of a message, then by definition, it's not a message any more, but just a regular data value. A message is an OOP concept (and Erlang is an OOP language, where processes are the "objects.") An OOP "message" is a piece of data the meaning of which is up to the interpretation of the recipient; where that interpretation can change as the recipient's internal state changes. The whole point of the "receiving a message" code that you write in an Erlang actor-process, is to allow you to do custom logic for that interpreting part. To use the value itself, in making the decision of what the value is.
In fact, I would extend that: the whole point of Erlang as a language is to do that "interpreting" part. Once you know what something is and have put it into a canonical validated structure, you may as well hand it off to native code (using e.g. https://github.com/rusterlium/rustler). If you think of native code as being a pure domain of strongly-typed values, then picture Erlang as the glue that lives in the ugly world of "not yet typed" values, making decisions under partial-information conditions on what types to try to conform received messages into, before they can enter that pure strongly-typed domain. That's Erlang's idiomatic use-case! (You can tell, because using it for that produces absolutely beautiful code; whereas using it to do e.g. crypto math, produces an abomination.)
Which is all to say: the interpretation (or, if you like, constraint) of a message into a typed value is a Turing-complete operation; and the logic for doing so is best represented as an Erlang program. Erlang doesn't need a type system for messages; Erlang is a type system for messages. :)
I miss Joe quite a bit. Such a keen mind with seemingly infinite curiosity.
Yes. If only GenServer had an OOP syntax instead of a handle this / handle that list of functions which obfuscate what the process actually does. Elixir in particular lost a chance to make it easier to deal with.
I will say one thing:. If you are thinking of BEAM processes as actors/(Kay objects), you're missing the real meat of what makes BEAM processes special; what they really are are atomic units of failure domains. The other stuff is just a useful analogy that lets people grok the code structure by comparing it to something familiar. The trouble then is that people will take habits from OO and apply them to BEAM where blindly copying the organizational form will lead to performance regressions. If you're treating BEAM processes like python or Java or Ruby objects, you're going to bottleneck your system and turn it into a needlessly complex mess.
In short, not all function calls (cheap) should be message passing (expensive). Sorry Alan Kay.
It's good to hear this, though. I might have to try and re-prioritize learning it again.
Do you have any good resources on hand that you used and would recommend?
The thing that confuses me most about Phoenix and Elixir is how do I actually deploy the application to production? I have my own server where I can run whatever I want (running Ubuntu), so what do I have to do to get my Phoenix application working there? How do I get my code running as a systemd service?
To answer your second question, that's not entirely easy to answer, but I personally use mix release command, ship it to s3 as a tar.gz, then pull it down into the server, unzip, put the command line entry point "start" (there are instructions when you do mix release) into a systemd service, and it's good to go. Couldn't be easier. It takes us several ansible scripts, and annoying wrangling with virtualenvs if something goes wrong for us to create a new Ubuntu Django server from scratch and literally two commands (one is a 5 line shell script) to get Elixir up and running.
2. There are several ways. You can use releases [1]; copy the code to the box, run mix release, then ./app start, and point whatever you want to that process. If you're running phoenix you could always compile it on the box and just run mix phx.server. I wouldn't do that though. You could also use a tool like distillery[2] which is kind of like releases. There is another more esoteric option where you build locally without the runtime, but I wouldn't recommend that.
[1] https://elixir-lang.org/getting-started/mix-otp/config-and-r...
This gives you folder packaged with everything you need to run the application on the architecture you compiled it on. Very straight forward.
If you want to set it up to be a systemd service I guess it would be like any other binary command for systemd. I usually run things under something like Supervisor (http://supervisord.org/)
By the way how do you deploy Python code? There isn't a standard AFAIK.
Finally, check how systemd handles rabbitmq. It's an Erlang application and you could run and stop your application in the same way.
https://stackoverflow.com/questions/32085258/how-can-i-sched...
So simple. Something that would require a job queue and a job runner fades away into a piece of the OTP application tree. When it crashes, it will even come right back up!
Phoenix feels a little too heavyweight for really small projects—maybe I'm spoiled having used Mojolicious's [0] single-file web servers. (Example on the linked page.) But for anything slightly larger, Phoenix scales really well. I work on a decently-large application in Phoenix for work and it's been an absolute joy to work with this langauge.
Typing could be better. Though, Dialyzer does a decent job of catching type errors. That's saved my neck on more than one occasion.
For any serious application you want a job like that to be persisted so you can guarantee it runs even if your application is restarted.
I know that Erlang/Elixir is designed for stateful applications and if you have a cluster and do hot deploys this is less of a problem, but who does that? Most Elixir applications I've seen are deployed as stateless systems just like any Ruby, Python, Node etc systems.
[1]https://github.com/samsondav/rihanna [2]https://github.com/koudelka/honeydew
A couple of years ago the RabbitMQ team has published a raft library[1] which they use in their implementation of persistent queues. It has a flexible API and implementing your own state machines is quite straightforward, as it follows the OTP gen_* behaviour paradigm.
And by now, I'd say it's pretty well battle-tested.
An BEAM process, and therefore a GenServer, has an atomic message queue built in. Only the process itself can pop messages out of its queue, so there is no need for locking. It is not possible to get messages out of the queue concurrently. Messages can be processed concurrently, but that's after they've already been removed from the queue.
Just curious as to how the BEAM solves it as I imagine it can.
Note that this works by taking a transaction lock on the entire cluster, so it's not recommended if you're making requests frequently. It also doesn't respect netsplits, unless you instruct in code to fail with a deficient cluster membership.
You lose everything about the scheduled jobs when your app crashes.
This doesn't even address the rampant memory leak issues that plague long running genservers.
> Rampant memory leak issues
As long as you’re not sending this genserver bogus messages, I don’t see how memory would leak in this example. Erlang applications are known to run uninterrupted for years at a time, so I figure this is a solvable problem.
When you do need persistence for more robust job scheduling, yes, having a real queue system in place is much better. You are correct there. I think the answer is good because it is so small and fits these simple use cases—the fact that you can handle those cases without needing something heavier is pretty awesome.
It's a great path for rubyists to move to Elixir/BEAM and every rubyist should give it a whirl! I'm back working on scala and akka.
It's an ML inspired, statically typed language that compiles down to Erlang, and supports interop with the existing ecosystem. This means that you get access to ADTs, type inference, etc, while still being able to lean on OTP for your concurrency primitives. There's also examples of calling it from Elixir, so there's the option of falling back to statically typed Gleam for an especially gnarly piece of code, and calling it from your Elixir application [1]. I wouldn't necessarily recommend this for commercial apps yet, but Gleam today is about as usable as early-Elm was, in my opinion.
The project is also very welcoming to new contributors, and Louis (the language's creator) does a great job of curating a list of beginner friendly issues to tackle in the compiler. I've been spending my evenings learning Rust by adding onto the language, and it's been a ton of fun. If you want to help out, there's a fairly active IRC channel on Freenode, in #gleam-lang :)
I want ReasonML (language!) and Erlang (OTP!) to have a baby, and I want it birthed by the Go runtime. (Go? Yeah, Go. I don't love the language, but I am a lover of low latency and garbage collection, what can I say?) Yes, there's Gleam, but if something's based on BEAM, the throughput generally won't impress. :-( Would seem a shame to do all that static typing, and then not reap the speed benefits.
Relatedly, I think there's a sweet spot for a language that accepts mutability inside of actors, but only allows immutable objects to be sent as messages, with an escape hatch available if needed. (Pony explored this space, would love to see it evolve.) Combine that with OTP for happy-path programming, and an ML so you catch most of your errors at compile time, and you could end up with great throughput, low latency and great ergonomics, all at the same time.
Here is some benchmarks where Erlang beats Go in throughput:
https://timyang.net/programming/c-erlang-java-performance/ https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixi...
And maybe I missed something in the second link, but Go showed very similar I/O performance to Elixir, while consuming a boatload less CPU doing it. That's what I'm after. Open to being told I missed something, though.
The Bean keep using CPU even with no work to be done, so avoid context changes. We can't compare about CPU values.
Can you explain more why we can't compare? Looking at this chart...
https://stressgrid.com/blog/benchmarking_go_vs_node_vs_elixi...
...shows pretty clearly how the CPU utilization grows linearly (ignoring some sawtooth) as the load increases, plateaus once the load remains constant, and then comes down linearly as the load decreases on the other end. Looks like a very clear mapping between work done and CPU load to me.
Essentially in order to optimise responsiveness the BEAM uses busy waiting, which in reality is not actually utilising the CPU as much as is reported by the OS, which results in misleading CPU usage being reported by the operating system.
Absolutely agree. I had great hopes an "Actor" type would be created in Swift, where every public function would be thread-safe, accept only "pure" structs / value objects, and which would automatically run on its own coroutine.
Unfortunately the concurrency story seems completely abandoned for this language.
Do you have a benchmark showing this?
As for Pony, in theory it should be great but it looks very complex..
I singled out Go because it's the only reasonably mainstream multithreaded (need this for actors), statically typed (need this for CPU throughput), garbage collected (need this for ergonomics) language out there with an emphasis on keeping latency under control, and as such would be the only sensible target I know of to host the language I proposed.
It allows the runtime to schedule coroutines effectively - they can't block for more than a function call (recursion is how you do "infinite" loops).
I think a future competitor to BEAM languages would need this feature.
I call that productivity.
That's kind of what akka is on scala or java. Messages are immutable _BY CONVENTION_ but you can do whatever you want.
I'm not sure where you're getting that from- that's typically the area it does well. It's bad at number crunching, but you if the work is IO bound (say, like a web application backend) it offers consistently low latency with high throughput.
Dialyzer helps a bit but is difficult to work with due to really cryptic errors. Also the workflow of having the typechecker run as a separate process not part of the compiler feels really cumbersome; it's easy not to notice you have a type error somewhere (often miles away from where the error originates!).
For non webapp stuff I've been happy using golang.
[0]:https://hn.lddstudios.com/ [1]:https://github.com/ldd/hn_comments_game/
Not to share my views or anything, and I have nothing to sell about it, but I thought I would share it with people anyways
On the other hand, it also reflects functional programming is also more straightforward than Object-oriented programming because it's more objective to just model the data, than modeling the data and procedures at the same time. If you have two people writing the same thing, there's a much larger chance they'll yield similar or identical results.
I would love to find a job doing it.
There's an awesome blog post on their website. where Chris McCord, creator of Phoenix, builds a real-time Twitter clone in 15 minutes.
https://www.phoenixframework.org/blog/build-a-real-time-twit...
Hopefully with the focus on the web and the fresh coat of paint Elixir has given the ecosystem it will eventually gain more adoption. But I still have my doubts because despite looking more mainstream than Erlang, it's still kind of a "funny language".
It's funny how much the first languages you learn (for lots of people, Java/C/JS) have such an impact on what "feels weird," when every programming language is basically magic anyway.
LiveView can be used for way more than that, but I think replacing "glue" JavaScript is the most slam-dunk case.
But I'm also a full-stack developer and I'm not afraid to write JavaScript. I realize not everyone is on the same boat, and LiveView might cater to them.
The value of LiveView is not having to write javascript but the only way to preserve any non-complicated state in LiveView when the web socket disappears is to write javascript. It is only recently that LiveView added support to automatically preserve form input values across a state crash. [https://github.com/phoenixframework/phoenix_live_view/blob/m... | Recovery form input data automatically on disconnects or crash recovery] Prior, to 0.7.0 if the web socket died while you were typing input values in a form then the page would reload and all your input into the form would be wiped out.
For example, go back to the Twitter timeline example. Imagine that you want to do a banner that says "N tweets unread - click here to expand them". You could keep this counter in the LiveView but if you reconnect the counter will be lost. But you can also solve this by storing a pointer to the last shown tweet on the DB. Then if you reconnect, you can compute how many tweets are pending.
If you are doing a multi-step wizard you can persist each step on the DB too, etc. Depending on the use case, this could even yield further benefits, such as automatic synchronization between devices. I.e. if you read some tweets on your phone, you can automatically bump the "last shown index" on all devices. Or start the wizard in one place and finish it elsewhere.
That's the point of LiveView, you can build interactive applications that are more server-centric. Some of those features are new but that should be no surprise, it is still pre-1.0 software. Even as we add new features, you may still have to (or want to) rely on JS on many scenarios, and that's totally fine.
The job posting is a little bit generic, but it's specifically for my team: https://sjobs.brassring.com/TGnewUI/Search/Home/Home?partner...
Feel free to reach out to me (my website is in my profile) if you want more info!
For a while I was using Phoenix<>Elm as my stack and I enjoy programming in Elm. But there's a lot of boilerplate you have to tediously connect for every input and output on both sides of the server and client.
LiveView eliminates all that boilerplate by letting you write templates in Elixir that are macro-compiled to javascript. Getting rid of that boilerplate eliminates time writing it as well as time debugging the extra complexity that boilerplate introduces.
The first six of eventually 15 lessons are already available and they really helped me grasp the basics. I've already implemented a few small tools using LiveView - it's a real breath of fresh air!
Working with Elixir is really cool, too, coming from C++ & JS backgrounds. Pattern matching feels like a programming technique from the future.
Btw. If anyone is interested, I'm working on prettier plugin for html l?eex files. Probably gonna publish it next week.
The first piece is longer, on my journey learning Elixir and building machine learning libraries.
https://fredwu.me/blog/2016-07-24-i-accidentally-some-machin...
The second piece is shorter, on Elixir’s functional aspect and how doctest changes the way I code.
https://fredwu.me/blog/2017-08-07-elixir-and-doctest-help-wr...
MIT Licensed. Source:
Curiously, unlike most others here, I'm not completely loving Ecto. I appreciate the philosophy and get changesets, but I'm finding the query syntax clumsy.
https://ekarak.com/2020/05/16/of-elixir-phoenix-and-analogie...
But fundamentally the issues caused by lacking static typing is different from the problems caused by being weakly typed like python and node systems are.
It's when you have JavaScript (not even TypeScipt) sending and receiving JSON (not even bothering with JSON-Schema) to/from Node/Ruby/whatever, saving everything to a permissive document store or to some poorly-normalized RDBMS schema that's missing half the constraints it ought to have and has a few columns that aren't the most correct reasonable type, that you end up with a brittle and slowly-advancing pile of shit pretty reliably. "Let's do a Rails REST API with Mongo and a Javascript SPA & React Native clients!" = run far, far away.
There’s entire classes of short term and long term issues that you get to avoid by working within the BEAM.