Elixir and Rust is a good mix
fly.io
fly.io
slightly /s
kubectl apply is 1 command!
ansible-playbook is 1 command!
sh ./do-the-thing.sh
you get it
Because many times you value fault-tolerance and distribution more than performance.
I found others like Lunatic before, but cannot remember right now.
You might be interested in Gleam[1].
Ecto is prolly one of the best ways to interact with db. Genuinely curious, what other ORMs have you used?
That's very common when you don't know DBs. But DB savy developers usually claim the opposite, because the syntax is more familiar.
I'm not a 20 year DBA greybeard veteran, but I'm comfortable enough to write schemas and queries by hand without any issue, and the entire time I wished I could do just that instead of using Ecto.
But good luck creating composable SQL with raw string interpolation, which Ecto excels at.
But if you come from an ORM perspective and expect the same experience on Ecto, yes I agree it won't please you.
In my case I prefer to work with Elixir because of the community, as I find easier to work professionally with Elixir than some other mainstream languages, as mostly projects follows the same good practices, use the same tools and have good documentation.
[1] - https://gleam.run/
You might be interested in Gleam[1].
I haven't seen hot loading for Rust (but a quick search shows there's some out there), and I'm not sure how amenable Rust is to dlopen and friends to force the issue.
Erlang (and Elixir) have a constrained language that allows for BEAM to be effectively premptive in a way that a Rust concurrent runtime can't be. At every function call, BEAM checks if the process should be preempted, and because the only way to loop is recursion, a process must call a function in a finite amount of time. A Rust runtime cannot preempt, if you need preemption, you've got to use OS threads, which limits capacity, or you need to accept cooperative task switching.
Also, some of us are as anti-typing as you are pro-typing. :)
Assuming ample experience with both, how does one reach this conclusion?
I have yet to see a project of any size that needs to be worked on by multiple teams and is written in an untyped language not descend into dumpster fire.
Github? Dropbox? I mean they both eventually went to type hints in their respective languages or migrated to a typed language but for a long time I'm certain it wasn't.
Can we say that's misuse of the tools? Sure. Is it less likely than things becoming a mess without types? Probably? Even more so as the tools improve and as the people involved know better how to use them.
Sure, at boundaries between teams, you need to specify the data in some way. That could be a type, but for me, often the other team is using a different language than me, so it needs to be a language agnostic type, and it can't include unsigned numbers because Java can't cope, and it can't include large integers because Javascript can't cope, etc. Protobufs are popular, json is too.
I have a lot of unpopular opinions though, and that's fine. It's just tiresome that everyone wants to come in and add types to things that don't need them. Also, I agree with dllthomas, most developers and teams are capable of creating dumpster fires in all sorts of environments, with all sorts of tooling. :)
Putting the fire out in an untyped language is a Herculean effort.
This assumption is changed, IMHO, by Erlang. Hot loading makes the cost to make small changes very low. So the question becomes, do you pay the definite cost of build time type checking (usually includes coding time type annotation), or do you accept the possible future cost to making small fixes.
Of course, if you work in an organization where even a small fix requires months to release, then do all the things you can to prevent making small mistakes.
It really depends on the domain but it definitely is more than pushing an update.
I’ve lost weeks to a memory leak once in javascript that was a 2 line change to fix. If I realised the problem when I wrote the code, I would have saved myself a lot of trouble.
Erlang was designed to achieve 9 nines of uptime. It achieved this without static typing across very large applications. The fact that this is a regular occurrence with Erlang disproves the idea that a lack of types is fundamentally unsafe.
Static types are most useful with monolithic application design. They counter your massive ball of code growing too complex and the complete lack of introspection at runtime. They attempt to handle the problem that any error crashes your entire system.
Erlang uses a different approach.
First, it uses safe datatypes. You aren’t going to crash because you chose a 32-bit integer and rolled it over (something a type system won’t actually help with). This is maximally likely to corrupt user data without even raising any flags. Integer rollover has actually killed people (famous in the Therac-25).
Second, it uses all immutable data, so data sharing is safe. Also something types don’t help with. This is also a maximal risk of data corruption. Incorrectly mutating data has also killed people (also in Therac-25).
Third, it is functional. Toys reduces passing around giant balls of mud. Those balls are unusable disasters unless you add some types. Functional programming with immutable records means that a program can’t accidentally change the types of incoming data. Because the pattern encourages separating data and mutable state, the most common typing accidents are simply avoided.
Erlang is designed with concurrency first. This helps to keep those balls of code even smaller and further reducing the chances of typing errors. And of course, combined with immutable data, we eliminate another set of errors typing does nothing about and that have caused massive damage and probably deaths (a deadlock causing a NYC blackout leaps to mind).
Finally (I probably missed some points), Erlang is designed expecting crashes to happen. Few runtimes are capable of anything close to the elegant Crash handling of BEAM. Instead of fearing crashes, you understand they’re inevitable and embrace them. This means that you are prepared for not just an occasional type error, but will also elegantly handle null exceptions that plague most of the most common statically typed languages.
A small number do. Assembly languages are generally untyped. The Forth language is also untyped.
> At build time you catch bugs that would appear later at runtime. Which is more costly to fix later.
Generally agree. Programmers proficient in Haskell or Ada tend to consider types to be integral to their development process. The real question is whether this is a good tradeoff against development velocity, for your given project. Neither language markets itself for rapid application development, instead they tend to emphasise that the language aids with correctness and the ability to reason about code's behaviour.
With the way that Erlang and Elixir pattern matching can be used in function heads, I can have much the same feeling of certainty that people express from Haskell and Rust. (Erlang typespecs help here, but are not checked by the compiler itself, only by additional tools like dialyzer or gradualizer.)
That said, a dumpster fire usually has no or little tests, so maybe we're arguing non-existent hypotheticals :|
- Types and tests find different bugs. I’ve found new bugs by converting a project from javascript to typescript. The project in question had a 2:1 test:code ratio but as soon as the typescript compiler could read it, it spotted a couple obvious errors.
- Large test suites often make refactoring harder, not easier. If you have a clear, fixed API boundary and your tests test that boundary, then testing helps. But most refactoring also involves changing up those APIs as well - since bad APIs are often the reason you want to refactor in the first place. When you do that, you have to also rewrite all your tests. Good type systems help refactoring. Writing rust in Intellij, I can globally rename functions and types in my project, promote tuples to structs, reorder function arguments, and all sorts of other handy refactorings. My tests get updated too. And the compiler tells me immediately if I missed anything, without needing to rerun my tests.
- Reading the types is my favourite way to get up to speed on a project, or get back up to speed on something I wrote myself that I’ve forgotten. "Show me your flowchart (code) and conceal your tables (type definitions), and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious." -- Fred Brooks, The Mythical Man Month (1975)
- I find I need far fewer tests to write reliable software when I’m using a language with a good type system. Most rust code I write works correctly once it compiles. Javascript is easier to write than typescript, but it’s harder to test and debug.
So with all that, I’m personally in camp type these days for most software. I think it’s usually the right choice.
A clear, fixed API boundary is exactly what Phoenix tries to encourage with contexts. Unfortunately, a lot of developers find them hard to understand. They're simple if you read up on DDD but again, a whole host of developers won't, or don't, do that either. LiveView in particular has a really a really great testing library [0] where you can write what are essentially end-to-ends that never touch even a headless browser. Since I'm always writing LiveViews, I pretty much only write LiveView tests and contexts tests which gives me large coverage (also some unit tests for the odd utility function). Otherwise, it's really important when writing non-typed functions to make it really obvious what is coming in and out, which is arguably a nice forcing factor.
The number one thing people bring up when shilling types is large codebases (it's been brought up in these comments). My opinion there I have found is quite unpopular and that is that pair programming should be far more prevalent than it is. I think the whole notion of "just stick a junior on that" is broken and I don't understand how types make that situation _that_ much better.
All said and done, I'm not actually anti-type. I mostly just find them to be incredibly noisy compared to a well-written function. I really like Ocaml where it's statically typed without needing to actually specify them.
Yeah; I haven't worked with ocaml but I've done some haskell (where you think about types so much more). Personally I don't mind rust / typescript's approach of needing types at the function boundary (function input & output types must be specified) while doing inference wherever possible inside each method. As an example, here's a very complex function in a project I'm working on chosen vaguely randomly[1]. The function diffs a run-length encoded DAG using a breadth-first search.
Visually scanning for types, there's a couple at the top of the function - both in the function definition and the BinaryHeap:
let mut queue: BinaryHeap<(LV, DiffFlag)> = BinaryHeap::new();
[1] https://github.com/josephg/diamond-types/blob/66025b99dbe390...
You definitely still think about types in Ocaml, you just don't need to annotate due to the language design. A big part of what makes it possible is that there are no overloaded operators, eg, you can't add an int and a float without casting as the mathematical operators are different: `1 + 1` v. `1.0 +. 1.0`. While I've dabbled in both, I'm no expert in either Ocaml or Haskell, though.
Really for me it's just that I've never felt the pain as I haven't worked in a big enough project, I guess. There is something that just kind of annoys me about (pseudocode): `(name : string) :: string -> "Hi, #{name}"` because, like, no shit it takes a string and returns a string! It's a death by a thousand cuts thing where I don't want to read that stuff and the compiler doesn't need to be explicitly told that in order to do static analysis.
Anyway, again it's really not the end of the world as I'm not anti-type. I just don't yearn for them in Elixir or anything. If it had a solid typing system I even might use it, but I don't yearn for them or anything.
You have some really interesting projects on your github, though! I mostly build glorified CRUD web apps! I do always get a sense that a lot of the type-talk is centred around organization disorganization.
Disclaimer: I work at Microsoft, but not in the Developer Division
However, a language with an insufficient type-system indeed makes things harder than they are without it. I would count all the languages you listed into this category.
As another poster mentioned, typescript is fairly expressive. There are other (production) languages too, such as Scala or maybe D. And there are lots of academic/very-niche languages.
> It's just tiresome that everyone wants to come in and add types to things that don't need them
Well, types are there, if you like them or not. There's a reason that you have e.g. typeof in javascript, gettype in PHP. The question is rather if you explicitly annotate them or not. But yeah, sometimes it's not helpful to annotate types, especially if the language is incapable of expressing the correct type anyways, which is true for most programming languages.
I have started programming in untyped languages, but simply can’t remember back at all, and now I can’t really imagine dealing with objects in my mental medal as not having some type.
Note: this is not a rebuttal for/against dynamic typing, I do think that types are really important at boundaries, but they may not be the silver bullet - contracts may be better at some things, for example. This may be an open question.
In Elixir, I do think about shapes more than capabilities (because Elixir is not OO), but with pattern matching, I can either specify "this must be a MyApp.Account struct" (which is just a fancy map) or I can specify "we will handle any map that has the keys X and Y, and Y must be a map itself".
I replicate this more formally when writing TypeScript, usually by building up type definitions and specifying those.
Look at Erlang. It has bigints, floats, Booleans, but-strings (sequence of bits —added because it is so common in telecom), string (not technically a primitive data type), functions, atoms, list, tuple, and map.
None of these look the same or act the same. People dream about seeing `123 == myMap`, but it simply isn’t a common thing because it doesn’t make sense.
The common rebuttal becomes: but how do I know if property X is a string or number when I’m using it?
If that’s your question, you are already messing up. What you really want to know is what X actually represents. Otherwise, you’re just shooting in the dark which is at least as dangerous as getting the type wrong and probably more so because a wrong type will become obvious quickly while mangling that number or string may not be caught until a much later time after serious damage has propagated throughout the data.
Let’s say you have something called `login`. Is it a number or string? If it’s a string, is it an ISO date, UTC date, or something else? Is it when they logged in or when their login expires? If it’s a number, it could be a Unix string. It could also be a calculated value for how long the user has been logged in and could be days, hours, minutes, seconds, milliseconds, or something less common.
How do you know which thing is correct?
In a good codebase, you read the docstring comment on the data constructor that describes what it does. If it says “milliseconds since last login” vs “token expiration using ISO datetime format” do you have any question at all about whether it’s a number or string?
If there isn’t a docstring, you’ll be digging through that code or playing around with the responses and will see the data type anyway.
The result is that you’re forced to better understand what you’re doing which isn’t a bad thing in my opinion. There may still be mistakes, but that leads to the next point.
Dynamic languages generally make it easy to dynamically check the types of incoming data and tend to be more flexible with mistyping (especially JS). I can’t count the number of major errors from common typed languages because they make introspection hard, so programmers don’t do it and crash on malformed data or when an API suddenly changes.
It’s also worth talking about null exceptions. Many dynamic languages expect type weirdness and handle it well. This usually includes null. Most statically typed code out there has LOADS of null exceptions lurking about which only trigger in obscure cases during runtime. In this regard, you could argue that the worse type issues also happen in typed languages, but are more dangerous in those languages too.
Untyped languages also tend to use safe numbers everywhere. Infinity is mostly useless, but not usually dangerous. Bigints everywhere are slightly slower in some cases, but completely eliminate overflow errors. Most typed languages use risky numeric types, so they must also force users to think about those things.
Finally, no common typed language offers good runtime introspection like smalltalk, Common Lisp, Erlang, or even JS. I feel static types are just a crutch to make up for this deficiency.
That brings us to good static languages. Typescript offers all the benefits of normal, unsound static typing combined with all the robustness of JS’s dynamic environment. The same can be said for Coalton and Common Lisp.
StandardML offers a language that feels like a dynamic language, but still offers static checks that are actually sound and completely eliminates null exceptions. Rust (an ML in spirit if not in syntax) does the same things in environments where garbage collection and other such amenities aren’t possible.
I like both kinds of languages and good examples from each word around the problems of each approach to make them (in my experience) about equal in productivity for equally skilled and experienced developers.
The issue is working on projects once they've reached a certain size where you have no idea what the intent of the original author was and you maybe need to refactor, add-in major pieces, or change anything with the expectation that it continues to work.
I think it’s a question of understanding how to work and think without explicit types rather than something that makes statically typed codebases easier to maintain.
They are exceptionally easy to refactor, add-in new parts, etc.
The keyword is microservices, you need to know how to do proper microservices if you are using untyped code.
They can be the correct solution sometimes, but blindly applying them everywhere is just dumb.
Some times, some type systems actually make people jump through hoops to accommodate their design and then it can actually have a negative effect. Other times, the typing helps.
It’s kind of like really good grammar and punctuation. They can make a story you write better and clearer. But they far from guarantee it. You can write a very good story with subpar grammar/punctuation. And you can write a really lame story that is grammar perfect.
One thing that I haven’t seen much in the discussion, is any discussion about Elixir’s matching abilities. Does Rust have that as well? I love what Elixir matching does for my code.
(Edited spelling)
The only thing you can't do is have both untyped and monolithic at the same time.
People use Python and JS, not Lisp.
Dynamic typing in Python and JS + medium-sized project = dumpster fire.
For example the Filesystem Browser (FSEdit) from 1980 was written in Flavors. It actually uses an explicit OOP system with hierarchical classes.
https://tumbleweed.nu/r/attic/sys78/file?name=lmfs/fsedit.li...
Dynamically typed microservices is where it is at.
It is easy to be maintainable at iteration 1 when the requirements haven’t changed 20 times yet.
That said - if you don't benefit from what BEAM has to offer - I agree Rust is a really attractive alternative.
Sure, you just need to reimplement light-weight threading with preemptive scheduling prioritizing latency over throughput, extremely robust fault tolerance with a supervision hierarchy, and runtime introspection with code hotloading capabilities. Maybe you could add frictionless distributed system support as well.
No big deal, why not right?
Tasks for backend systems are usually pretty homogeneous. I'm not sure how in such cases the overhead of preemption is in any way better than cooperative multitasking.
> light-weight threading with preemptive scheduling prioritizing latency over throughput
Rust has Tokio for light-weight threading which might well be sufficient for the majority of use-cases.
> extremely robust fault tolerance with a supervision hierarchy
One could argue that Rusts compile-time guarantees together with something like the Result-type make it so that such a supervision hierarchy isn't quite necessary and a few "manually implemented" error-boundaries are sufficient. This is also true for errors like network-hickups.
> runtime introspection with code hotloading capabilities. Maybe you could add frictionless distributed system support as well
Fair enough points.
I don't think your attitude against the OP is justified though.
Fault tolerance and supervision hierarchy might be unnecessary as mentioned.
Hotloading capabilities are unnecessary, most shops go with blue-green deployments, so the hotcode loading is usually unused (and for a good reason, so much complexity!). Distributed computing also goes unnecessary as most applications are deployed with containers with some form of autoscaling, so the industry went a different direction than elixir.
That leaves us with runtime introspection, which is pretty cool indeed. But that has to compete with Rust performance.
Pretty tough.
I love elixir, but when Go got premptive scheduler for goroutine, the need for elixir dropped dramatically. Which is sad because i loved the language and phoenix.
I'm hoping it makes a comeback, though!
It's not like there is suddenly no fault tolerance though. Erlang has a certain way of handling/dealing with errors/faults and so does Rust and other languages. I would not by default assume that Erlang's errorhandling is superior.
> That leaves us with runtime introspection, which is pretty cool indeed. But that has to compete with Rust performance.
I would much rather say that runtime introspection has to comete with a static type system.
As I said many times, I really like the BEAM (saying that after having worked with Akka quite a bit) but Erlang/Elixir... those languages are really not great. There are many languages that way better. I also know that there is a new one for the BEAM (forgot the name) but so far we are mostly stuck with E&E.
Elixir could teach some really good practices to people writing code, which is why I'm somewhat sad that other languages can supersede it (go, rust).
My problem with this statement is, that it emphasizes the language (Elixir) but what you really mean is the paradigm of using the BEAM no? As in, using Erlang or Elixir doesn't really matter for those good practices, even if Elixir is nicer in some regards.
In that case, it should be written like that, otherwise it will confuse people and prompt them to disagree, like I did.
- no state, unless when you really need it
- avoid mocks unless you really need
- pipeline-style approach where writes happen only at the end (io at the edge of the system)
- prefer integration tests
Just because you have a Result type doesn’t mean you actually properly/meaningfully handle the error at all, it may just happen that “restart” is the correct solution. Also, Rust is not safe from dead/live locks and many other concurrency issues, only data race free.
This is not against Rust, but against the very biased hype for it.
I never said that. But would you claim that Haskell programs are generally more faulty / less stable than Erlang programs?
> Just because you have a Result type doesn’t mean you actually properly/meaningfully handle the error at all
Okay, but the same is true for Erlang and the BEAM.
> it may just happen that “restart” is the correct solution
Yeah, but is very easy to do with Rust and Haskell or even just on the infrastructure level (i.e. restart the failed container/instance).
> Also, Rust is not safe from dead/live locks and many other concurrency issues, only data race free.
How is Erlang safe from those things in a way that cannot or only with a lot of effort be replicated when using Rust?
Each BEAM process (other runtimes would call them preemptable green threads) has its own heap, communicates with other processes by messages, and only works on immutable data without the involvement of Native Interface Functions (NIFs). There’s no shared memory at all. It is natively massively concurrent without most of the risks involved. Sure, it’s possible to write code that results in mailbox deadlocks, but it also means stepping outside of normal program design for the BEAM, and it means stepping outside of OTP (which provides structured programming constructs like gen_server).
Rust makes it harder to program without memory safety. Erlang (and other BEAM languages) make it harder to program without concurrency safety.
Edited to add:
Joe Armstrong’s thesis on Erlang is available for download and is written in very accessible language[1].
If you want more, you can also read Programming Erlang (2nd Edition)[2] also by Joe. (I would love to see a 3rd edition tackled by someone to address the newest stuff added in the 9 years since the last publication. I can understand why no one would want to approach this, since it was Joe’s.)
[1] http://erlang.org/download/armstrong_thesis_2003.pdf [2] https://pragprog.com/titles/jaerlang2/programming-erlang-2nd...
Is this related to my question?
I can see that this helps to prevent (or ease) out-of-memory errors. Other than that, what's the difference to using Rust's green threads, given that the developer knows what they are doing (but are still human and can make mistakes of course)?
Maybe a concrete example would help me (and others) to understand the difference.
An analogy: The BEAM process model is to "developers know what they are doing" as the Rust borrow checker is to "C developers know how to write memory safe code."
In other words, your program will have bugs. Your program will not correctly handle every failure mode. Your program will fail. The BEAM process model makes it so that a failure in one process won't take down all processes. And furthermore, after a process fails, there's always a deterministic way to recover from that failure without you the programmer having to think about it too hard.
The system is so robust that having processes fail on error conditions is encouraged. Once you really internalize this "let it crash" way of thinking and writing code, you only program for the happy path and let the process system handle the rest. The code ends up being much shorter and easier to understand. It's the complete opposite of writing code in something like Go, or I imagine Rust, where you explicitly handle or punt every error you can think of at every step.
And interestingly enough, despite not handling errors at every step, the share-nothing process model ends up being much more resilient in the face of errors.
Yeah. But the deveveloper still decides on how many and what processes there are. They have to understand the concept of a process and spawn them accordingly.
The same is true for Rust Tokio (and similar solutions) as well - you have to create tasks and manage their lifecycle.
For example, if you were to implement an http server, you'd have to use one erlang process per request so that if something goes wrong it only impacts this request and doesn't kill the server. In Rust, you would create a Task (green thread) per request as well, which then (if it fails) will not impact that Task that is "supervising" and creating those per-request-tasks, no matter if the request fails for a "valid" reason or because of a bug like an endless loop.
And even if there is memory and CPU resources (even OS threads) shared between those tasks, they are logically separated and for the developer it only matters in very rare cases (such as OOM errors).
I'm not saying that you get exactly the same level of fault tolerance or convenience with Rust here but I also don't see the fundamental difference. Hence, I feel your analogy would only make sense, if the developer has to work without a Task/Greenthread library.
Note that Rust does not have green threads (RFC 230: https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...), so without using the coroutine crate (which most developers don’t know how to use; the truth is that most people don’t know how to use threads).
The features that I talked about have nothing to do with preventing out-of-memory errors—they don’t really help with that. For non-external resources, the features described prevent memory contention (no shared memory).
Much like it’s hard to understand the Rust borrow checker quickly, the "example" you’re asking for is not possible in a comment on HN. I recommend looking over Joe Armstrong’s thesis, for which I provided you a link.
Rust has Tokio, which does have green threads.
From https://docs.rs/tokio/latest/tokio/task
> A task is a light weight, non-blocking unit of execution. A task is similar to an OS thread, but rather than being managed by the OS scheduler, they are managed by the Tokio runtime. Another name for this general pattern is green threads.
Hence I don't see a fundamental difference here.
nil is an especially big problem. Any value could be nil, and this will absolutely bite you over and over. nil even allows you to use square brackets for some reason (some_nil_value[:some_key]) which is a great way to disguise the actual issue.
There is optional type checking with Dialyzer, which is good but has some problems. The warning output can be really hard to read, and unless you're diligent in using it across most of your project, it's not very useful, because you'll end up with 'any' values all over.
Square bracket dictionary accesses are a code smell, because you should be using %{^key = val} = dict or Map.fetch(map, key) or rarely Map.fetch!(map, key).
If you do that, managing typing in Elixir just boils down to defining structs to differentiate cases where dictionary A and dictionary B contain similar keys but strictly are not interchangeable.
That access acts on nil is unfortunate, but it's necessary for things like get_in.
Ah, that explains it at least. Other atoms don't implement the access behavior.
Because Rust brings none of the benefits of the BEAM ecosystem to the table.
I was an early Elixir adopter, not working currently as an Elixir developer, but I have deployed one of the largest Elixir applications for a private company in my country.
I know it has limits, but the language itself is only a small part of the whole.
Take ML, Jose Valim and Sean Moriarity have studied the problem, made a plan to tackle it and started solving it piece by piece [1] in a tightly integrated manner, it feels natural, as if Elixir always had those capabilities in a way that no other language does and to put the icing on the cake the community released Livebook [2] to interactively explore code and use the new tools in the simplest way possible, something that Python notebooks only dream of being capable of, after a decade of progress
But they do not not stop there, the documentation is always of very high quality, even for stuff not coming from the core developers, and they also regularly release educational material that is worth a hundred times a gain in speed.
They've set a very high quality standard and I noticed how much it is important only when I stopped programming daily in Elixir and went back to other more hyped or establish ecosystems.
That's not to say that Elixir is superior as a language, but that the ecosystem is flourishing and the community is able to extract the 100% of the benefits from the tools and create new marvellously crafted ones, that push the limits forward every time, in such a simple manner, that it looks like magic.
Going back to Rust, you can write Rust if you need speed or for whatever reason you feel it's the right tool for the job, it's totally integrated [3][4], again in a way that many other languages can only dream of, and it's in fact the reason I've learned Rust in the first place.
I must also say that the work done by the Rust community looks refreshing as well, if you look at the way rustler works it was very well thought and made writing NIFs, something that seemed arcane and distant, only for the proverbial mad professor to try, a breeze. Kudos to them.
But the opposite IMO is not true, if you write Rust, you write Rust, and that's it. You can't take advantage of the many features the BEAM offers, OTP, hot code reloading, full inspection of running systems, distribution, scalability, fault tolerance, soft real time etc. etc. etc.
But of course if you don't see any advantage in them, it means you probably don't need them (one other option is that you still don't know you want them :] ). In that case Rust is as good as any other language, but for a backend, even though I gently despise it, Java (or Kotlin) might be a better option.
[1] https://github.com/elixir-nx/nx https://github.com/elixir-nx/axon
Runtime instrospection: the ability to log into a running application to inspect its state, start and stop processes
Compile times: elixir compiles very quickly and has tight feedback loops where as rust compiles slowly and has long feedback loops
Elixir also doesn't get in the way that the borrow checker does, allowing a programmer to just get on with work and not become saddled with related debugging.
Ecosystem: data pipeline processing via GenStage, Broadway, Flow -- wow. Rust developers should take note of what can be achieved in Elixir. However, Rayon and crossbeam are fantastic. Elixir cannot compete with Rust on performance in the category of pipeline processing, but it has very high marks in other very important categories that need to be considered for professional development.
I don't think that fault isolation is as compelling an advantage over Rust as it is other languages. Rust makes defense programming a regular part of development, unwrap used sparingly. Faults hardly happen because the program isn't designed to crash. In the rare event that a well-designed Rust program crashes, it's probably managed by an orchestrator that will restart. Both well-designed Elixir and well-designed Rust applications can enjoy very long uptimes if that is a goal.
This is just a starting point of discussion.
[0]https://github.com/ityonemo/zigler [1]https://podcast.thinkingelixir.com/83
This is where Rust falls short of C#: scaling to the issue at hand. C# can build you a beautiful app at a high-level but also lets you dick with pointers and assembly at a low level. Rust insists on defaulting to pass-by-move and an arcane trait system that hold it back from being usable in large projects.
Github link for others: https://github.com/maciejgryka/regex_help
IIRC one threat was Rust sharing memory with BEAM which could exhaust it and cause OOM crash?
I go into this in the article a little but but the ruslter team has made dirtycpu and dirtyio macro's to help reduce the risks.
The only proper solution is to audit and understand the code you're running, but hoping it's fine often works too; maybe formal methods, but to a first approximation, nobody uses those. Did you audit all of ERTS and/or OTP? I'm guessing probably not, but it's there to review if you run into a problem.
IMHO, it's not worrying about if BEAM will crash; worry about it not crashing instead. If your Rust NIF ties up a scheduler with an infinite loop, that has the potentially to lock up the whole BEAM once another scheduler needs to do something that requires full cross scheduler coordination.
BEAM can certainly crash on OOM; although I recommend setting a ulimit to ensure it will, because when it crashes, you can recover. I've also run into situations where instead of crashing or being killed by the OS OOM killer (which is close enough to crashing), the OS gets into some tricky to debug state where your application is neither functioning nor killed. Sometimes, you even get into a state where BEAM is making progress, but very slowly; that's a fate worse than death.
If you follow the Erlang philosophy, you'll have a recovery strategy from crashes or other deaths. Heart can be used to turn completely blocked into death, although I never used it professionally. But you've still got to worry about working but not well.
The Rust / BEAM memory sharing problem does exist, but it's not nearly as bad as in more traditional C NIFs, because almost all C programs leak memory due to bad manual memory management. Hence all the buzz about Elixir+Rust.
It's a more appropriate choice for something that's asynchronous, although NIFs do have ways to fill the same role. A port driver would probably be a better choice for specalty networking that ERTS doesn't provide (raw packets? netgraph, etc).
I showed them a piece of software that was mighty fast, Internet enabled and GUI intensive. They liked the software but asked where did you get this particular screen control from. You've got to see their faces when told that the whole software was written by a single person in Delphi.
EDIT: thanks for pointing out where in the article this is talked about.
> Change `#[rustler::nif]` to `#[rustler::nif(schedule = "DirtyCpu")]`
> This tells the Rustler and BEAM to automagically schedule this in a way that won't block the entire world while it works. Again amazing, this is called a DirtyNif and is way more difficult to work with when you are manually using this via C.
Essentially, regular NIFs have to be extremely fast (< 1ms) because the VM can't preempt them - they run on the same scheduler threads the BEAM itself uses. Dirty NIFs solve this by running jobs in a completely separate thread pool ("dirty schedulers"). Rustler's docs explain it succinctly (https://docs.rs/rustler/latest/rustler/attr.nif.html):
> For functions that may take some time to return - let’s say more than 1 millisecond - it is recommended to use the `schedule` flag. This tells the BEAM to allocate that NIF call to a special scheduler. These special schedulers are called “dirty” schedulers.
> We can have two types of “lengthy work” functions: those that are CPU intensive and those that are IO intensive. They should be flagged with “DirtyCpu” and “DirtyIo”, respectively.
(Somewhat OT, but since I'm here: excellent article @ peregrine! I really enjoyed the read. Elixir and Rust are such a perfect fit. Plus, some of the specifics will be helpful for certain image-related things I'm actively working on, which is always nice. :) )
I see this which mostly seems to be content sites. https://www.wappalyzer.com/technologies/paas/fly-io/ Same on the first forum result: https://community.fly.io/t/customer-success-stories/4882
Small sites because it is low-bandwidth to figure out how to use. Time is money and if I'm just tinkering on the weekend I don't really want to learn Kubernetes or the labyrinth that is AWS; I just want to ship an app.
Big sites because your users get routed to the node closest to them and again you can do this without a lot of time investment. For Elixir I just wire up Libcluster and my nodes can talk to each other.
I really want GPUs on Fly soon though. Just take my money Kurt. (I hear they're working on it)
Further Fly builds its Dashboard internally with Phoenix LiveView. We want the Phoenix and Ruby and Laravel and more communities soon, to grow because we believe if they grow, we will too.
I remember when Fly.io used to tout Firecracker but that is just a KVM engine, along with QEMU used on a zillion hosts.
What I'd like to see are customer success stories.
Edit: looks like you have to set up your own cross-region links on DigitalOcean and Vultr. So that interests me somewhat. :)
Erlang is generally considered to be compute-slow (which is generally the case without dropping to nifs).
Either way, I wouldn't expect doing I/O in NIFs or ports to substantially improve throughput, unless you're also moving significant processing into that layer as well, or you're going to end up doing substantially the same level of marshaling work as ERTS does, just in a different language. Setting up a different set of kqueue/epoll descriptors sounds like a lot of work for not much gain too, IMHO; again, maybe io_uring would be useful, but I think you'd be better served to bite the bullet and integrate it with ERTS.
Then add efficient pattern matching for binary data, and networked servers on the BEAM are the most ergonomic than any other language.
All this stuff that the BEAM offers you out of the box can be replicated in any native language with a lot of boilerplate and ceremony.
Your "Hello, <name>" webapp in Rust will probably need two allocations and a string concatenation, while on the BEAM, if constructed as an iolist, it's a single writev syscall, using a static "Hello, " string and a shared "<name>" reference from the parsed HTTP data.