Comparing Elixir and Go
blog.codeship.com
blog.codeship.com
You realize then that clustering, hot reload, availability etc... are not only features but the logical consequence of a beautifully crafted environnement that aims at developer productivity.
I'm sometimes amazed on how easy I can achieve stuff on the Erlang VM that would take ( if it's not impossible at all ) at least 10 times the time in a more usual language ( Ruby or PHP when you work in the web industry as I do ).
My last example was when I needed to batch sql inserts in an events database. In a normal language I would have needed a queue, the libraries for it, workers, new deployments and infrastructure to monitor, monitoring, supervision, etc... In Elixir, in 20 lines of code, it's done.
If you do not need complex calculations, the Erlang VM can basically become most of your architecture. It's already per se a SOA.
IE is it viable in your opinion still to use Erlang as a system for distribution and routing of lots of heavy calculations to many users, if said calculations are performed outside of BEAM? I am looking at building a multivariate financial calculation engine, which must be interactive for up to 1000 users, with large firehose of real time data coming in, being massaged, and then distributed, with the calculation graph being customizable for each user interactively.
You can also write NIFs (native implemented functions) which run in BEAM process (see http://andrealeopardi.com/posts/using-c-from-elixir-with-nif...). The latter option should be the last resort though, because it can violate safety guarantees of BEAM, in particular fault-tolerance and fair scheduling.
So using BEAM facing language as a "controller plane" while resorting to other languages in special cases is definitely a viable option.
Basically a NIF blocks the scheduler, so if you run a tight loop for a long time, there will be no preemption. Therefore, invoking foo(), where a foo is a NIF which runs for say 10 seconds, means a single process will get 10 seconds of uninterrupted scheduler time, which is way more than other processes not calling that NIF.
There are ways of addressing that (called dirty schedulers), but the thing is that you need to be aware of the issue in the first place.
If due to some bug a NIF implementation ends up in an infinite loop, then the scheduler will be blocked forever, and the only way to fix it is to restart the whole system. That is btw. a property of all cooperative schedulers, so it can happen in Go as well.
In contrast, if you're not using NIFs, I can't think of any Erlang/Elixir program that will block the scheduler forever, and assuming I'm right, that problem is completely off the table.
However, in some situations the overhead of communicating with a port might be too large, so then you have two options:
1. Move more code to another language which you run as a port.
2. Use a NIF
If you go for a NIF, you can try to keep its code as simple as possible which should reduce the chances of crashing. You can also consider extracting out the minimum BEAM part which uses the NIF into a separate BEAM node which runs on the same machine. That will reduce the failure surface if the NIF crashes.
I've also seen people implementing NIFs in Rust for better safety, so that's another option to consider.
So there are a lot of options, but as I said, NIF would usually be my last choice precisely for the reason you mention :-)
For example, if you want to call some C API from Erlang where the C API takes a struct and returns a struct, you'll want to actually populate the request struct--and parse the return struct--on the Erlang side, using binary pattern matching. The C code should just take the buffer from enif_get_binary, cast it into the req struct, make the call, cast the result back to a buffer and pass it to enif_make_binary(), and then return that binary. No C "logic" that could be potentially screwed up. Just glue to let Erlang talk to a function it couldn't otherwise talk to. Erlang is the one doing the talking.
On the other hand, if you have a big, fat library of C code, and you want to expose it all to Erlang? Yeah, that's not what NIFs are for. (Port drivers can do that, but you're about the right amount of terrified of them here: they're for special occasions, like OpenSSL.)
The "right" approach with some random untrusted third-party lib, is to 1. write a small C driver program for that library, and then 2. use Erlang to talk to it over some IPC mechanism (most easily, its stdio, which Erlang supports a particular protocol for.)
If you need more speed, you can still keep the process external: in the C process, create a SHM handle, and pass it to Erlang over your IPC mechanism. Write a NIF whose job is just to read from/write to that handle. Now do your blits using that NIF API. If the lib crashes, the SHM handle goes away, so handle that in a check in the NIF. Other than that, you're "safe."
@compile [:native, {:hipe, [:verbose, :o3]}]
I've no idea of the relative performance against numpy...
The team at Github took another approach, described here: https://github.com/blog/531-introducing-bert-and-bert-rpc It's an old article - so I no idea if it's still in use. The Github sources haven't been updated since 2010.
I don't know how effective it is to chuck around 8MB data structures via message-passing. I have no experience of this myself.
It's my understanding that above a certain size and within the same running Erlang VM (machine), a reference is passed instead of a full copy.
Re: supervising external processes, an easy hack if you're writing the processes is to add a deadman's switch to both sides, and then launch the processes from a port in BEAM-land.
This effectively makes them supervised; kill the beam process and the external process will die, and whether they both get relaunched depends on the restart strategy the supervisor launched the child with.
With 20.0 coming up it will be even easier. A nifty feature called "dirty schedulers" will become stable and so building long-running calculation in C will be much easier. Previously had to take care not to block the running scheduler thread.
All-in-all Erlang is really good at connecting to and managing things, C and hardware being one of those things.
https://www.youtube.com/watch?v=xj3smNjGLaE It is a common pattern to use erlang has a controller plane.
Can you provide more details on this? AMQP is pretty recent and people have been batching SQL inserts for much longer than it has been around. An external queue is not a requirement of non-Erlang languages, but I'm curious about specifically about how the implementation in Erlang would substantially differ/allow new approaches from that in other languages.
(In fact, I'm writing a GenStage consumer right now.)
The Erlang VM is "reduction"-scheduled. This means that the given Erlang process currently running on a scheduler thread can get pre-empted, but only as a result of executing a call/return instruction. (Effectively, the pre-emption is a check inside the implementation of the call/return opcode.) As long as you don't execute any call/returns (don't call functions and don't return from your own function), your function body can run as long as it likes.
This is a design choice: because processes won't be pre-empted "in the middle" of a function, any Erlang process can feel safe executing an instruction that calls into native code, while not having to worry that that native code could itself be pre-empted and leave dirty state in the Erlang process's heap while some other process gets scheduled and tries to then message or introspect that process. It gives you a lot of leeway "for free."
So how does Erlang ensure that processes don't hog a core forever, given that you could theoretically just write a loop that spins forever? Well, in Erlang, you can't write a loop. Instead of loops, you have tail-calls with explicit accumulators, ala Lisp. Not because they make Erlang a better language to write in. Not at all. Instead, because they allow for the operational/architectural decision of reduction-scheduling. Without loops in the language, every function body will execute for only a finite amount of time before hitting one of those call/return instructions, and thus activating the reduction-checker.
The Erlang "platform" has been shaped around the choices of how to best construct a production runtime that gives you "hard things" (like calling into native-code libraries while maintaining thread-safety) for free. Or rather, you could say that where everyone else pays these costs when they hit the particular problem, Erlang pays the cost up-front in the design of the language+platform and how you're forced to code at all times, in order to make these hard things easy.
The same is true of so many other Erlang things:
- how synchronous messaging has to be implemented on top of asynchronous messaging with expected reply-refs and timeouts, so as to make the sender process, rather than the receiver process, be the thing that defaults to crashing if the receiver doesn't recognize the message;
- how OTP-framework code has to be structured as delegate functions that return to the framework, so that the framework can "be there" in each process to handle hot code upgrades and process hibernation;
- how sockets either block (when {active, once}), or will saturate a process with packet messages (if just active) until that process crashes on overload--because the network listener is a separate part of the runtime that lives in a hot loop and wants to just be given a place to stuff packets into, and isn't allowed to do anything that's not an O(1) operation, like expanding the size of a process's message inbox;
etc.
Erlang is not a programming language in the sense that other languages are. Erlang was not designed from the language in. Erlang (ERTS) is a runtime, and was designed from the runtime out, with Erlang being effectively a pure side-effect: the language that ended up being required to interact with the features of the ERTS runtime.
Of course, you can also go back and apply some design sense to the language, and then you get something like Elixir. But, despite large visual differences "in the small", your large Elixir app will end up looking very much like a large Erlang app. And this is because a large part of what you're doing in an ERTS language is not programming using the language, but rather weaving together the features of the runtime. (Contrast: using DirectX vs. OpenGL to manipulate the GPU. Two very different APIs, but one "runtime" they're both speaking to, consisting of features like shaders et al.)
An Erlang VM is a living system that has a shell which you can connect to, and control both the VM and the applications running in it. You can also remotely connect to another VM, execute arbitrary code, debug, stop processes, start processes etc. It really is an operating system in itself, that was _designed_ to be that way.
And the best part is that you get all this for free. Whether that is a good thing depends entirely on your needs. You probably wouldn't want to replace your bash scripts with Erlang programs :)
What Erlang is not really suited for is where you need multiple levels of abstraction, such as when implementing complex business logic. You would think that the functional nature of the language lends itself to that, but then you quickly realize that because the primary concern of an Erlang engineer is to keep the system alive, and for that reason you must be able to reason and follow the code as it is running on the system, all kinds of abstractions are very much discouraged and considered bad practice (look up "parameterized modules" for an example of a feature that was _almost_ added to the language but was discarded in the end).
I think that from this perspective Erlang and Go are actually very similar - both prefer simplicity over abstractions.
Go multitasking is based on the compiler inserting switchpoints on function calls and syscall boundaries. But this affects the scheduling of a single OS-level threads executing that specific goroutine. The number of OS-level threads that the Go scheduler uses can arbitrarily grow, and OS-level threads are preemptively multitasked.
So I think the description is focusing a narrow view of the problem. What is usually required by applications is low latency in reply to system events (e.g.: data available on network sockets), and Go performs very well in this context. For instance, the fact that Go is transparently using a epoll/kqueue based architecture under the hood is probably affecting latency much more than the whole "cooperative" issue as depicted.
That's still not the entire story:
GC & tight loops in Go: https://github.com/golang/go/issues/10958
Per process vs per runtime GC: https://news.ycombinator.com/item?id=12043088
Please /do not/ read my OP as a dig/boost at any level. Just pure geek interest in language architectures and sharing info.
The cooperative issue is simply that until a thread does become free, the application can not respond to an event -- even given the epoll/kqueue architecture you describe.
* "G": these are goroutines
* "M": these are OS-level threads (the ones I was mentioning). These are bounded by debug.SetMaxThreads (default: 10000).
* "P": these are basically locks that Ms must acquire to run Go code. These are bounded by GOMAXPROCS.
So when you way "GOMAXPROCS=2", you're saying "I want at most two Gs executing code simultaneously", but you can still have a very high number of OS-level threads that are I/O blocked. Notice that when a G does a blocking I/O there are two possibilities: if it's a pollable operation, the handle is passed to the epoll/kqueue thread, the M releases its P and is recycled to do something else. If it's a non-pollable operation, the M releases the P but stays allocated to wait for the operation to finish (e.g.: the syscall to return). At that point, given that at least one P is free, any ready G can be scheduled, but a new M might be needed.
> Within Elixir, there is no operator overloading, which can seem confusing at first if you want to use a + to concatenate two strings. In Elixir you would use <> instead.
When this popped up, it reminded me of something people try to do often and then have issues with performance. You probably do not want to concatenate strings.
"Yes I do" you'll first think, but actually erlang has a neat commonly used thing to help here.
Let's say you're doing some templating on a web-page. You want to return "Welcome back username!". First pass (a while since I wrote erlang so forgive syntax errors):
welcome(Username) ->
"Welcome back " ++ Username ++ "!"
Instead, many of the functions you'd use to write files or return things over a connection will let you pass in a list of strings instead.
welcome(Username) ->
["Welcome back ", Username, "!"]
full_greeting(Username) ->
welcome(Username) ++ unread_messages()
Anyway, there's a better way of doing this. The functions that take lists of strings actually take lists of strings or other lists. So we can just do this:
full_greeting(Username) ->
[welcome(Username), unread_messages()]
So, for people about to get started, try not to concatenate your strings, you can probably save yourself and your computer some time.
For more info on this, you want to search for "IO Lists" or "Deep IO Lists".
Erlang was the first functional language i really learned and i can definitely attest to the pains mainstream developers go through when switching the paradigm. In fact with Erlang you're jumping two paradigms at once - functional/immutable and actor model of concurrency.
Eventually it clicked and i think learning Erlang was single most impactful thing on my way to becoming "a better developer".
Throw away your best practices of procedural/oop world and unlearn your optimization tricks when diving into functional/immutable. It's a totally different way of thinking about problems.
There's nothing about the ++ syntax that makes it necessarily so. The compiler/interpreter could understand that we're getting a concatenated string and automatically only create one.
Which is the case in several languages.
welcome_user(Username) ->
<< <<"Welcome ">>/binary, Username/binary, <<"!">>/binary >>.
If the username came from the database, it's going to be stored in memory before this function is called, and so concatenating must require building a new string.
E.g. no:
A + B + C + D -> AB + C + D -> ABC + D -> ABCD
And for immutable strings, it might not even require to build a new string at all, even if the string is in memory: it can just use a string data structure pointing to the various parts.
(
To review: https://gobyexample.com/errors
Manually checking every possible error (and then, only in the spots where you can imagine an error occurring) is a heck of a lot of extra work for the programmer (and code for the code reader/reviewer) and still won't catch all possible errors (both conceivable and inconceivable) properly. And arguably, the fact that an unchecked/undetected runtime bug in Go will basically send it into an "indeterminate state" which is impossible to reason about (much less debug), is an incredibly strong argument against this philosophy, IMHO. As far as I'm concerned, as soon as my code goes "off the beaten path" state-wise (read this as: "significantly differing from my mental model"), it should crash, ASAP. Isn't every bug literally a situation the programmer didn't account for? Aren't runtime errors by nature unexpected by the programmer? Why would you then give bugs and errors even more room to corrupt the state of the world, then? ;)
We are all obsessed with computers and languages when the real limit is the programmer's mind and ability to reason about the code s/he's building and the states that code can get into. I think BEAM langs and purely functional langs more generally (along with functional/immutable data structures, etc.) do a much better job of addressing this root problem. I'm going to quote John Carmack from his great blog post about functional programming here (http://www.gamasutra.com/view/news/169296/Indepth_Functional...):
"My pragmatic summary: A large fraction of the flaws in software development are due to programmers not fully understanding all the possible states their code may execute in. In a multithreaded environment, the lack of understanding and the resulting problems are greatly amplified, almost to the point of panic if you are paying attention. Programming in a functional style makes the state presented to your code explicit, which makes it much easier to reason about, and, in a completely pure system, makes thread race conditions impossible."
To be fair, you can't get Erlang's semantics without also inventing processes and supervisor trees. And those things would be largely meaningless without immutability.
Erlang's magic comes from how all the puzzle pieces fit together into a whole: For example, with immutability comes the ability to always be able to restart from a known state, and probably greatly simplifies the internal implementation of the per-process heap.
Those things fundamentally change the language; Go couldn't adopt Erlang's semantics without eschewing almost everything that makes it "Go".
Error handling is really the least of it. Attempting to replicate some of Erlang's high-level design principles at a quickly lead to roadblocks. For example, goroutines cannot be killed; this means it's impossible to fully implement supervisor trees in Go. It's possible that the runtime could support this at some point, but it's a big can of worms.
In Go, it's extremely easy to accidentally send a pointer (or a data structure that contains a deeply nested pointer somewhere; this includes embedded maps, slices and channels, which are all reference types) on a channel, which is bound to break mutability guarantees at some point.
If you pass a "local" on a channel to a different goroutine, Go can't guarantee to the receiver (which now has it as a "local" variable) that the sender (which still has it as a "local") won't change it:
d := map[string]int{}
ch <- d
d["foo"] = 42 // Oops!
Sure, you could instead mandate that senders never hold on to data sent on channels, but can you ensure this 100% across a codebase contributed to by an entire team?
Isn't that precisely what Rust is all about?
Erlang processes do have their own isolated mutable state.
But it's walled off and accessed through calls rather than variable references; in the non-SMP implementationthis enabled copy-free local (same node) message sends (and it still does, but only for large binaries -- the more general use turned out to be less efficient than copying in SMP because of locking issues related to garbage collection and some other implementation details.)
Small typo I found: """Go channels implement buffers which can either receive a a certain number of messages"""
(2x a)
Go methods have to be defined in the same package as their receiver's type. As a result, you can only add methods to your own types, not someone else's.
Also, the article claims that Go interfaces are similar to Elixir's pattern matching but I don't see a resemblance. Perhaps clarify that?
The main similarity I was going for was the way that an interface fits anything that matches it, rather than being directly implemented by the type. A method could be attached to an interface that matches a struct with two specific variables while Elixir could define a pattern on a function that worked for any struct that contained those two variables.
They are both doing a similar thing a different way.
[0] https://github.com/zeromq/ezmq* Elixir doesn't infer types. The language is dynamically typed, just as Erlang is. Using the word "inference" is somewhat dangerous since it means something very specific to a type theorist.
* Likewise, the compiler catches nothing, unless you are passing in a constant or literal. If you have `X <> Y` for arbitrary X and Y, the compiler can only catch it in statically typed languages (with or without type inference)
I think the main power of Elixir/Erlang compared to Go is the ability to handle arbitrary errors implicitly without the programmer having to write a single line of code. In Go, since there is a large shared heap, a goroutine must clean up after itself. In Erlang, memory is truly isolated, so a failing process can be cleaned up much like a traditional UNIX process can by the kernel. This creates system software far more robust.
It's such an automatic part of the stack with no drawback to using it that it's why I wrote it that way. Check that talk though, it's really interesting.
I fully understand you intended to write something else, and I very much agree with the power of the dialyzer :) It was more a "I dualized this section by accident, perhaps other readers would do the same?"
One of the links around that section does include a link to a conference talk on Dialyzer, for what it's worth.
A quick question, you talk about Elixir yet mention Erlang a lot. Is the choice of Elixir over Erlang purely syntactic preferences? Or is Elixir just an all around win above Erlang?
Erlang, with its small and hard to extend syntax, its libraries, frameworks, and conventions, is geared towards stability. Erlang is meant for creating reliable systems, which will be maintained for decades. Erlang is the most conservative (in the sense Steve Yegge's post) of dynamic languages out there: it's verbose, predictable, boring, consistent, reliable and focused solely on fault-tolerance (ie. long term stability).
Elixir tries to be a less conservative Erlang, but also good enough scripting and general purpose programming language. It's not a drastic change in capabilities, but rather re-focusing the language a bit to be more friendly to abstraction and extension. Elixir delivers on its promises magnificently: for my latest project, where I need the most expressive language I can get, I considered Elixir among such languages as Smalltalk, Io, Common Lisp, and Lua.
So it's more than just syntactic preferences, but the two languages remain very closely related. Elixir functions are Erlang functions, you can use Elixir modules from Erlang just fine and so on. Erlang is a simpler language, so my advice is to start learning Elixir and you'll learn Erlang when reading articles about its libraries that you want to use.
In short: Elixir is more expressive and easier to write. There are a few syntax/stdlib peculiarities but frankly, every language has them. I'd only classify Go and Java as languages with slightly less WTFs in their stdlib compared to Elixir. I'd give 2-3% of Elixir's stdlib functions a slightly bad color. Perfectly acceptable IMO.
Erlang is probably just fine for people who are used to it but for me Elixir was much easier to pick up. Also it started gaining additional functionality which uses OTP's primitives. Check `GenStage` and `Flow` for higher level complex multi-process scenarios, they are really next-level stuff and they help a lot.
Having coded in imperative languages like Java, Python and C++, I had been on the lookout for a practical general purpose language which provides good abstractions/high expressiveness. Elixir appealed to me more than Go in that regard. It's been six months since I started writing Elixir and it's been a pleasure.
Sometimes mutations are useful. There are lot of good programming design patterns which depend on mutations.
Also, always copying objects by value every time you call a function seems very expensive; especially if you're dealing with very large objects/structs/maps/strings which have to be processed by many functions.
According to this answer http://stackoverflow.com/questions/11055391/time-complexity-... it claims that for dictionaries it's O(log n). How is that possible if mutations are not allowed? Wouldn't the function have to process at least n items? Or it uses parallel processing to get a speedup there?
The BEAM seems to do magic here and do much less work than you might think when creating "new" things.
Passing around complex mutable structures feels more like procedural programs.
In practice, there are also many cases which ends up on the stack and never reaches the heap, so the memory allocation needs are somewhat smaller than what people think.
Another point is that short-lived garbage is collected in O(1) time in the GC. So you don't pay for its collection either. Things are slower than a mutation, but the overhead is lower than people tend to think.
It's a reasonably complicated topic, but the basic idea is that since the data structures are immutable, much of their structure can be shared between versions with few changes. Most of these data structures end up using a Tree of some sort. Performance characteristics can be influenced to an extent by using bucketing to control the width/height of the tree.
[1] : Purely Functional Data Structures (Amazon: https://www.amazon.com/Purely-Functional-Structures-Chris-Ok...) [2] http://hypirion.com/musings/understanding-persistent-vector-... [3] https://www.youtube.com/watch?v=7BFF50BHPPo
The FP folk aren't close-minded about this. They just use mutability only when it's provably more performant than their builtin immutable data structures.
For instance, prepending to a linked list results in a new head node, but nothing is actually copied. Appending requires a full copy. A developer has to be aware of where the limitations of these data structures are, but I think that's true of using any library in any language.
While I agree that this Linked list memory layout should be optimized, with real world tests Array backed normally win by a large margin.
The ownership model gives the benefits of immutability (no shared mutable state and race conditions) without the need to copy most of the time.
You can mutate by making a new copy, you modify the old one and save it to a new variable. The old variable is unchanged.
If you set x <- 1, you can't change x anymore but you can do y <- x + 1. Your modification is in a new variable. Elixir you can change x to a new value but that's just a new copy, unless you pin it..
You can also keep state by passing an accumulator via function parameter, it's also how you to tail call optimization. It's a weird pattern and I haven't use functional paradigm enough to give one on top of my head.
There are also optimizations that are straight-forward to understand and implement once you assume immutability. For example, take the following function in Elixir:
def some_list do
[1, 2, 3]
end
Here is an example that leverages this in the context of a web framework for great rendering performance https://www.bignerdranch.com/blog/elixir-and-io-lists-part-2.... While this is definitely achievable in other languages, we get it pretty much for free with immutable data structures and it is a natural mechanism to reason about. IanCal talked about it a couple comments above: https://news.ycombinator.com/item?id=13498532
Regardless, you are still correct when you say that mutations are useful. There are many algorithms that will be more performant if implemented on top of mutations. Luckily, most functional programming languages, including the pure ones, provide mutable data structures or memory references for such cases. For example, when working on GenStage/Flow for Elixir, I optimized the hot paths by using mutable dictionary and called it a day.
Many claim immutable data structures are a better "default" for writing software since it is conceptually simple to reason about code when data cannot change right under your feet. However, I won't try to argue if this is actually the case or not, as this reply is already quite long as is. The point is that many concerns regarding immutable data structures are solved and functional languages also provide alternate paths when you need mutability.
especially in a hello-world sized projects. as you scale up mutations become a constant source of bugs, bottlenecks and complexity.
> always copying objects by value every time you call a function seems very expensive; especially if you're dealing with very large objects/structs/maps/strings which have to be processed by many functions
structural sharing in functional persistent data structures makes this problem go away 99%
what you get in exchange for giving up mutability are programs that are easier to reason about, have less bugs, better modularity, less man-hours wasted on maintenance. all the good stuff.
FP with a focus on immutability is good for the default, but ignoring its limitations is bad.
So where does the myth that you can't exit out of immutability come from?
Sadly the place where I worked was not keen on open source posture, so it's locked away for all of time probably. However, the place I now work is a lot more open source friendly and I expect to take another whack at it, but using Rust on the NIF side this time around instead of C.
hard to discuss without specific examples, but it is provably bad thing whenever you share the value that you're mutating.
if your problem boils down to crunching numbers, maybe even in a couple threads without needing to communicate and cooperate between them - sure, mutability is hands down best approach, but in other 99.9999% of problems it is a hindrance.
ref <- newIORef 0
writeIORef ref 1
y <- readIORef ref
print y
There you go, mutation in Haskell.
Sometimes mutations are useful. There are lot of good programming design patterns which depend on mutations.
Study the Chomsky Hierarchy and the Theory of Automata. < https://en.wikipedia.org/wiki/Automata_theory > Here it is in a nutshell:
Languages are synonymous with computation. These can be divided into just a few types, based on how they can use state. One can observe that the complexity of an automata and its corresponding language increases dramatically with its use of state. (Turing Tarpit https://en.wikipedia.org/wiki/Turing_tarpit) You can avoid complexity by avoiding complex interactions with state.
Also, always copying objects by value every time you call a function seems very expensive
In the case of parallel/concurrent computation, it can be the less expensive option. < https://mechanical-sympathy.blogspot.com/2011/07/false-shari... > As always, the devil is in the details, and it's all about context. So you are correct. Sometimes mutations are useful. However, sometimes, you can get a benefit from avoiding them.
It's just easier to think of a pure functional construct, there's less possibility for mistake.
No one claims it to be efficient at single operations. Though some people blame the hardware for that, as in theory, there's nothing slower about it.
> Also, always copying objects by value every time
> you call a function seems very expensive
Also, if you use iolists for text processing (instead of appending strings), you absolutely will reuse all string chunks you use in the entire Erlang/Elixir OS process, even if you don't want to. It's mega-efficient.
EDIT: @josevalim (Elixir's creator) explains it much better than I can, right here in this thread: https://news.ycombinator.com/item?id=13499328.
I definitely want to read more about this.
This claims from a study of a particular model of Ericsson's ATM switch whose software was written in Erlang.
A can tell you for sure that: * Ericsson has other switches and other telecom equipment whose software is written in C++ * other companies (Nokia, Cisco, Alcatel etc.) also built similar complex telecom equipment whose software was in C++
I'm going to bet that at least one of those devices was more reliable than that particular switch, because it's not the language that creates reliability but people writing and testing the software.
Some languages make it easier than others but attributing some magic qualities from extrapolating a single case study of a telecom equipment is just silly.
Telecom equipment is reliable because it has to be. If you're incapable of writing reliable software for telecom equipment then your company will go bankrupt and be replaced by one that can, even if the language they use is C++.
https://en.wikipedia.org/wiki/Erlang_(programming_language)
It also had this along the same lines though:
>As Tim Bray, director of Web Technologies at Sun Microsystems, expressed in his keynote at OSCON in July 2008:
"If somebody came to me and wanted to pay me a lot of money to build a large scale message handling system that really had to be up all the time, could never afford to go down for years at a time, I would unhesitatingly choose Erlang to build it in."
But also: Of course a language creates reliability. Null pointers are impossible in Erlang, whereas they are provable impossible to avoid in C++.
So people bring this up because Erlang programs are pretty much reliable by default (assuming you stick to OTP and its design).
To help it reach insane uptimes, Erlang also has a few features that are hard to replicate in C++, if not impossible. The main one is hot code replacement; being able to incrementally upgrade a system in a safe and controlled manner without restarting it is a huge benefit in such a system.
It's also trivial to attach to a running Erlang program and introspect it as if you're truly running inside it. Goes hand in hand with hot code replacement. With C++, you get fairly crummy, low-level things like gdb, without the ability to actually write C++ code while in the debugger.
Also, remember that crashes can occur in bug-free programs. Hardware failures can be hard to predict and handle.
They are trivially possible to avoid in C++ as you do not have to use raw pointers. References, value types, custom non-null smart_ptrs, etc.
The trade-off you get however, is that the explicit control in C++ tend to produce systems which run faster under nominal operation.
In practice, many Erlang systems contains some C/C++ parts at their core for fast operation. Depending on their complexity, you try to isolate them from the rest of the system as much as you can. In short, you use Erlang as a top-level control backplane for your C/C++ code. The advantage is you get the speed of C++ where needed, but the added productivity of Erlang everywhere else.
Erlang programmers are usually practically minded in this regard. "functional purity" doesn't really matter that much if the problem isn't solved.
* Even smart pointers are still pointers, which are trivial to access after release by mistake; and references are only safe as long as the caller still exists. C++ does not have anything like Rust's borrow checker.
* C++ threads allow race conditions that can screw up even smart pointers.
* Plenty of libraries (C and C++), which are usually impossible to avoid and completely impractical to vet, will still use raw pointers.
Generally, the word "trivial" and C++ don't go well together.
I'm making no claim about general the memory safety of C++, and its issues are well known.
I did, at one of the the companies you list there.
We had outsourced and offshored code for modules on those switches you mentioned.
Have you ever had the pleasure to review C++ code in such type of projects?
I did, and hope to never do it again.
- Code either "C code compiled with C++ compiler" or C++ OOP Spaghetti like in the CORBA/DCOM days before JEE was a thing.
- functions/methods that span several screens
- barely any kind of testing
- due to emphasis on C style programming, lots of fun tracking down pointer misuses
- re-implementation of code that is already part of C++ standard library, even the ones from C
- lots of copy-paste on the same file because devs don't understand neither source control nor modularity
I wonder how Rust with its focus on memory safety could play out in such scenarios.
I need to have a look at Erlang/OTP. It is a bit of a hidden, alien gem.
The enterprise is not the space for it, specially among companies whose bread and butter is not to sell software.
The only measures of quality management cares about, are "does it work" and "delivers what customers pay for", anything else are just costs that need cutting.
Just like the type systems from Algol family of languages.
Every time a language outsources safety to an external tool, the majority of developers and their employers don't care about it.
Had lint been part of the compiler, like clang did it, instead of external tool, C code would have been much safer than it ended up becoming.
I'd recommend it. I think even in the worst case, it's interesting and can change how you approach problems in other languages.
In the best case, it'll fit some problem you have suspiciously well and you'll be left wondering why it only took you a couple of days, a tutorial and 50 lines of easy to understand code to solve a really annoying problem you had.
If some of the syntax or tooling feels like it's getting in your way, Elixir looks like a nice setup on top. Personally I think erlang is fine as it is, though I do want to experiment with Elixir.
Interesting result from this study - erlang implementation is not only more robust but also faster.
Lightweight processes, messaging, OTP, supervision all play a role when it comes to runtime fault tolerance.
Erlang is a rare beast in that the language and its VM are both very opinionated and very focused on certain types of problems (such as telecom and fault tolerance). This is not Java and its VM trying to be all things to all people.
1) Go will crash or at minimum (arguably much worse) go into an unknown state.
2) C++ will crash, and ideally, restart, but lose all "live" state. (So calls get dropped, etc.)
3) Erlang/Elixir process will crash and then get instantly restarted by the supervisor. If the bit flip hit the supervisor instead, then its supervisor will restart it (most larger BEAM apps have a supervision hierarchy). The only way BEAM will go down is if the cosmic ray hit BEAM.
Checkmate, non-BEAMs? ;)
@brightball Go has had first class support for cross compilation for a while now, no?
So basically use Golang as a modern C for specialized tasks which require [almost] the best your hardware can do, and use Erlang/Elixir for everything else.
The other trade-off that comes from mutable versus immutable data comes from clustering. With Go, you have the ability to make remote procedure calls very seamlessly if you want to implement them, but because of pointers and shared memory, if you call a method on another box with an argument that references to something on your machine, it can’t be expected to function the same way.
Never used it, but lots of people in conferences are happy with it.
I think that this makes a lot of sense. My experience in just about any language, is that the official means of error handling already feels like a code smell, even before you start using it. And if that's not the case, then it still manages to feel that way when used in a large project.
Lots of Smalltalk projects would actually handle errors by saving the image on an unhandled exception. For server applications, this was like having a "live" core dump where the debugger could open up right on the errant stack frame, and you could perform experiments to aid in debugging.
Erlang just let it crash and you can restart it with a supervisor process. I think most of the time this model is much better since you won't be doing numerical stuff with BEAM anyway, it's not for it and it's too slow.
I know the syntax is more similar to Elixir, but the format seems closer to Go in the sense that it compiles to a binary.
Or in other words, it is very inspired by the Go concurrency solution.
This is an implementation detail.
Anyone can write a VM for Go, or an AOT compiler to native code for Elixir.
Of course, writing very good ones is a different matter.
A CS degree just makes it easier.
Which would be useless for production without several years of maturity, a tools ecosystem, and a community adopting them -- and of course continued support.
So this is mostly theoretical.
> So this is mostly theoretical.
No, it is setting the facts straight about the widespread ignorance of mixing languages with implementations.
No, but they are only relevant for the purposes of the discussion, that is, when considering whether to adopt a language platform based on if it's AOT or interpreted etc, when they are v1.0.
That somebody can always make an interpreter for an AOT language, for example, is nothing people care about when checking whether to use a language for their projects. It's what's there that matters.
Anything else will just add entropy and development costs to projects, due to 2nd class tooling and lack of libraries on the target platform.
Then it doesn't matter how the code gets compiled at all.
That's true, but for application programming. For the server side there are other considerations. WhatsApp can get away with using Erlang for example, and still get the sweet billions.
The current setup is also mentioned on Wikipedia's page about AOT: https://en.wikipedia.org/wiki/Ahead-of-time_compilation
Also even Erlang has HiPE.
Not everyone cares about compiler design.
Every programming language has a runtime, even Assembly (microcode).
Instead if you project is a higher level application server, you'd choose between Go, Elixir, Erlang, Java, Python, Node etc. Go should be compared with these languages and ecosystems.