Hacker News Reader

Comment by randomstudent | Hacker News Reader

The author doesn't talk a lot about preemptive scheduling, which is probably the best thing about the Erlang virtual machine. This video explains what preemptive scheduling is and why it is extremely useful: https://www.youtube.com/watch?v=5SbWapbXhKo

First, the speaker creates an application endpoint that runs into an infinite loop on invalid input. Then, he shows how that doesn't block the rest of the requests. Using native BEAM (EDIT: BEAM = EVM = Erlang Virtual Machine) tools, he looks for the misbehaving process (the one running the infinite loop), prints some debug information and kills it. It's pretty impressive.

Another great (and lighter) resource, is "Erlang the Movie" (https://www.youtube.com/watch?v=xrIjfIjssLE). It shows the power of concurrency through independent processes, the default debug tools, and the power of hot code reloading. Don't miss the twist at the end.

View on HN

I completely agree. I used to be a big believer in monad based concurrency using Promise/Future/Async/etc. But the simplicity of using preemptive, user-space scheduling has changed my mind. It makes concurrency, especially IO concurrency, cleaner. There are fewer moving parts in your code because the VM is doing a lot of the heavy lifting for you.

Um...you know this is basically threads right?

(not disagreeing, but having watched the past few years as "threads bad, don't know why though..." it is amusing to see thread-style concurrency rediscovered with a new name).

This quote from Joe Armstrong explains the difference better than I can. There are other factors than just this in play of course.

> [Erlang] is a concurrent language – by that I mean that threads are part of the programming language, they do not belong to the operating system. That's really what's wrong with programming languages like Java and C++. It's threads aren't in the programming language, threads are something in the operating system – and they inherit all the problems that they have in the operating system. One of the problems is granularity of the memory management system. The memory management in the operating system protects whole pages of memory, so the smallest size that a thread can be is the smallest size of a page. That's actually too big.

> If you add more memory to your machine – you have the same number of bits that protects the memory so the granularity of the page tables goes up – you end up using say 64kB for a process you know running in a few hundred bytes.

Great point! However, I've found that the implementations of user space, preemptive threads (UPT) have friendlier APIs than the implementations of kernel space, preemptive threads (KPT). Also, UPT APIs generally come with really nice synchronization primitives (like STM in Haskell) or designs that remove the need for them (like message passing on BEAM). Also, I really like not worrying about creating too many threads and having to use a thread pool because the overhead of UPT is generally so much lower.

While it can be a little unclear what people mean, the Wikipedia take on threads is:

"Multiple threads can exist within one process, executing concurrently and sharing resources such as memory"

Whereas Erlang/Elixir processes don't share memory and have their own independent heaps and garbage collection.

Shared memory can lead to complicated effects if two threads try to change the same bit. You don't have that with Erlang processes.

That is like saying queues are basically the same as lists!

size, scheduling and the lack of shared state add up to vast differences. Also the message parsing primitives are way more lightweight than the thread based equivalents.

What do Promises/Futures/Async/etc. have to do with monads? Are monads a good abstraction to model those things?

Continuations / Promises / Futures form a Monad (https://www.stackage.org/haddock/lts-9.1/transformers-0.5.2....)

A Monad is any data structure with the follwing two methods:

    (>>=) :: m a -> ( a -> m b) -> m b
    pure :: a -> m a

the monadic bind is exactly `andThen` in javascript:

    (>>=) :: m a -> (a -> m b) -> m b
    (>>=) :: Future a -> (a -> Future b) -> Future b
    Future.andThen :: Future a -> (a -> Future b) -> Future b

and the monadic `pure` is exactly the `Future` constructor:

    pure :: a -> m a
    pure :: a -> Future a
    Future.always :: a -> Future a

Every Monad is a Functor, so you also get this one for free:

    map :: (a  -> b ) -> m a -> m b
    -- if I know how to convert a's to b's I can convert a future a to a future b
    Future.map :: (a -> b) -> Future a -> Future b

So basically, saying "futures are a monad" gives you all the useful functions that you would expect from a future based API, which is nice. It means you can use all kinds of higher level functions not specific to futures to abstract behaviour:

Like, given a list of element,s and a way to create a future for each element, create a future that returns a list of elements:

    forM :: (Monad m) => Array a -> (a -> m b) -> m (Array b
    Future.forEvery :: Array a -> (a -> Future b) -> Future (Array b)

Noted down, thanks for that. Will be handy for future whiteboard interview questions.

How do you begin to even read that notation stuff? Do you know of any resources for learning it?

The notation in the GP is Haskell-ish type notation, which is certainly different to the majority of languages out there today, but it's actually fairly simple (difficult to learn, but simple).

Quick glossary:

    ::      Defines a new function signature. The left hand side is the function name, and the right hand side is the signature.
    a -> b  A function which converts something of type a to type b
    m a     A generic m of type a. In Java/C# this might be written as m<a>. eg, `Array String` means an array of string elements, and might be written in C# as Array<String>.

Types starting with a lowercase letter (m, a, b) generally mean the type is generic - as in the function will take any type.

It might be helpful to define a function you already know:

    string_to_int :: String -> Int

Or, for changing an Array of something to an Array of something else:

   map :: (a -> b) -> Array a -> Array b

Here, we're defining a function that:

   1. Takes a function that converts an 'a' to a 'b' (a -> b)
   2. Takes an array of 'a'
   3. Returns an array of 'b'

Notice that 'a' and 'b' can be anything! Since the first parameter* is a function that handles the conversion from a to b, the map function actually doesn't need to know what type a and b are.

Let's say we pass in the `string_to_int` as the first parameter, now the type checker will actually infer the following function type:

    map :: (String -> Int) -> Array String -> Array Int

* Haskell functions actually only ever have one parameter, it uses Currying to accept multiple arguments: https://en.wikipedia.org/wiki/Currying

That's the Haskell language. There are a lot of "big" concepts in the language itself, compared to just about any other language from the last 50 years that people get paid to write, but the notation isn't hard to straighten out after a few tries. There's a really good intro here: http://learnyouahaskell.com/

They are just a way to wrap up the code/data somewhat similar to how callbacks are used in Javascript async code. And from using Haskell I believe they are an excellent abstraction when doing any async programming, once you wrap your head around them.

Although in terms of concurrency I've found Elixir/Erlang actor-esque process based system with messaging much more intuitive when building complex programs and much simpler to reason about than the various solutions Haskell offered. But I'm also not an expert at Haskell so take that comparison with a grain of salt.

Many of the actual implementations of these types are not actually monads by the monad laws, but they are monad-like in that they are containers for execution that can be composed with each other.

I have just started reading Elixir in Action[1] by the same person in the video. I've been enjoying it so far, I think things like the stuff demonstrated will be in the book.

[1] - https://www.manning.com/books/elixir-in-action

Preemptive scheduling is great, but there are some times when it is bad. For example, I was having problems in my Elixir code for live streaming that it was taking too many reductions to fully deserialize and process the incoming message (nothing major was in this code path, just a parsing the TCP packet and unwrapping the different layers). This ended up causing the process to get (what seems to be) overbalanced and I ended up at having delays transferring 5+mbps video through it.

I broke the deserialization up into 3 separate processes (1 for handling the I/O, 1 for packet deserialization, 1 for message deserialization) and while this fixed all the issues (presumably because it can balance the processes more effectively since no single process went over the reduction limit) it turned the effectively synchronous parsing process into a split async process and caused a lot of extra complexity around that.

I'm sure there's more to what caused the issue than just reduction limits but it also made me wary about the magic happening under the hood.

Not just scheduling — the ability to forcibly terminate processes itself is a key part of why Erlang can be resilient to failures.

It's a key area where I think Go went wrong. Without the ability to kill a goroutine, you can't implement caller-enforced timeouts and canceling — anything that should be able to be canceled has to explicitly check for some signal sent by the controlling party. This is why Go ended up with Context, which infects everything it touched.

And you also can't create Erlang-style supervisor trees; you can't create self-repairing hiearchies of control. And you can't do things like use a debugger to attach to a process and terminate runaway goroutines.

You may have been able to bump the process priority to give that process more scheduler time if it's a critical process in a hot path. Perhaps you tried that though. Definitely have to be careful with it, but always an option to keep in mind!

I haven't tried that, but I'd have to make sure that every inbound and outbound RTMP connection had high enough priority to correctly execute, and that seemed like a lot of tweaking that had to be done in order to maintain the constant flow of video throughout the app with low latency.

At least for my operation it looked very much like I would really need to become a BEAM vm expert to make sure things ran smoothly (not just with that but other potential issues I had in the back of my mind as well).

> At least for my operation it looked very much like I would really need to become a BEAM vm expert to make sure things ran smoothly

From someone with zero experience in building concurrent systems: isn't this true for every sufficiently complex system? If your system is complex enough, then no general framework will help much, right?

At some point sure, but I can get concurrent systems up and running pretty easily in Go, C# and Rust without going too deep into the underlying details of the asynchronous framework. Erlang's VM does a lot more than any of those frameworks or runtimes do though because of how it handles per process garbage collection yet globally shared binaries, the preemptive scheduler and how it balances process load, the hot-reload capabilities etc... It does a lot of things really well but there's a lot of magic behind it, and I seemed to brush against that much faster than I have in other languages.

Do you mind me asking what kind of throughput you are seeing for a process like that where you unpack video across some processes? I'm working on something similar using Go and it'd be great to know if Elixir is a good alternative or not.

I know of some Erlang based platforms that achieve 10gbps throughput. I hadn't done exhaustive testing on my prototype as I've changed gears away from my home-grown solution to modifying some open source C++ media servers that are a bit more proven than my own stuff.

Isn't pre-emptive multitasking the same strategy used by OS threads etc. where some global scheduler decides which tasks are running? I suppose the VM is managing the threads here rather than OS, so "green threads" like Go. Regardless, it would be pre-emptive multitasking in any multi-threaded Java program or Go program. Just trying to understand what the "secret sauce" of the Erlang VM is.

Is it just that context switches are a lot cheaper within green threads rather than the OS thread?

Pre-emptive scheduling is indeed the approach used by most operating systems (those of us who are bit older will remember when the most common version of Windows was co-operative scheduling - that was fun, any single application misbehaving could hold all the resources on the machine and that happened quite a lot!).

The genius of Erlang is that instead of a per machine it was designed to be distributed from the start so you have a lot less inter dependencies when you want to scale across machines (or cores since that's where Erlang/Elixir shines at the moment), that was one of the design goals since it was designed for the kinds of software where that was a desirable quality (back when hardware was way slower than now), dropping a single call/circuit wasn't the end of the world if the rest stayed alive.

Lifted from the wikipedia article :-

    Everything is a process.
    Processes are strongly isolated.
    Process creation and destruction is a lightweight operation.
    Message passing is the only way for processes to interact.
    Processes have unique names.
    If you know the name of a process you can send it a message.
    Processes share no resources.
    Error handling is non-local.
    Processes do what they are supposed to do or fail.

I think when you get down to it there isn't really anything you can do in Erlang you couldn't do in another language, at one time Erlang went through C as a build step, it's that it was a fairly radical departure as a solution to a problem at the time (telephony) that also happens to fit a fair few problems we run into on the internet.

I'm not an Erlang/Elixir programmer (so I've no dog in the fight) but I've been fascinated by the language since I read about it in the early 90's in a programming journal, it seemed alien then and it seems alien now (much like APL).

Amongst my programmer friends Elixir seems to be really popular with Ruby programmers (I'm sure there is a reason but I'm not a Ruby programmer either so I couldn't tell you).

Reply on news.ycombinator.com