Hacker News Reader

Comment by dnautics | Hacker News Reader

If by "done right" you mean "the working programmer's smalltalk". Erlang and, probably even more by philosophy, Elixir, pushes you towards Kay's model, but the underlying BEAM gives you enough "cheats" to let you be incredibly productive. Especially in elixir those cheats are sufficiently ugly (as in code esthetics, nothing more) that you don't really want to use them unless you must, and when you do use them it's reasonably good practice wrap it in a macro so that the implementation details can be hidden away. In some cases where your using a non-kayish cheat (like it only helps to make your test parallelize over stateful services) you can make your macro no-op in prod.

View on HN

The Actor Model has been used in many important large applications [Bonér and Reisz 2017, Cesarini 2019]. In order to implement next generation Reusable Scalable Intelligent Systems [Hewitt 2019], the following extensions to Erlang are needed:

* Automatically reclamation of storage of an unreachable future [Baker and Hewitt 1977] (e.g. process in Erlang). For example, an Erlang process can be orphaned if its Process Identifier becomes inaccessible.

* Language support for holes in the region of mutual exclusion of an Actor implementation. For example, in Erlang it is necessary for application programmers to explicitly code a message handler for each re-entry into the region of mutual exclusion potentially enabling cyberattacks from outside the process.

* Automatic cancellation of requests that have taken too long. For example, in Erlang it is necessary for application programmers to explicitly code a time-out for each message send that might take too long.

> For example, an Erlang process can be orphaned if its Process Identifier becomes inaccessible.

Determining this property of a program seems to me to be about as hard as the halting problem, since (1) an Erlang process can send its Pid to another process in a message, and (2) the VM/compiler can't tell a priori whether an erlang process will access its own Pid, or even send a message, in the future. I haven't read the paper you linked though, so I'm sure they found a way around this.

We need a strongly typed version of Erlang so that garbage collector can tell when a future (i.e. process) inaccessible.

Do you feel it's possible to achieve that while retaining the hot code loading mechanisms and flexible message handling that allows distributed systems to process versioned, legacy messages?

Thanks for contributing to the thread, Professor Hewitt.

Should be possible to achieve your goals. Researchers are working on prototypes.

You are very welcome!

As a fan of Elixir, I'd love to hear Jose Valim, Chris McCord or any of the other core devs to chime in on this subthread!

That's what Pony was trying to do, but sadly it's lost most (all by now?) of its momentum.

Yeah I do still check in with Pony from time to time and it is still puffin along but it has lost most of its momentum. Imo it went to far in the classes direction for its objects and started to have a too complicated syntax, while still missing making some parts clear and easy to get that imo needs to be very clear in the syntax of a high performant language. Things like if a thing you create needs to be allocated or not. I have several times thought about implementing something Pony / actor model -like in zig or something, but without all the classes stuff.

It wasn't strongly typed exactly, but Microsoft's DCOM protocol featured a simple distributed garbage collector that would automatically unref objects and allow them to be deallocated in some cases. DCOM allowed processes to send handles to (relatively for the time) strongly typed interfaces for objects across the network, in a form of PID that was recognised on a LAN and could be resolved back to the original object in the original process. It was called an OXID (object exporter ID), if I recall correctly.

It's not well known but what we now call the actor model was pretty widespread in Windows programming in the 1990s. Partly because Visual Basic was like JavaScript: single threaded runtime, no shared memory. To aid developers in exploiting parallelism and using blocking APIs you could create parallel "workers" to use JS terminology (apartments in MS speak), but Windows gave you object oriented RPC between the workers. So it was an RPC layer on top of message passing. You could do asynchronous RPCs if you wanted to:

https://docs.microsoft.com/en-us/windows/win32/rpc/asynchron...

This was used to great effect in rather mundane programs like InstallShield, where the GUI ran in one worker and the install engine ran in an isolated (but in process) world, with the two communicating using RPCs/typed message passing.

for anyone else wondering what this dude is taking about https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3428114

“this dude” appears to be Carl Hewitt, creator of the actor model

Promise management features are coming to Monte soon, satisfying your first and third bullet points. The second point, of course, comes for free with E-style vats.

What is the precise definition of a "promise"?

I.e., what is the interface of a promise?

Monte is based on E, so they are using E-style promises. An example of ActorScript in Monte is at [0].

There are two main operations that can be done with a promise. We can wait for it to complete, and then perform another action when it is complete, known as "when-blocks"; we can also send a message on top of another sent message, informally known as "promise pipelining".

When-blocks:

    when (action) -> { doMoreStuff() }

Promise pipelining:

    promise<-add(4)<-subtract(3)

[0] https://groups.google.com/forum/#!topic/cap-talk/y1bEqOoeswE

Promises are used most prominently in current-generation JavaScript where they lack a proper interface. Instead promises are standardized using language constructs. Promises have the well-known disadvantage that promises are contagious, i.e, once a promise is used then everything that continues must be promises.

On the other hand, a Future [Baker and Hewitt 1977] has the following well defined interface:

   Future<t> *interface* resolve[] ↦ t

Not sure what apps you refer to in your two citations, one is probably just about Akka and not an app per se. And these two papers were written in the last two years.

There is still no credible evidence that the actors system is better than the more traditional way of writing distributed apps, especially now that more and more languages offer an increasing amount of asynchronous approaches.

To see if it is Actor, you have to check if it follows the rules. Akka and Erlang have been used to implement some important scalable systems.

What do you recommend as a "traditional" way to implement scalable systems?

> Akka and Erlang have been used to implement some important scalable systems

Very few, really. Maybe you can come up with a few (besides the Ericsson router), but they are a tiny, tiny minority of how we build scalable systems today, and even if some systems are built this way, it's still no evidence that using Akka or Erlang was the best technology.

> What do you recommend as a "traditional" way to implement scalable systems?

I'm not recommending anything, I'm just observing that a crushing majority of scalable systems in use today, reliably serving billions of people and billions of API calls, are built on traditional, imperative technologies (C, C++, Java, C#, JavaScript, etc...).

This past decade, these services have been supplemented by increasingly powerful threading (thread pool, fork join, work stealing, etc...) and asynchronous (coroutines, promises, futures, channels, etc...) approaches.

The actor model has so far completely failed to prove itself as an improvement over these traditional approaches.

I also happen to think the actor model is terrible to debug, deeply violates encapsulation, and often forces developers to forego static typing, but that's more of a personal opinion than an observation.

> Very few, really. Maybe you can come up with a few (besides the Ericsson router), but they are a tiny, tiny minority of how we build scalable systems today,

Yeah, it's a small fraction because it's a rare language, but it's responsible for an outsized number of highly-scalable systems: WhatsApp, several massive ad companies, several massive trading systems, several large analytics companies. Plus most of the production XMPP systems that have been used at scale.

Look, I'm not even an Erlang/Elixir programmer, but it's pretty clearly a great technology for building scalable, fault-tolerant systems.

> I also happen to think the actor model is terrible to debug, deeply violates encapsulation, and often forces developers to forego static typing, but that's more of a personal opinion than an observation.

I mostly work in statically-typed functional programming languages at work, so I'm sympathetic to this complaint. But Erlang has the concept of fail-fast and fault tolerance that make the lack of type safety not as big of a deal as it normally would be.

It's very hard to know exactly how much of WhatsApp uses Erlang (there is hardly any documentation of that besides vague public release docs), and at the end of the day, like I said, just because a system is written with a technology doesn't mean it was the best idea.

> But Erlang has the concept of fail-fast and fault tolerance that make the lack of type safety not as big of a deal as it normally would be.

That "fail fast" / "let it crash" thing that Erlang brags about has never been convincing to me. First, because you still need to implement all these supervisors yourself anyway so the complexity is still there. Second, it's an admission that because of its lack of static typing, Erlang developers are just throwing their hands in the air and try to make pass what is a fundamental flaw in the language (that you can't effectively and exhaustively catch errors) as a strength.

Besides, Erlang doesn't have the monopoly on "let it crash": plenty of languages have better ways to catch and recover from errors, either through exceptions or algebraic data types that allow the compiler to make sure you have covered all the error cases.

> First, because you still need to implement all these supervisors yourself anyway so the complexity is still there.

Have you implemented an erlang/elixir supervisor? It's not hard. It would probably take me about 60 seconds.

> Besides, Erlang doesn't have the monopoly on "let it crash": plenty of languages have better ways to catch and recover from errors, either through exceptions or algebraic data types that allow the compiler to make sure you have covered all the error cases.

Simply handling the error is not the primary reason for "let it crash". The reason for let it crash is to have the remainder of the system continue to make forward progress and with minimal disruption.

I think if anything the greatest thing about let it crash is that I can have a relatively green junior put something into prod without worrying that an uncaught error will bring down the whole system.

I don't think there is a traditional way of writing distributed apps. The whole field is extremely young(/aka immature), and the ways are still being discovered. The actor model is just one way, but it lines up fairly nicely with what Joe Armstrong was doing with Erlang. I think they both deserve high marks.

> an Erlang process can be orphaned if its Process Identifier becomes inaccessible.

Has this ever actually happened? Surely with as many deploys as erlang has, it would have been observed. A casual google search of "orphaned erlang process" shows the parent comment in the top four results, and no other results that mention orphaned erlang processes.

My point about erlang is that it's great precisely because, like all great systems, it doesn't dogmatically adhere to the academic design, and rather makes compromises and is refined by practitioners.

> re-entry into the region of mutual exclusion potentially enabling cyberattacks from outside the process

Typically you run your erlang actors inside of a trusted system. The only way to give untrusted actors access to your system, is explicitly (so no one does it). There are warning signs all over the place around every place where that might happen (for example distributed erlang) and in prod you really really shouldn't deploy distributed erlang without tons of security in depth in a deeply buried vLAN, or, TLS (instructions provided in the standard libs).

The only realistic vectors of attacks on your actor system is by compromising the entry points, like the actors which live to process your TCP or TLS entry points, but you can bet that ericcson has taken a lot of effort to harden those systems and, in fact, the ericcson SSL system was not vulnerable to the Heartbleed attack (despite using OpenSSL) because there are some very smart engineers at ericcson, and also the language generally doesn't have buffer overflows.

BTW, thanks for coming to my stanford ee380 talk a couple of Februaries ago -- unfortunately I wasn't aware of what you looked like, so I didn't get that you were in the audience at the time.

Erlang's insecurities will become more of a problem in the implementation of Reusable Scalable Intelligent Systems. See the following:

    https://papers.ssrn.com/abstract=3428114

PS. You are very welcome. Thanks for your EE380 talk!

I believe you're mistaken about this one:

> Automatic cancellation of requests that have taken too long. For example, in Erlang it is necessary for application programmers to explicitly code a time-out for each message send that might take too long.

https://hexdocs.pm/elixir/GenServer.html#call/3

> timeout is an integer greater than zero which specifies how many milliseconds to wait for a reply, or the atom :infinity to wait indefinitely. The default value is 5000. If no reply is received within the specified time, the function call fails and the caller exits.

That's the gen_server functional interface, not the message send primitive (!).

Aha, thanks. I guess I'm not clear as to why an actor would care about the egress time of a message (unless it were waiting for a response)

That's OK. It doesn't matter at what abstraction layer the timeout is taken care of as long as it is taken care of. That it's in a library rather than the language core is an irrelevant distinction.

It’s technically the receive primitive doing this using the after clause. The addition of unique references and selective receive then allows “call” style interactions to built (like gen_server).

Likewise, links and monitors are used for lifetime concerns instead of reference counting actors like Pony or using a global tracing collector. This has some advantages and disadvantages but for the most part I personally find it far more practical than other models which are more like the classic actor model.

The process id is not the only way to code the sending of a message in Erlang. You also can send a message to a named group if I remember right. Also, if you are using a supervisor tree (and for anything non-trivial you should), then the supervisor retains the process id. If that was not enough, as part of the Erlang runtime there is a process that handles binding and naming lookups, and it has the process id. Really don't see how this could happen.

I consider bullet 2 to be a feature of Erlang, not a bug. If a module implements a behavior you know it implements those functions, there's no checking needed. Yes, it increases your attack surface, but outside apps should not be messaging Erlang processes directly.

Bullet 3 seems like sugar. You can wrap the call in your own code that specifies a default timeout.

I'm not saying your concepts are wrong, they are interesting and a more complete/accurate Actor model implementation in Erlang could be a good thing. However, the points you specifically made about Erlang do not seem to me to be needed adds to the language.

> Bullet 3 seems like sugar. You can wrap the call in your own code that specifies a default timeout.

Which is exactly what GenServer does. You rarely need to directly send a message to an Erlang process - that is usually done under some abstraction providing the properties that you need.

Reply on news.ycombinator.com