Comment by abernard1 | Hacker News Reader

Comment by abernard1 | Hacker News Reader

Parent Full thread

I've used Akka for a decade (basically as soon as it came out) and am quite familiar with systems designed by the Erlang/Elixir community.

While actors definitely are a powerful mechanism for reasoning about stateful systems, most business systems are better off without having to design their own _technical_ distributed systems or in-app persistent state model. There is a reason that dedicated systems like Zookeeper, Redis, squid, relational databases, etc. exist, and that's because those things are insanely hard. Those systems also do not need to know about your business problem.

The actor model, and Elixir w/ OTP subtly encourage coupling of technical, distributed systems problems in the same application scope as the business problem. This is where that thorny state comes into play, and I believe it's a fundamentally wrong instinct. (It is also an instinct that is relatively vocal within the Elixir community: "throw OTP at it!", "once you need to scale, you can just add more machines to your problem!", etc.).

I have a rule on design reviews. I ask the team "is this system a technical system or a business system?" If they say "both", I ask them to make it into two systems, one technical and one business. Most likely, a "simple" technical system that does what they want already exists, and we can use it. While OTP and actors are good at designing interfaces or protocols for distributed systems, writing bespoke error handling for your business code with actors is going to be buggy and wrong. Make the business code pawn off state coordination or storage problems to systems with known, provable properties and a remote interface. (That interface will also be usable by other languages, and not tied to the Erlang VM).

> The actor model, and Elixir w/ OTP subtly encourage coupling of technical, distributed systems problems in the same application scope as the business problem. This is where that thorny state comes into play, and I believe it's a fundamentally wrong instinct.

I disagree that Elixir encourages coupling those systems. While erlang comes with no such help, in Elixir, you should be decoupling your distribution problems from your business logic by putting them into different applications within the same umbrella. Sure, there will be a coupling in the sense that they will live in the same VM, but that is abstractly similar to "our microservices all live in {AWS/GCP/K8s}", especially given the level of process isolation that the BEAM affords you.

But for all intents and purposes (program design, testing, heck you can even pick and choose components for deployment) they are decoupleable.

> Sure, there will be a coupling in the sense that they will live in the same VM, but that is abstractly similar to "our microservices all live in {AWS/GCP/K8s}", especially given the level of process isolation that the BEAM affords you.

I disagree that these are conceptually similar. If all my microservices are in a certain cloud, it does not imply that any important state is maintained in those services and not a database. It also does not require me to have my services in Erlang or whatever the actor model is written in. Actor supervision systems (in whatever language) are essentially narcisstic: they have to be in control over the state management lifecycle.

Actors fundamentally are of the philosophy where some state lives in the application, and it's mutated by these incoming messages. This is totally different from systems that scale, where any application "state" is just transient from a request or message in order to facilitate a mutation in a dedicated store outside of the app. There's many reliability benefits to that: language independence, being able to know the operational characteristics of state management don't change out from under you when you release a new version of code, ability to query state operationally without having to run map-reduce across code in your cluster (how fun would it be to write "get me the relevant state of my system" for a custom-written actor hierarchy instead of SQL?).

> Actor supervision systems (in whatever language) are essentially narcisstic: they have to be in control over the state management lifecycle

I'm not sure why you think elixir discourages using databases. Phoenix by default ships with ecto bindings and you're encouraged to use postgres or Amazon rds or whatever for your bulk state management. Phoenix pubsub ships with redis bindings as an option if you'd rather not use distributed erlang.

Fwiw, I have written a multinode graphql query handler that pulls results from actors instead of a database (in this case using a database as a central source of truth is a bad idea because the data being queried lives in "its own databases", libvirt, if you are curious, distributed across nodes and having competing sources of truth can be very hard to manage). Handling these queries was not difficult. And your actors, btw, don't typically live in a very deep hierarchy anyways, you use something like Registry, which lets you query your actors in a flat fashion (the only hierarchical components are supervisors, which don't store much in the way of business logic state)

I think the belief is that if all your state is held in a transactional database, then your actors don't have any state in them and at that point it's not clear what they're doing with messages that the sender couldn't just do directly themselves by talking to the database.

My own experience with Akka style actor designs has been pretty poor. The problem is you lose the notion of a cross-machine call stack. Actors just have mailboxes and process inbound messages/outbound messages, which means you have to maintain state machines yourself and/or handle re-entrancy. Lots and lots of bugs in designs like that.

On the other hand, simply doing blocking RPCs across services where the RPC framework handles re-entrancy and stack consistency for you, it doesn't have that issue. That's how services like Google search are built. They shard across many machines vs talking to a database directly primarily for performance or to enforce team boundaries, not as part of an over-arching architectural design pattern.

Yeah probably 90% of actors in elixir hold state that is relevant to a connection status. Think, tcp state machine or http connection state. It is veeery nice to have a state machine back your inbound http long poll or websockets connections if you want to do serverside rendering or serverside content streaming etc.

The other case is when you're doing something cqrs-ey where you can have competing load balances requests and you need to make sure that writes are logged, committed, then executed, and reads are cached. You should use an actor for that but ultimately your data will reside in a database.

This is totally different from systems that scale, where any application "state" is just transient from a request or message in order to facilitate a mutation in a dedicated store outside of the app.

It's notable that this is the model on which the world is built. This is the model that runs almost all the biggest systems in the world.

Please tell me if I am wrong, but I get the feeling that you conflate the idea of "bad programming and design of a system" with the actor model in general.

You can do just as badly with an object oriented model as you can with the actor model. Bad applications will be bad.

Reply on news.ycombinator.com