`async: false` is the worst | Hacker News Reader

`async: false` is the worst

saltycrackers.dev

23 pointslobo_tuerto12 comments

I wouldn't have expected this much dependence on global state in erlang software. I thought erlang was all about encapsulation, but I guess some problems are basically universal to production software no matter what language you use, and global state is one of them

You have to be careful about your definition of "global state". We kind of have this definition based on C programs from the 1970s, but it doesn't always trivially translate into more modern architectures. BEAM programs (Erlang/Elixir) are basically microservice architectures by default. They lack global state in the sense that there is no way for process (in the BEAM sense, not the OS sense) A to reach into process B and directly manipulate the values. That is the form of isolation that BEAM and its accompanying languages offer. But for any network architecture to function, there's always going to be unique identifiers for the actors within the network, and that is intrinsically at a higher level than a single process. It doesn't have to be "global", in that two processes may have a different idea what it reaches if it asks for "leader", but it's certainly bigger than the single process. And yeah, there's not a lot that can be done about the fact that's going to be somewhat global, and if the identified network targets are not in the local OS process, intrinsically mutable in the sense that the state of those services can at the very least go down without notice.

In the modern world there are many useful definitions of "global state", some of which as I suggested may not even be "global" but may be in some scope that lives between "global" and "local", and depending on which definition you choose, various languages may have different "globalnesses" in their state.

(Another common example is, "does the language have global variables?" Many languages have the concept of "packages", that you can import once into the system, and everyone will see the same "print" package. That package may have variables, like for example maybe "print.default_format". Is that a global variable? Well, in the old C sense, maybe not. "default_format" can not be reached simply with "default_format" from all locations in the program, such that is globally defined. It can't be reached at all from places that did not import "print". But if what you're concerned about is whether changes made by one module are witnessed by all, they are, so from that sense it is a modern global variable, even if more nicely organized than traditional ones. There isn't a right or wrong answer here, it depends on the definition you choose which answer you get.)

In general, immutability at the value level does not automatically make things encapsulated. It more easily affords it, but it doesn't just happen.

I think a slightly more succinct way of thinking about mutability in actor systems is this: (I think it's from Erik Meijer, but not quite sure.)

Ultimately, any actor system has global mutable state via message queues. You can perfectly emulate a shared mutable variable by just sending it back and forth between message queues. (At least it's only atomically modifiable, I suppose.)

It's very different from abient authority to modify a mutable variable, but still...

This is one of the reasons I tend to consider "immutability" a bit of a misfire. A completely understandable one, an inevitable one, and one I am not at all critical of, but it is not the way to build systems. Mutability is a fundamental fact of life. The key is control over mutability. Uncontrolled mutation is the problem. Code needs to know that if it reaches for a value of "x" now, three lines later it will not be different unexpectedly. Code needs to be able to know when it is "updating" with the rest of the world and thus all the old guarantees are gone, but whatever value it sees now is now reliable until the next sync. Uncontrolled mutability is the problem. It is difficult, and arguably simply impossible, to write correct code when your foundation is constantly unobservably shifting. Even single threaded code becomes a monster if you call a function and it starts scribbling all over whatever variables it feels like.

Transactional isolation and Rust's borrow checker are in my opinion much closer to what we are really reaching for.

Immutability is still a nice simplification of those things, though. Both of those things are quite difficult to work with. Full, true transactional isolation tends to deadlock at the drop of a hat, and Rust's borrow checker is much more complex conceptually than immutability. It can be a reasonable choice to work with full immutability in some circumstances.

I don't think anybody (even us Haskell weirdos) are clamoring for full immutability of anything. For data structures it's AMAZING unless you really cannot live with a log factor, but that's neither here nor there.

Anyway, for me it's all about controlling and sequestering off (shared) mutability... and -- at a slightly higher level of abstraction -- capabilities given to random bits of code. Being able to conclusively say that a function "foo" can only perform operations on a "User repository" (or whatever) and not randomly go prodding the database directly is amazing.

(Think of "mut" as a capability and you'll probably get the gist of what I mean. It's just a bit more general.)

this can be using a test database for integration tests, etc.

at least it is in these situations I have had to consider the async setting.

> async: false is the worst

I suppose the reason is because it takes more time? (The article doesn't mention, and just takes it as a given)

> I suppose the reason is because it takes more time?

I assume so. I didn't even notice that the article didn't motivate that `async: false` is bad. I always avoid it if I can since you might as well perform independent tests concurrently.

From the docs [1]:

* `:async` - configures tests in this module to run concurrently with tests in other modules. Tests in the same module never run concurrently. It should be enabled only if tests do not change any global state. Defaults to `false`.

[1]: https://hexdocs.pm/ex_unit/main/ExUnit.Case.html#module-tags

One feature I've never seen in test frameworks that I think would be good:

Instead of concurrency being a boolean on/off, tests should be able to declare a list of shared dependencies (e.g. using arbitrary string identifiers). Then the test runner could make sure no two tests with the same dependency run at the same time, but still use as much concurrency as possible.

You could extend this further by marking certain dependencies as read-only, so the read-only tests can still run in parallel with each other.

this sounds suspicious. bits of code are going to be walking the whole process tree looking for something that is not going to be there when the code is running in production. i guess the operation is relatively inexpensive so maybe it is ok.

Hey, I wrote the blog post under discussion. It's good to bring suspicions! But yeah, the rarest situations aside, there's not going to be any relevant performance impact.

For one thing, in production, for a given key, the process tree will be "climbed" only once by any calling process. Thereafter, ProcessTree will cache, in the process dictionary of the calling process, either the value it "finds" in the tree, or a provided default value. For any later requests for the data, the calling process will find the cached value in its own process dictionary with a single, local lookup.

Along with persistent terms, a process looking in its own dictionary is as fast as data access gets on the BEAM. Faster, for example, than reading from the global application environment. In situations that warrant micro-optimization, the process dictionary is something to be leaned into, not away from.

In considering the initial "climb", we're talking about accessing the dictionaries of maybe 10-ish parent processes. In typical situations like mounting a LiveView, time spent climbing the tree will be dwarfed by other factors.

The main theoretical performance issue of climbing the tree is copying the full contents of other processes' dictionaries into the space of the calling process. Unless there's drastically large amounts of data in the parent dictionaries, it's not going to matter. Even then, OTP 26.2 introduced a new API for accessing specific keys in the dictionaries of other processes. Once an upcoming Elixir release supports 26.2, ProcessTree will integrate the new API, and the theoretical impact of copying full dictionaries will no longer be an issue.

(Elixir)

Reply on news.ycombinator.com