Comment by mrdoops | Hacker News Reader

Comment by mrdoops | Hacker News Reader

OTP is definitely more of an expert's toolset than a framework for throwing together features quickly. This is a good thing because it's not trying to be more of an abstraction than necessary. With OTP you can make fault-tolerant, concurrent, self-healing applications that handle stateful operations with ease, but you'll still need a fair bit of need-to-know about the callbacks, how to manage process lifecycles, etc.

What a lot of people seem to want is a batteries included framework that does the OTP for you, but doing at a high level of abstraction is difficult. So more commonly you'll see the OTP managed by libraries for more concrete use cases (web servers, database connection pooling, back pressured job processing, etc). Trying to wrap or rebuild OTP from the primitives available in the BEAM is possible, but definitely not an easy task.

Could we potentially have an OTP-like library that cuts down on the boilerplate significantly? Yes. But how much effort would that take and is it much of a win over OTP as-is? Not sure the trade-off is entirely there yet - OTP really isn't that bad. You're still writing less boilerplate than Java in general.

I think we're much better off writing good library-level abstractions over OTP that can be embedded in a supervision tree or used in more specific use cases. If we were to actually attempt some kind of higher level abstraction in the scope of OTP - maybe we can do so with a very different approach like decorated dataflow graphs and runtime property based testing.

Elixir already started the process of extending OTP: Task is a hugely important addition to the family of genserver, genstatem, etc.

Isn't Task an entirely different beast to the gen_* behaviours?

To me it looks like a convenience wrapper around Erlang's primitives. I usually use them directly if I need the flexibility to "await" later. If I don't, I have a small function "run_in_subprocess" (mostly to not run into binary gc problems) that spawns and receives within one function.

The only real advantage I see is that it is separately supervised. When avoiding trap_exit, "spawn_link"ing within a process that is itself supervised has a similar effect, though.

GenServer is also a convenience wrapper around Erlang's primitives. You could just do a loop handling messages appropriately and get something close to a GenServer in very few lines, but once you start thinking about all the edge cases you end up with GenServer.

Task does the same thing but for a usage model that wasn't previously well supported. It basically removes the last reason I ever did spawn_link directly in Erlang.

1) Task makes it easy to supervise non-complicated async workflows. In prod, you REALLY want to supervise. Don't use spawn_link, it doesn't show up in the supervisin tree.

2) Task also implements the $callers process library key. What this means is that if you spawn a process with Task, it knows which process was responsible for calling it (note that this is in general different from "the process that supervises it"). In tests, you can use the $callers value to shard global state in a concurrency-friendly fashion.

Examples:

1. Make a mock in Mox. Spawn using Task. Call your mock from the task, Mox knows that the parent test is and serves the "correct mock".

2. Check out a database sandbox. Spawn using Task. Use the database in your task, Ecto knows what is parent test (and checkout) and serves the correct db view.

3. Make an HTTP request from your test. Stuff the $callers parameter into an HTTP header with term_to_binary and hex encoding (probably user-agent is a good choice). Use a plug to put $callers into your phoenix connection genserver. Spawn a Task that accesses your DB. Ecto knows what is the parent test (and checkout) and serves the correct DB view. So the cool thing is that YOUR REQUEST LEFT THE VM and it still worked! and this is composable too, if you do it right.

1) It shows up in the application view of the observer. Also, if one both processes dies the other one dies too, what more supervision would I want? A Task is also not magically rerun if there is a problem.

2) Every OTP behaviour keeps the ancestor info as well, I could use proc_lib instead of spawning directly.

ancestor is not the same as caller.

In this particular case it is. If I spawn a process directly (and not separately supervised) to run a function asynchronously (or just out-of-process to make the gc happy), then the ancestor is the same as the caller.

Reply on news.ycombinator.com