Hacker News Reader

Comment by mccoyb | Hacker News Reader

Broadly agree with the author's points, except for this one:

> TypeScript/Node.js: Better concurrency story thanks to the event loop, but still fundamentally single-threaded. Worker threads exist but they're heavyweight OS threads, not 2KB processes. There's no preemptive scheduling: one CPU-bound operation blocks everything.

This cannot be a real protest: 100% of the time spent in agent frameworks is spent ... waiting for the agent to respond, or waiting for a tool call to execute. Almost no time is spent in the logic of the framework itself.

Even if you use heavyweight OS threads, I just don't believe this matters.

Now, the other points about hot code swapping ... so true, painfully obvious to those of us who have used Elixir or Erlang.

For instance, OpenClaw: how much easier would "in-place updating" be if the language runtime was just designed with the ability in mind in the first place.

View on HN

> 100% of the time spent in agent frameworks is spent ... waiting for the agent to respond, or waiting for a tool call to execute. Almost no time is spent in the logic of the framework itself.

But that’s exactly where multi threaded Elixir is better! You want a single thread like Node for CPU bound work, you want extreme multi threading for I/O bound work like AI agents. In Elixir you can do both: heavy CPU work without worrying about stopping the world, and heavy concurrency across millions of threads where work is I/O bound and you want to saturate your network connection. In Node you can’t do either of those things easily - it’s just a single thread.

> Even if you use heavyweight OS threads, I just don't believe this matters.

It matters a lot. How many OS threads can you run on 1 machine? With Elixir you can easily run thousands without breaking a sweat. But even if you need only a few agents on one machine, OS thread management is a headache if you have any shared state whatsoever (locks, mutexes, etc.). On Unix you can't even reliably kill dependent processes[1]. All those problems just disappear with Elixir.

[1] https://matklad.github.io/2023/10/11/unix-structured-concurr...

Presumably if you can afford to pay for all those tokens the computational cost should be mostly insignificant?

Spending too much time optimizing for the 1% of extra overhead seems suboptimal..

Even if I was building one single agent for a one off hobby project I would still use Elixir. It’s elegantly suited to the job. With any long running failure prone process you’re constantly writing try/catches, health checks, network timeouts, retries, and a whole lot of other orchestration stuff just to keep a flaky real world agent running. With Erlang it’s all just a built in state machine. The process knows what state it is in, crashes, recovers gracefully in exactly the right state.

> How many OS threads can you run on 1 machine?

Any modern Linux machine should be able to spawn thousands of simultaneous threads without breaking a sweat.

Reply on news.ycombinator.com