I understand the need for memory footprint in some situations, but what's the point of seeking performance for a software that mostly calls LLMs and waits?
But seeing how slow claude code and copilot cli are and how much ram they use I'm flabbergasted. If you have long running sessions they can both take tens pf gigabytes of ram and feel quite sluggish.
The people "writing" agents are not themselves experts in how to write performant code. Claude Code is so massive and ugly it can only be realistically maintained by continuing to throw LLMs at it. But that's not a replacement for good software design.
Of it's actually getting OOMed (and not backing off by itself), I'm pretty sure that's logged in dmesg. Or earlyoom or systemd-oomd if userspace is in play and getting there first.
So even if it doesn't create tangible advantage in a particular use case - its still good for the whole industry.
Systems programming is only a tiny fraction of code out there.
Approaching every problem as a systems programming problem is a massive waste of resources and intellect.
Considering how easy this is now, why choose a heavier, slower and less typesafe language?
Easier to read for humans is easy to read for LLMs. A more expressive language will bring about fewer misunderstandings when you apply stochastic tools like LLMs.
Just be sure you don’t choose something heavier/slower that is not more expressive.
Lots of options with no jit or warmup
and is riddled with timeouts and intervals in useffects
The resources I was talking about are developers × time.
>Approaching every problem as a systems programming problem is a massive waste of resources and intellect.
If someone took their time and decided to use Rust maybe their first couple projects won't be as efficient, but they will learn the language and how to use it in the best ways, they will contribute to the ecosystem, etc. Again, I'm looking at the big picture and not specific project case.
So basically Go?
The type system of Go is very weak. I'd say that'd be my main reason to pass on Go, even when the concurrency paradigm fits the project perfectly.
Next to none of them will get you nearly as good cold start times as of native app, if using free java.
There was GraalVM and its ecosystem which included Java Native Image - first thing I’d evaluate if thought about non-server side, performant Java application.
But it all had been sadly swept away by Oracle from free tier.
But then it distributes as one binary and starts in milliseconds.
Rust would have been a better fit (cargo-and-done, smaller binary, quicker to compile); but I wanted to use Kotlin as we use in all other projects.
As for throughput: you have 1-2 requests going at a time, the next one waiting for the reply. What throughput are we talking about?
That's like speeding to the post office and expecting your letter to get to the recipient faster.
Anyways, consider how higher memory usage can affect the systems performance dramatically once the system needs to start swapping memory to disk signficantly
Java is used in plenty of embedded systems and other memory constrained environments. Yes, it’s not going to perform well compared with Rust, but that doesn’t mean it’s an Electron-equivalent bloated clusterfuck of an ecosystem that’s going to eat all your system resources.
1) the agent is probably not the only thing running on the system, so more is just worse generally
2) I am fine if a developer needs Rust or similar to write a resource efficient app. I wonder what the developer could achieve when he put the optimization effort into the Rust app instead.
People are so narrow minded about programming on this forum. They talk as if only Rust fills the void between unsafe C and node.js behemoths. But the reality is there are a plethora of other good languages out there too.
I mean, the post office is not a magic box. Actual people will take your letter somewhere, sometimes batching sends. So running to the post office might actually get your letter in an earlier batch, same as ordering on amazon or your online supermarket in the morning or in the evening might change the delivery time.
Pedantic, I know, but interesting example.
Rewriting things in rust is "cool". Bun did it, other projects did it. Therefore, writing a coding agent in one should be cool too.
And apparently enough HN crowd agrees with it to take the #1 spot on the board.
Having said that, I considered a Go/Rust rewrite of Dirac (https://github.com/dirac-run/dirac) for some modules to support cases when someone wants to run like 30 agents, but it quickly became obvious that, a) while the node event loop is a bottleneck, it is not the sole bottleneck and b) if you have a VSCode extension, you can't totally get rid of TypeScript, so it just becomes the case of bi-lingual project and the maintenance burden that comes with it
The author made the choice. Open sourced it (thanks!). So now we all enjoy more options. Saying author did so because "cool" does not sit well with me. It's feels like you get a no-strings attached gift of significant value and then going saying the giver gave it to be seen as cool.
After switching away from GNOME Terminal + Zsh to Ghostty + Nushell, I started to appreciate how instant everything feels. Why not make everything just as fast?
Problem is it is nowhere near as smart, so what speed I get in conversation gets killed by iteration.
If you install too many of these "single binaries" then at some point you would be better off just having a single interpreter and using npm/pip.
By contrast the Rust binary only contains the machine code for this program and can be directly executed.
Starting with a resource-saving attitude may be a very good long term strategy.
Also: with Rust there are many features of high-level, modern, type-safe, FP-inspired languages that you do not have to miss.
See projects such as picoclaw, nullclaw and more.
At first I also thought that it would be just call and wait, but a lot of work is done locally (any tool calls).
And in my experience is not that much faster to start than more complex software like Visual Studio Code.
- prolonged life of hardware
- less electricity
- less expensive hardware