Hacker News Reader

Comment by minimaxir | Hacker News Reader

A RewriteInRustBench would be unironically useful at this point since all the main agents can write it reasonably well despite its relative scarcity in the input data.

View on HN

Rust is the best language for LLMs b/c it gives by far the best debug messages. Just tons of verifiable reward signal for post-training. Even the most rudimentary LLMs can school me on idiomatic Rust

Evidence needed. I think for certain kinds of outcomes it has very strong advantages, but these advantages are not a given as 'best for LLMs' :)

On the other hand, Rust's borrow checker is very picky, and even a frontier LLM still sometimes struggles to respond to roadblocks sensibly (refactoring so whatever it's trying to do can be done safely) rather than stupidly (introducing some horrible global arena thing so it can make the borrow checker go away). A lot depends on how good your instructions are, and how good the existing code is, since bad input begets bad output.

I've (more or less; I've read quite a bit of the code) vibecoded several houndred thousand lines of Rust and I've not seen this happen a single time. It sounds like something it'd do when you ask it to "write a linked list while satisfying the borrow checker". Are you sure you haven't (possibly unknowingly) been giving it instructions which ended up luring it into doing these things?

Couldn't it just look at what std::collections does?

A good eval benchmark suite could really improve this then.

Yep! In our tests, we found Zig to be a pretty good fit to translate C++ codebases.

And static analysis + agents are good enough at keeping the memory management in check. Compared to Rust, there's no magic so it's easy for devs and agents to reason about.

If you're curious: https://github.com/okcontract/oksolc

TBH a "global arena thing" can be very good for performance rather than a bunch of random allocations/deallocations

Not always. In my experience, if you're not working on a small, trivial codebase, LLMs will sometimes just create spaghetti unreadable, inefficient code to satisfy the constraints of the type system/borrow checker.

I've narrowed in on only using Go or Rust generated code (Go for APIs right now) and rust for some TUI or other thing. TS for web interfaces (w/React).

in my experience LLMs write pretty bad Rust code. sure it works, strong typing system, etc., but it's unmaintainable and not readable.

I doubt these two features will be a defining quality of a product soon enough.

Many pieces of software are going to be just blackboxes worked by AI. You will be maintaining output quality and stability and that's it.

I have the same problem with the assembly produced by compilers.

Maybe we should let them do their thing and instead focus our attention on the higher-level stuff like specifications and testing.

From DARPA’s “Translating all C to Rust” TRACTOR program:

https://www.ll.mit.edu/r-d/projects/translating-all-c-rust-t...

I wonder why not to Ada or SPARK.

I wouldn't be surprised if we're already at the point of more LLM-written Rust than hand-written. Models training off models

All the main agents can write Jai code reasonably well despite being even more scarce in input data!

Have each agent rewrite openssl in $lang and call it the RollYourOwnCrypto bench.

Reply on news.ycombinator.com