Hacker News Reader

Comment by sharperguy | Hacker News Reader

LLMs are famously bad at producing rust code. I'm not sure how much of it is the lesser amount of Rust code in the training data, or just the fact that Rust has a very large number of pitfalls, and a large standard library with many edge cases and things you'd imaging should exist but don't for a variety of reasons. Rust also has a much wider variety in the way things could be structured, compared to something like go where there is often only one way of doing a particular thing.

View on HN

Honestly, I don't think these are problems that Rust has. What I see LLMs struggle with in Rust is more to do with understanding the language semantics at a fundamental level - exactly the things that the compiler statically verifies. For example, they will see things they think are "use-after-free" or "use-after-move", neither of which is a thing in (safe) Rust, because they don't understand that the language does not have these problems.

Largely I think LLMs struggle with Rust because it is one of very few languages that actually does something new. The semantics are just way more different than the difference between, say, Go and TypeScript. I imagine they would struggle just as much with Haskell, Ocaml, Prolog, and other interesting languages.

Obviously you can write a use-after-free in Rust. The fact that it won't compile doesn't really matter when you're feeding the text to a non-compiler program like an LLM. I trust you don't mean to get carried away and suggest that they're somehow grammatically impossible.

I feel like I have had just as much luck with LLMs writing Rust as I have had with Java, Kotlin, and Swift. Which is better than C++ and worse than Python. I think that mostly comes down to the relative abundance of training data for these types of codebases.

But that is all independent of how the LLMs are used, especially in an agentic coding environment. Strong/static typed languages with good compiler messages have a very fast feedback loop via parsing and typechecking, and agentic coding systems that are properly guided (with rulesets like Claude.md files) can iterate much quicker because of it.

I find that even with relatively obscure languages (like OCaml and Scala), the time and effort it takes to get good outcomes is dramatically reduced, albeit with a higher cost due to the fact that they don't usually get it right on the first try.

I have had very good results using Claude to write Rust. My prompting is often something like

'I have a database table Foo, here is the DDL: <sql>, create CRUD end points at /v0/foo; and use the same coding conventions used for Bar.'

I find it copies existing code style pretty well.

Reply on news.ycombinator.com