Hacker News Reader

I've found that LLMs are only as good as the code it is trained on.

So for basic CRUD style web apps it is great. But then again so is a template.

But the minute you are dealing with newer libraries, less popular languages e.g Rust or Scala it just falls apart where for me it constantly hallucinates methods, imports etc.

View on HN

I spent weeks trying to get GPT4 to get me through some gnarly Shapeless (Scala) issues. After that failure, I realized how real the limitations are. They really cannot produce original work and as far as niche languages, hallucinates all the time to the point of being completely unusable.

Hallucination from software generation AI is worse than software written by a beginning programmer, because at least they can analyse and determine the truth of their mistakes.

They can do surprisingly things if prompted correctly.

But their ability to do complex logic falls apart, and their limits are pretty much a hard wall when reached.

And so another level of abstraction in software development is created, but this time with an unknown level of accuracy. Call me old-school, but I like a debuggable, explainable and essentially provably reliable result. When a good developer can code while keeping the whole problem accurately in their heads, the code is worth its wait (deliberate sic, thank you) in gold.

Reply on news.ycombinator.com