Hacker News Reader

Comment by pfdietz | Hacker News Reader

I don't see why it shouldn't be even more automated than that, with LLM ideas tested automatically by differential testing of components against the previous implementation.

EDIT: typo fixed, thx

View on HN

Defining tests that test for the right things requires an understanding of the problem space, just as writing the code yourself in the first place does. It's a catch-22. Using LLMs in that context would be pointless (unless you're writing short-lived one-off garbage on purpose).

I.e. the parent is speaking in the context of learning, not in the context of producing something that appears to work.

I'm not sure that's true. Bombarding code with huge numbers of randomly generated tests can be highly effective, especially if the tests are curated by examining coverage (and perhaps mutation kills) in the original code.

Right, that method is pretty good at finding unintentional behavior changes in a refactor. It is not very well suited for showing that the program is correct which is probably what your parent meant.

That doesn't seem like the same problem at all. The problem here was reimplementing the program in another language, not doing that while at the same time identifying bugs in it.

Conversion of one program to another while preserving behavior is a problem much dumber programs (like compilers) solve all the time.

I'm assuming you meant to type

> I don't see why it *shouldn't be even more automated

In my particular case, I'm learning so having an LLM write the whole thing for me defeats the point. The LLM is a very patient (and sometimes unreliable) mentor.

Reply on news.ycombinator.com