Hacker News Reader

Comment by Filligree | Hacker News Reader

The biggest benefit, I’ve found, is it makes me comment my code. If I can make the AI understand what I want, then it turns out that three months later I’ll also be able to understand the code.

View on HN

That's the worst part about generative AI IMO - it makes writing new code faster - it barely helps with editing existing code. So when someone eventually updates the code and forgets to update the comments I wouldn't be surprised if the misleading comments made AI hallucinate.

I believe that AI will get so good at creating new code that a lot of existing libraries will be let unused. What is the point of using lots of libraries if AI can generate the code we need directly? The AI will be the library itself, and the generated code will embed the knowledge about doing lots of things for which we used libraries.

>What is the point of using lots of libraries if AI can generate the code we need directly?

Theyve been debugged.

And also have documentation.

At some point the models will produce code with a lower error rate than existing libraries.

How do you ensure quality?

Review and testing.

Reviewing is easier when there is less code (i.e. libraries are in use)

It’s crazy how many people miss this. GPT models can review code too! They can also write and run tests. Once the context window is big enough to fit the whole code base into it they will be better at review than you are. Eventually we’ll have fine tuned models that are experts in any subject you can think of, the only barrier is data and a lot of recent research is showing that that can be machine generated too.

GPT 4 pre nerf was terrible at reviewing non-trivial or non textbook code. I've decided to test it for a few weeks by checking stuff I caught in review or as bugs, to see if it would spot it. It was like 0% on first try (would always talk about something irrelevant) and after leading it with follow up questions it would figure out the problem half of the time and half of the time I'd just give up leading it.

These were tricky problems that were small scope - I've picked them so I could easily provide it to GPT for review.

So I doubt larger context window will do much.

It’s hard to tell why you ran into such a problem without seeing how you prompted but I can offer a few pointers. Use the OpenAI playground instead of chat, it allows you to specify the system prompt and edit the conversation before each submission. System prompt is good for providing general context, tools and options but you absolutely must provide a few example interactions in the conversation. Even just two prompt and response pairs will strongly influence the rest of the conversation. You can use that to shape the responses however you like and it focuses the model on the task at hand. If you get a bad response, delete or edit it. Bad examples beget more bad responses.

What's the point of comments then. Just feed it back in in three months.

Reply on news.ycombinator.com