Hacker News Reader

Comment by bluefirebrand | Hacker News Reader

Nah, you just throw it out and have the AI generate an all new one with different problems!

I really look forward to all programs now having new strange bugs every release. They already do, but I expect AI to do that more at first.

I don't know if AIs will ever get really good at QA in general, but I do think that AIs can get quite good quickly at regression testing.

The hacking opportunities will be endless. Feeding AI the exploit will be new.

That's how bads use GPT to code. The right way is to ask GPT to break the problem down into a bunch of small strongly typed helper functions with unit tests, then ask it to compose the solution from those helper functions, also with integration tests. If tests fail at any point you can just feed the failure output along with the test and helper function code back in and it will almost always get it right for reasonably non-trivial things by the second try. It can also be good to provide some example helper functions/tests to give it style guidelines.

If you're already doing all of this work then it's trivial to actually type all the stuff in yourself

Is GPT actually saving you any time if it can't actually do the hard part?

It's not really "all this work," once you have good prompts you can use them to crank out a lot of code very quickly. You can use it to crank out thousands of lines of code a day that are somewhat formulaic, but not so formulaic that a simple rules based system could do it.

For example, I took text document with headers for table names and unordered lists for table columns, and had it produce a database schema which only required minor tuning, which I then used to generate sqlmodel classes and typescript types. Then I created an example component for one entity and it created similar components for the others in the schema. LLMS are exceptionally good at this sort of domain transformation, a decent engineer could easily crank out 2-5k lines/day if they were mostly doing this sort of work.

Now your description of "good prompts" to reuse has created an abomination in my mind. I blame you.

The abomination: prompts being reused by way of yaml templating, a Helm chart of sorts but for LLM prompts. The delicious combination of yaml programming and prompt engineering. I hope it never exists.

You know with GPT you can do these steps in a language you are not familiar with and it will still work. If you don't know some aspect of the language or it's environmental specifics you can just chat until you find out enough to continue.

How do I know if a problem needs to be broken down by GPT, and how do I know if it broke the problem down correctly? What if GPT is broken or has a billing error, how do I break down the problem then?

1. Intuition built by trial and error 2. Domain expertise backed by automated checks 3. The old fashioned way, and if your power is out you can even bust out a slide rule

Reply on news.ycombinator.com