Hacker News Reader

Comment by choeger | Hacker News Reader

Let me make this very clear: Generating syntactically correct code in various languages that also looks plausible is no small feat. It is, in fact, extremely impressive and will certainly have an impact on SE.

But. Every single test I ran lead to functionally wrong designs from smallish memory errors in C (that hilariously ChatGPT was able to correct ND explain when pointed to) over misplaced/hallucinated methods in python APIs to completely hallucinated perl packages. It never gave me a useful answer.

I don't know if that tells us something about my work vs. your work or my approach to the model vs yours. But it definitely tells me that such a model cannot replace a developer.

View on HN

I have the same experience. I think the transformer-based models are impressive, but whenever I try to actually use them for something, e.g. having ChatGPT write something or using GitHub Copilot, the results I get are terrible.

I was legitimately afraid for my job when GPT-4 was released. After using it, however, I sleep easy now.

You know, I went ahead and paid to access GPT-4 and tried to use it for something that I guess is programming related, but maybe not what you think.

I was using it to try and help me create Arma3 scenarios, which often involves using an ugly little DSL called SQF.

The last thing I asked it about was how to play a custom audio file sitting on my disk. It gave me a simple one-liner, that didn’t do anything. I asked it several questions to clarify, but it seemed confident that all I needed was this one liner to play an audio file sitting anywhere on my disk.

Eventually I said fuck it, and decided to google it myself, and it turns out it’s much more involved than just a single function call with a path as an argument, as suggested by ChatGPT. In fact you have to create an entirely separate config file with classes to represent the audio, and then you can reference that class to play the audio, not just a simple string path.

I’ve had it fail with a lot of other things, mostly trying to make computer controlled units perform deterministic, scripted actions, but it’s hard to tell if GPT-4 or the game’s engine was at fault here, because in my experience making Arma’s AI units do anything reliably is near impossible, no matter how much scripting you do.

I had the same experience, I was worried and now that I've used the tools I sleep easier. But I still worry that these tools will improve dramatically and quickly. Somebody tell me I don't have to worry, please.

If an analogy helps, I think the current AI craze will lead to some obvious and permanent changes to how the world works in a way that’s very similar to the self-driving Tesla hype over the past 5+ years.

Things like ChatGPT might get 90% of the way there very, very quickly for many use cases (similar to how lane keeping and adaptive cruise control is 90% of “self-driving”) but the remaining 10% of edge cases will require humans for a very long time (similar to how we never quite managed to get “100% self driving” because the vehicles still can’t respond reliably 100% of the time). But we still value it because the 90% works.

Once people understand that 10% limitation and once the 90% value has been gained/normalized/integrated, the hype will die down and we’ll be watching a slow grind to figure out how to “fix” the last 10% which is necessary to live up to what’s being hyped today.

"I was legitimately afraid for my job when GPT-4 was released. After using it, however, I sleep easy now."

Just wait until GPT-5, though. It will blow your mind... \s

I can guarantee you that the equivalent code which I generate (taking >100 times as much time) has a comparable error/typo rate. I can say this with confidence because ChatGPT, while not perfect, "knows" lots of things I don't know and absolutely makes me 10x as efficient. Now, I am a pretty crummy software engineer so ymmv, but it's worth noting how far the goalposts have moved for assessing AI: from "simulates understanding of language in a general way" to "comparable in any way to a professional specialist in a random task (and much faster)". A few months ago everyone was losing their minds over how good ChatGPT was, and while some of that was overblown, nothing has fundamentally changed except we have become used to it and are looking for the next shiney tech thing. At least we're not talking about crypto anymore.

>But it definitely tells me that such a model cannot replace a developer.

Certainly not an average developer (however that's decided in C's culture). Not making any statements on skill of anyone here, but I'd bet it could replace the very low ends of skill as it is (GPT-4 and maybe Bard). I doubt there's too many of those kinds of developers though.

It's not a guaranteed positive for all languages/people, but it's farther along being useful as an enhancement technology than it is as replacement technology. I think it's making progress on both axes, but one is clearly ahead of the other and it's still possible it stops progress until other innovations are made.

ChatGPT which version?

Even Copilot, which is a smaller model, generates working code most of the time.

As a counterpoint, I am absolutely horrible at writing Bash but I used ChatGPT to write a fairly complex (~50 lines, multiple functions) script in Bash that works just dandy. It wasn't perfect, and I made a couple adjustments, but it was 100x faster than what I could have done without it.

As a counter-counterpoint I tried to use GPT to avoid learning bash argument parsing. The script was buggy. After about 30 minutes of iterating with ChatGPT I gave up and read the docs. The issue was immediately apparent: GPT inconsistently mixed up getopt and getops. sometimes the code was correct, sometimes it mixed up the syntaxes each take into a nonsensical blend.

If it can correct itself when promoted to do so, there should just be a loop made that automatically does that. You can't get all of GPT's abilities from a single passthrough. Humans are also not very capable without iterative reflective processing.

It’s not more fraught than stack overflow, plus it explains itself, so you can check it’s reasoning and it’s code.

Reply on news.ycombinator.com