Hacker News Reader

Comment by oxygen_crisis | Hacker News Reader

If you can't write the code yourself, you're not a good judge of how competent it is. If you admire any code that simply works at all when you don't know how it should be done, then your admiration isn't a measurement worth considering.

Ask an LLM how to do something you already know very well how to do properly, only then can you see its flaws through its mindless bravado.

View on HN

On the other hand, somewhat in the style of NP-Completeness, it is often easier to verify that something is correct than it is to generate it from scratch. Even if I can't cook, I can say it's a tasty meal (:

It's usually somewhat easy to verify a piece of code is not obviously wrong, what's much harder is proving that a piece of code is not subtlety wrong. When given a complete piece of code that appears to work, it can be very easy to convince yourself that you understand it well enough to know that it is correct, even when it's not. This problem isn't unique to LLMs, refer to the case of programmers copying binary search from textbooks without understanding how it works in their programming language of choice [1]. The problem is avoided (or at least minimised) by formal verification, which is where I think we should be heading with LLM code generation; this additionally avoids the problems with trying to accurately provide a specification in plain English.

[1] https://ai.googleblog.com/2006/06/extra-extra-read-all-about...

"this looks about right and has no obvious bugs" is my standard when reviewing human code, and it's my standard for machine-generated code too. no reason to formally verify GPT-4 outputs if I'm not formally verifying my coworker's either.

Well... after fairly long experience, we have discovered that your standard is mostly adequate for human generated code (as long as it's not going into a critical system). That may be based on the (empirically collected) statistics of how human-generated code fails - that if it's wrong, it usually either "looks" wrong or obviously fails.

GPT-produced code may have different failure statistics, and therefore the human heuristic may not work for GPT-produced code. It's too early to tell.

My standard changes based on which human's code I'm reviewing though.

Unless I'm feeling particularly lazy or the code isn't important.

I'm reminded of a friend who worked in radio hardware design. They'd use simulation and fuzzy/genetic algorithms to create a circuit, and then verify its performance with experiments. But they couldn't always say exactly why the circuit worked, just that it met the performance criteria.

It's an interesting divergence in software, between those who manage complexity by adding more human-understandable abstraction, and those who manage it by just verifying the results, letting the complexity fly free. All the ML stuff is definitely taking big steps down the latter path.

I don't have to have built a car to be able to judge the quality of a car to a decent approximation. And with code, surely, skill carry over is very real. Being able to generalize is literally our entire thing.

as someone who worked for a car manufacturer, you are wrong.

What people perceive as a quality car, is often a lack of understanding. Some think they are driving a fine car, but someone who pays attention notices that the door gaps are not uniform, that the plastic on the inside will start to smell bad in summer, that you will get a lot more fatigue because of wind noise and tire noise, that the sound of the door being slammed shut is very hollow instead of a nice THUNK, that the automatic gear box often eats its gears, especially when driving slow, ...

There are many details normal people don't even notice about their cars, and then they claim it is a quality car. It is not, I am sorry, and it is the perfect analogy for this ChatGPT/Bard being a code generator.

The very definition of a quality product is one that the user is happy with.

If the user doesn't notice the door gaps, then it isn't a quality issue. If the user doesn't drive the car enough to wear out the suboptimal gearbox, then it isn't a quality issue.

Yes, this means that the definition of quality depends on the needs of the user.

That’s very explicitly not the definition of “quality”.

I am not entirely certain you aren't joking. Everything you mention is felt by the layman and it is comically daft that you would assume it requires some kind of special power.

Getting these things right takes extra effort. If the customer didn't care, it would not be done.

Certainly, not every layman will be able tell you in so many words WHY it's nice and everyone cares about different things in a product (while it's the designers job to care about all of them)

But that was not really the contention now, was it.

This doesn’t ring true for me. I have ChatGPT write code for me that I could write myself. I’ve had ChatGPT even rewrite code for me to make it more legible. It’s pretty good at it, especially when it comes to more popular languages.

>> If you can't write the code yourself

> that I could write myself

I don't think your comment is very relevant, given how they specified it.

I was replying to the second part of it.

I tried asking it to write code to swap columns 1 and 3 of a CSV file, written in only x86 assembly. It refused, claiming that while it was theoretically possible, it would be stupid to do such a thing. It couldn't be persuaded...

It's not wrong...

You are agreeing with OP. You could evaluate the quality of ChatGPT because you could write the code yourself.

I'm disagreeing with the second part of OPs comment.

So much these discussions are pointless arguments over things we aren't even bothering to define in the discussion.

One person says "write code" in the context of chatGPT and they mean Common Lisp and chatGPT3.5

Someone else says "write code" and they mean React components and chatGPT4.

It is a mirror to how imprecise and intellectually lazy our own language and minds have become in the context of online discussion. Compressed thinking in order to fit in a twitter box along with years of being rewarded with attention the more muddled the thinking since the more people that disagree with the point the more responses it will generate.

It is like we have been running a giant language miscommunication RLHF model and the end result is an incredibly accurate system at creating miscommunication.

Reply on news.ycombinator.com