Hacker News Reader

Comment by arcturus17 | Hacker News Reader

I'm afraid I don't understand your argument. The programmer is still ultimately responsible for the correctness, and quality, of the program. These are just very advanced IDE hints.

How is this approach "wrong"?

View on HN

Sadly (?) there are many not so good programmers (e.g. because they are new to it).

Finding a bug in something which seems to be correct can be harder then writing the correct code yourself.

Especially if you might not (yet) fully understand what you are doing.

So Copilot is only a good idea if you are a experienced programmer with the discipline to put any auto generate part through a proper (ad-hoc) code review.

At the same time it looks especially appealing to someone just starting to learn coding...

Right but that all applies to any situation where you use someone else's code. Reading Stackoverflow answers. Blog posts. Even other code on Github.

Then it can be picked up in a code review.

Spotting bugs in code review is already hard when the code you're reviewing has been written by a human. In this case, at least you can rely on your knowledge of the co-worker who wrote it - over time, you learn the kind of experience they have, the kind of mistakes they make.

Spotting bugs in code that has been generated by a GPT-3 derivative, with all the subtle mistakes that implies, is going to be even harder.

In the end we need to tag each piece of code written by copilot. Doing a code review on a piece of code written by a human and on another one generated by copilot is going to be a very different experience. I would be way more wary of a PR containing copilot generated code. Turns out copilot will be a productivity killer.

> Spotting bugs in code that has been generated by a GPT-3 derivative, with all the subtle mistakes that implies, is going to be even harder.

I'm kind of skeptical! I think your claim is reasonable tho so maybe I'm more skeptical of your confidence?

I'd love to read a follow-up from you after you tried using Copilot for an extended period; even (or maybe especially) if it's as bad, or worse, than you expect!

But good code reviews are much harder at scale than building good code in the first place. The reason we write code _and_ do code reviews is because doing both is better than doing either.

But copilot isn't even equivalent to a code review: code review is not only checking for correctness. It's also asking questions and helping the author walk through their solution by having them reconsider it. Copilot doesn't ask questions, nor can it answer them or provide a rationale.

If the programmer dosent have the requisite knowledge to verify the code it's hard to know if the generated results are correct. Compare this to copy pasting solutions from stackoverflow. Atleast there you get a pretty good idea of the pros and cons of a solution. With copilot it's all upto the programmers understanding of the generated code. Of that programmer propmts copilot on a domain they don't know much about it could lead to a lot of subtle bugs being introduced

> Compare this to copy pasting solutions from stackoverflow. Atleast there you get a pretty good idea of the pros and cons of a solution.

I don't think I understand why this should be true.

Because on SO you:

- get multiple answers

- comments on the answers

- up/down votes

- explanations along side of the answer

and I still would argue you should never copy from stack overflow!! Instead understand why the answer is correct and then write your code based on that understanding, even if it produces the exact same code in the end.

Also, to get answers on SO you have taken the time to elaborate a question. Building up the right question is a significant part in the process of understanding.

The vast majority of visits to SO are by people who don't even have an account. They're not asking questions themselves.

They may not be authors of the question on SO, but to find that question you need to search for it, and it is not uncommon that doing so takes more than one research/thread parsing. In the end, finding the relevant SO answer is not unlike asking a good question in the first place.

If the question is more complex than "how do you do a parameterized query with ActiveRecord?" it's unlikely autocomplete is doing it for you either.

Stack Overflow answers only containing a code block are voted down.

In theory, I guess, but the type of person who just blindly commits code they didn't understand isn't going to read the explanation and isn't going to catch security issues with SO answers either.

The fact that some (bad) programmers already blindly copy SO code does not detract from the original argument that Copilot is dangerous because it effectively copy-pastes SO code blindly.

It presents a suggestion, which you are free to reject. What's "blind" about it?

The fact that it doesn't come with context. I just fail to see the usefulness of the suggestion, if the quality can't be trusted. Either:

a) I'm familiar enough with the code/topic that I'm able to judge whether the suggestion is good, or

b) I'm not able to judge, and need to consult other sources.

In case a, I could have just written the code myself. In case b, I still need to read up, so the potential gain is that Copilot gave me a hint as to what I should search for. But even that hint can't be trusted - there was another comment where Copilot suggested to use MySQLi in PHP, which has been the wrong choice for a long time.

So if the suggestions need scrutinization, the target group I see is people who need to write a lot of code in a familiar area, and are limited by the speed at which they type. This is arguably a very small set of people, and is unlikely to be the main user group of this tool.

When I visit SO, I generally read all the answers and can pretty immediately judge "I think this approach is good" or "I don't think this approach is good" just by reading the code. I can't see why these suggestions would be any different. And in the perhaps more common case where I already know exactly what I want to do it can still save time to have the plug-in autocomplete it (I make heavy use of tab completion today after all).

The mysqli extension in PHP is fine to use and has advantages over PDO if you only ever plan to interface with MySQL. Now, if it was recommending the mysql extension, that would be bad as that's been deprecated and replaced by mysqli for a very long time.

Yeah, this point I really don't get.

Your job as a programmer is to ensure the correctness of the code you write, Copilot isn't really changing that.

If you went to school in CS, think about the worst programmer in your graduating class. You don't think they are going to mindlessly accept the autocomplete as long as it compiles? I can imagine this will lead to a lot of bad code making it into production.

So? Systems should be in place to review code.

If you don't have these systems in place, you're getting what you deserve. Hiring subpar talent and not having processes in place isn't Copilot's fault.

This is almost like blaming the car for an accident. If you have a shitty driver and the traffic lights aren't working, it's not your Corolla's fault.

You’d have to believe auto completion is evil too I guess

It is going to have the same vigilance problem as partial self-driving cars. When the machine feels like it's handling most everything for you, you let down your guard.

It shouldn't be that way, but it's how people work, so we should expect it to happen. (Psychologically, if an action is very rarely rewarded, people are less likely to do it.) Even if you want to be vigilant, it will be a bit of an uphill battle to make yourself do it.

Also, checking the auto-generated code takes time. You, as an individual contributor programmer, may believe that the time is absolutely worth it, but you will need management to support that. This technology creates additional opportunities for managers to say, "Don't worry about that. Ship it." Which is another thing that really shouldn't happen but in reality does.

Reply on news.ycombinator.com