Hacker News Reader

Comment by badrequest | Hacker News Reader

This feels like a distinction without a difference.

Depends entirely on how close your desires are relative to the center of the bell curve.

Great observation. When working on a scrappy project (e.g. MVP), solutions that just work might suffice (far left of bell curve).

If CoPilot bumps these crumby solutions into 'average' solutions then it's a big improvement under those circumstances.

But when making highly refined software, CoPilot's suggestions might be relatively mundane; a degradation in quality.

Even in not refined software: the other day I asked it to, in js, create a table taking (hour, power produced) data from a json response. It managed to come up with an n^2 way of doing what should have been a linear traversal through the json.

Sometimes it's great, sometimes the middle of the curve is itself bad. :)

Great point; sometimes average is not good enough. (imagine what could go wrong with a banking application or software for dangerous physical/industrial process). A simple inefficiency like the that could cause degradation in a safety system or similar.

This strengthens the case for careful assessment of when CoPilot is used vs not used, and in cases that it's used, how much effort goes into rigorously testing and scrutinizing its suggestions (as opposed to unthinkingly accepting them as one may for tasks known to have no downside risk).

Very good point, context is important.

Maybe there is an option to make Copilot prioritize security concern over other aspects when working in the context of security related apps or features.

This is how humans work, too.

Technically, this is fine tuning or prompt engineering.

It feels important to me, from the perspective of someone who doesn't want to be negatively surprised by auto-generated code. I'd expect there to be a significant difference in uncaught errors in Copilot code between people who believe Copilot knows what they want vs people who think copilot just does what it's been trained to do, mainly based on the level of scepticism people approach reviewing the code with.

I think it is a huge difference. It didn't know what you wanted. It knew what most people want. For a lot of generic things what you want is probably what most people want. Shoot, most times we ask ourselves what other people do in this situation. But in a specific case, what you want is probably not what other people want. As that is why you are writing this code, to do something different.

Some people say "she doesn't listen to me" when their Amazon Alexa fails to properly parse particular prompts.

I personally say "it" as I see it as a machine.

Reply on news.ycombinator.com