Hacker News Reader

Comment by jncfhnb | Hacker News Reader

Did they prompt it to consider security weaknesses?

They did not prompt at all. They used GitHub’s code search to find projects where the repo owner specified that the code was generated “by Copilot” and the authors took that at face value for all code in the project. Whether the code was actually suggested by Copilot is not at all analyzed in the paper. As such, the results are highly questionable.

That would be kind of wild. Imagine a world where whether your system was secure was just a matter of remembering to tell the AI agent "& also make it secure" before it writes your code.

(could be quite real!)

This would likely help a little bit. We've already seen LLMs improve performance on some tasks by being instructed to "think carefully" first; presumably this biases it towards parts of the training set that are higher quality.

But security ultimately requires comprehension, which is not something LLMs have.

Security 100% does not require comprehension in the philosophical sense

Not sure what the philosophical sense would be. I just mean that an awful lot of people treat security as "hack on the code until it doesn't obviously break", and that's the wrong mindset for practical security.

I’m not sure what you could possibly mean by LLMs don’t have comprehension if not speaking to philosphical debates of what comprehension means.

I think your point is otherwise right, but the correct answer is standard best practices which is the easiest thing for bots to do

I think it’s more likely that you would use a security graded bot.

It’s perfectly reasonable to not use secure code for a large number of use cases.

> also make it secure

[proceeds to simply refactor the same code]

Reply on news.ycombinator.com