GitHub Copilot makes insecure code even less secure, Snyk says | Hacker News Reader

GitHub Copilot makes insecure code even less secure, Snyk says

infoworld.com

15 pointsluthfur6 comments

My team worked on this research at Snyk. If you think about it, it's pretty obvious behavior:

- Generative AI uses the context you provide to help generate additional tokens

- If the context you provide is bad (low-quality code, riddled with security issues), you'll similarly get low-quality code generated

- If the context you provide is good (high-quality code), you'll get better-quality code out

The thing we wanted to highlight with this research is that security is meaningfully impacted when you're generating code, particularly with low-quality codebases.

Is this a surprise, though? I think any sane developer would expect that a generative AI code helper making suggestions based on existing patterns in your code would do exactly this. It's not "thinking" about security, performance, or other non-functionals.

It's not a surprise to me, but it probably is to a lot of people who comment that they either couldn't code or wanted the AI to boiler plate something.

If you didn't write it, then it's going to be low quality, insecure code by default.

It's worth nothing that over the holidays, I decided to pick up Go for a personal project and tried GitHub CoPilot. I was tremendously impressed. At the beginning, it was mostly useful for helping me with syntax as I could ask it to write a for loop, access member variables, etc.

Over time, it would start to suggest larger and larger blocks of code based on what I was writing. It got to the point where it would auto-suggest entire functions based on my code comments. e.g. I'd type: "//Update a user" and it would then suggest complete code for doing just that, while managing to match the style of previous methods, use the correct ORM objects, etc. In general, it seemed to be 90-95% "correct".

However, it did like to remind me that it was just really good at putting words together. For the Update user, it suggested a function that exactly matched how I'd write it, except for this:

//Persist changes to Show object to the database

user.Save()

Just a nice reminder that the GenAI is not "thinking" about what it's doing.

Whenever I read articles like this (AI assistants having a negative impact on resulting work), I wonder about how much it affects "experienced" AI users. I've been interested in AI since the Cleverbot days, and have extensively used GitHub Copilot and ChatGPT since they came out. When I ask ChatGPT something that has an objective answer, but one I can't easily verify from my own knowledge or low stakes experiment (e.g. does this fix my syntax error?), I always make sure to not "ingest" it into my knowledge or product before finding one or more external corroborating sources. This doesn't make ChatGPT significantly less useful to me, from my experience, verifying an answer is typically much easier than researching the question from the ground up by conventional means (Google, GitHub Code Search). Similarly, when using GitHub Copilot, I am acutely aware that I need to critically evaluate the suggested code myself, and if there is something I am unsure about, it's again off to Google or Code Search.

Personally, the most risky AI stuff I do is if I am completely stuck on something, I might accept AI suggestions without much thought just to see if it can resolve whatever issue I am running into. But in my mind, those parts of the code are always "dirty" until I thoroughly review them; in the vast majority of cases, I end up refactoring those parts myself. If I am asking AI to improve a text I wrote, I rarely just take it as-is, I typically open both versions next to each other and apply parts I like to my original text.

In my opinion, stuff created by AI is inherently "unfinished". I cringe whenever people have AI do something and just roll with it (writing an essay, code, graphic design, etc.). AI is excellent for going most of the way, but in most cases, there need to be review and finishing touches by a human, at least for now.

So the main question is: if I rewrite it in rust, but use Copilot, should I even bother?

Reply on news.ycombinator.com