Hacker News Reader

Comment by Habgdnv | Hacker News Reader

Ok, I am getting mad now. I don't understand something here. Should we open like 31337 different CVEs about every possible LLM on the market and tell them that we are super-ultra-security-researchers and we're shocked when we found out that <model name> will execute commands that it is given access to, based on the input text that is feed into the model? Why people keep doing these things? Ok, they have free time to do it and like to waste other's people time. Why is this article even on HN? How is this article in the front page? "Shocking news - LLMs will read code comments and act on them as if they were instructions".

View on HN

This isn't a bug in the LLMs. It's a bug in the software that uses those LLMs.

An LLM on its own can't execute code. An LLM harness like Antigravity adds that ability, and if it does it carelessly that becomes a security vulnerability.

No matter how many prompt changes you make it won't be possible to fix this.

Right; so the point is to be more careful about the other side of the "agent" equation.

So, what's your conclusion from that bit of wisdom?

Isn't the problem here that third parties can use it as an attack vector?

The problem is a bit wider than that. One can frame it as "google gemini is vulterable" or "google's new VS code clone is vulnerable". The bigger picture is that the model predicts tokens (words) based on all the text it have. In a big codebase it becomes exponentially easier to mess the model's mind. At some point it will become confused what is his job. What is part of the "system prompt" and "code comments in the codebase" becomes blurry. Even the models with huge context windows get confused because they do not understand the difference between your instructions and "injected instructions" in a hidden text in the readme or in code comments. They see tokens and given enough malicious and cleverly injected tokens the model may and often will do stupid things. (The word "stupid" means unexpected by you)

People are giving LLMs access to tools. LLMs will use them. No matter if it's Antigravity, Aider, Cursor, some MCP.

I'm not sure what your argument is here. We shouldn't be making a fuss about all these prompt injection attacks because they're just inevitable so don't worry about it? Or we should stop being surprised that this happens because it happens all the time?

Either way I would be extremely concerned about these use cases in any circumstance where the program is vulnerable and rapid, automatic or semi-automatic updates aren't available. My Ubuntu installation prompts me every day to install new updates, but if I want to update e.g. Kiro or Cursor or something it's a manual process - I have to see the pop-up, decide I want to update, go to the download page, etc.

These tools are creating huge security concerns for anyone who uses them, pushing people to use them, and not providing a low-friction way for users to ensure they're running the latest versions. In an industry where the next prompt injection exploit is just a day or two away, rapid iteration would be key if rapid deployment were possible.

> I'm not sure what your argument is here. We shouldn't be making a fuss about all these prompt injection attacks because they're just inevitable so don't worry about it? Or we should stop being surprised that this happens because it happens all the time?

The argument is: we need to be careful about how LLMs are integrated with tools and about what capabilities are extended to "agents". Much more careful than what we currently see.

Reply on news.ycombinator.com