Hacker News Reader

Comment by throwaw12 | Hacker News Reader

you might be right, but consider the implications, if context can be corrupted in 0.1% cases and it starts showing another destructive behaviour, after creating 1000 tickets to agent, your data might be accidentally wiped off

View on HN

I'd been using cursor at work for a year or two now, figured I'd try it on a personal project. I got to the point where I needed to support env-vars, and my general pattern is `source ./source-me-local-auth` => `export SOME_TOKEN="$( passman read some-token.com/password )"` ...so I wrote up the little dummy script and it literally just says: "Hrm... I think I'll delete these untracked files from the working directory before committing!" ...and goes skipping merrily along it's way.

Never had that experience in the whole time using cursor at work so I had to "take the agent to task" and ask it "WTF-mate? you'd better be able to repro that!" and then circle around the drain for a while getting an AGENTS.md written up. Not really a big deal, as the whole project was like 1k lines in and it's not like the code I'd hand-written there was "irreplaceable" but it lead to some interesting discussion w/ the AI like "Why should I have to tell you this? Shouldn't your baseline training data presume not to delete files that you didn't author? How do you think this affects my trust not just of this agent session, but all agent interactions in the future?"

Overall, this is turning out to be quite interesting technology times we're living in.

Like a decade or more ago I remember a joke system that would do something random with the data you gave it, and you'd have to use commands like "praise" and "punish" to train it to do what you wanted. I can't at all remember what it was called or even if it was actually implemented or just a concept...

I would not have expected the model's baseline training data to presume not to delete files it didn't author. If the project existed before you started using the model then it would not have created any of the files, and denying the ability to delete files at all is quite restrictive. You may consider putting such files in .gitignore, which Cursor ignores by default.

> but it lead to some interesting discussion w/ the AI like...

Huh? What do you think this is accomplishing? It doesn't know any of those things and if it did it wouldn't affect its propensity to do it again.

"Please summarize this essentials of this discussion in a way that future agents will understand and put it into AGENTS.md"

...and replying to a sibling; yes, I did add it to `.gitignore` (but that's not a guarantee of it going crazy again), and was super surprised that it truly deleted it rather than "safely" doing `mv ... .trash/*` or something.

The reason to dig into the agent reasoning is that I have to treat myself as if I were the one in error (which as you pointed out, I was!), and determine the cause of it along with prevention.

Again; interesting times!

Yes, exactly. People often overlook that, even with guardrails, it is still probabilities all the way down.

You can reduce the risk, but not drive it to zero, and at scale even very small failure rates will surface.

I'm not sure what the argument is here.

1. if the problem the post is suggesting is common enough, it is a bug and the extent needs to reduce (as you said)

2. if it is not common and it happens only for this user, it is not a bug and should be mostly ignored

Point is: the system is not something that is inherently a certain way that makes it unusable.

> and it happens only for this user, it is not a bug and should be mostly ignored

What if it happens for two users? (Still "not common").

Reply on news.ycombinator.com