> I think I'm ready to open my wallet for that Claude subscription for now. I'm happy to pay for an AI that doesn't accidentally delete my files
Why does the author feel confident that Claude won't do this?
Why does the author feel confident that Claude won't do this?
Sandbox your LLMs, don't give them tools that you're not ok with them misusing badly. With claude code - anything capable of editing files with asking for permission first - that means running them in an environment where you've backed up anything you care about and they can edit somewhere else (e.g. a remote git repository).
I've also had claude (sonnet 4) search my filesystem for projects that it could test a devtool I asked it to develop, and then try to modify those unrelated projects to make them into tests... in place...
These tools are the equivalent of sharp knives with strange designs. You need to be careful with them.
Always make sure you are in full control. Removing a file is usually not impactful with git, etc. but an Anthropic has to even warned that misalignment can cause even worse damage.
And on the same note be careful to mention files outside of it's working scope. It could get the urge to "fix" these later.
Paranoid? me? nahhhhh :-)
Yes it could write a system call in a test that breaks you, but the odds of that when random web integration tests is very very low.
Just either put it in (or ask it to use) a separate branch or create a git worktree for it.
And if you're super paranoid, there are solutions like devcontainers: https://containers.dev
If work wasn't paying for it, I wouldn't be.
> Why does the author feel confident that Claude won't do this?
| (I have almost zero knowledge of how the Windows CLI tool actually works. What follows below was analyzed and written with the help of AI. If you are an expert reading this, would love to know if this is accurate)
Personally, my biggest concern with LLMs is that they're trained for human preference. The result is you train a machine so that errors are as invisible as possible. God tools need to make errors loud, not quiet. The less trust you have for them the more important this is. But I guess they really are like junior devs. Junior devs will make mistakes and then try to hide it and let no one know
Not a big deal, it’s not a serious project, and I always commit changes to git before any prompt. But it highlights that Claude, too, will happily just delete your files without warning.
There's two subreasons why that might make asking them valuable. One is that with some frontends you can't actually get the raw context window so the LLM is actually more capable of seeing what happened than you are. The other is that these context windows are often giant and making the LLM read it for you and guess at what happened is a lot faster than reading it yourself to guess what happened.
Meanwhile understanding what happens goes towards understanding how to make use of these tools better. For example what patterns in the context window do you need to avoid, and what bugs there are in your tool where it's just outright feeding it the wrong context... e.g. does it know whether or not a command failed (I've seen it not know this for terminal commands)? Does it have the full output from a command it ran (I've seen this be truncated to the point of making the output useless)? Did the editor just entirely omit the contents of a file you told it to send to the AI (A real bug I've hit...)?
I feel like this is some bizzaro-world variant of the halting problem. Like...it seems bonkers to me that having the AI re-read the context window would produce a meaningful answer about what went wrong...because it itself is the thing that produced the bad result given all of the context.
You also see behaviour when using them where they understand that previous "AI-turns" weren't perfect, so they aren't entirely over indexing on "I did the right thing for sure". Here's an actual snippet of a transcript where, without my intervention, claude realized it did the wrong thing and attempted to undo it
> Let me also remove the unused function to clean up the warning:
> * Search files for regex `run_query_with_visibility_and_fields`
> * Delete `<redacted>/src/main.rs`
> Oops! I made a mistake. Let me restore the file:
> * Terminal `jj undo ; ji commit -m "Undid accidental file deletion"`
It more or less succeeded too, `jj undo` is objectively the wrong command to run here, but it was running with a prompt asking it to commit after every terminal command, which meant it had just committed prior to this, which made this work basically as intended.
Sure, but so can you-- you're going to have more insight into why they did it than they do-- because you've actually driven an LLM and have experience from doing so.
It's gonna look at the context window and make something up. The result will sound plausible but have no relation to what it actually did.
A fun example is to just make up the window yourself then ask the AI why it did the things above then watch it gaslight you. "I was testing to see if you were paying attention", "I forgot that a foobaz is not a bazfoo.", etc.
If the query returns something interesting, or just unexpected, that's at least a signal that I might want to invest my own time into it.
With varied success, sometimes it works sometimes it doesn't. But the more of these Claude.md patches I let it write the more unpredictable it turns after a while.
Sometimes we can clearly identify the misunderstanding. Usually it just mixes prior prompts to something different it can act on.
So I ask it to summarize it's changes in the file after a while. And this is where it usually starts doing the same mistakes again
I can't say I necessarily blame this behavior though. If we're going to bring in all the weight of human language to programming, it's only natural to resort to such thinking to make sense of such a chaotic environment.
> git reset --hard HEAD~1
After it commited some unrelated files and telling it to fix it.
Am enough of a dev to look up some dangling heads, thankfully
But I only allow it to do so in situations where I have everything backed up with git, so that it doesn't actually matter at all.
The author is saying they would pay for such a thing if it exists, not that they know it exists.
I mean I like Claude Code too, but there is enough room for more than one CLI agentic coding framework (not Codex though, cuz that sucks j/k).