Comment by onion2k | Hacker News Reader

Comment by onion2k | Hacker News Reader

Parent Full thread

One of the first things I tell the junior/mid-level developers I mentor is "You can't debug something just by reading the code." We all have a mental model of how our code works, and it's usually a bit wrong. Bugs are the real world manifestations of those mistakes. When you read the code it's all filtered through your model, and that makes you blind to seeing why something unexpected happened. In order to debug something you have to be able to put the system in the state where the bug happens to see why it occurred.

LLMs generally only debug systems by reading the code with whatever information you give them in a prompt. The image in the article is meta-prompt - the prompt is whatever comes from the vision model the AI happens to use to 'understand' the red circle annotation. That won't work. To successfully debug what's going on it will need much better state information. Has the 'shelf' been explained to is? Is the contrast and lack of shadows in the image messing up the vision model? Why isn't the 'lid' in the image? And so on.

LLMs are clever but they're not magical. Treat them like a naive junior dev. Give them enough data about the state of something to understand it properly.

> You can't debug something just by reading the code.

You are doing those junior engineers a disservice with this message.

I'm usually trying to go in the other direction with them - you don't need to be able to reproduce it yourself or see the log files to narrow down or find the cause of an issue.

Logs/etc are just additional tool that make it easier.

Lost track of how many juniors I've seen throw their hands up and say they can't progress because they don't have logs/clean repro

> Lost track of how many juniors I've seen throw their hands up and say they can't progress because they don't have logs/clean repro

Not the parent, but I think their approach is the correct one even though their line that you quote "You can't debug something..." should be everything instead of something.

The issue with your juniors is them throwing their hands in the air without actually reading the code when they hit a wall, and not them trying to reproduce and inspect the bug first.

What's the issue with telling them, "Obviously if you can't reproduce it and inspect it with dev tools, read the code."?

I've won bets off devs who blamed me cause they said they couldn't reproduce the bug consistently when they tried but it would happen randomly in production. Mind you, the bet wasn't even about who was at fault. The bet was that I could reproduce it if I actually tried.

You are complaining about juniors in a role that's known for the motto "it works on my machine" of course they're going to whine the moment it takes some effort to reproduce something.

At the end of the day, how are you going to claim you fixed an issue if you can't reproduce it to begin with? The senior has to now read the code more carefully cause there is no proof the junior actually fixed anything.

I'm actually not complaining. I coach them through it, but the primary help I give is showing them how to walk through the code, and break the dependency on logs/repro scenarios.[1]

In addition, if you find the issue in the code -- unless it's a really unusual race condition -- you have what you need to construct a reproduction scenario. That's also part of the coaching.

[1] At least, I did until the place I work switched everyone to vibe coding. Now I give terse responses to their LLM-generated PRs -- which usually misses the forest for the trees -- since I know that they're just going to feed my words back into the machine.

I really disagree with this approach to debugging

It's a good approach if every run of your code is very slow or very expensive, but if you can iterate quickly by running the code it's way more effective to try to reproduce and narrow it down that way

I think you've been doing your juniors a disservice

Logs are great. Interactive debugging is great. Both of these can also help build an understanding of the code and systems involved. But they're just debugging tools, and they should rarely be /required/ to find a bug.

I am teaching the more fundamental skill: how to (learn to) think through the problems and the systems in a way that lets log files be a useful tool instead of the only option.

Hmmm, I don't think that's necessarily true. Often times once I have witnessed a bug, I have found it just by reading through the code with the behaviour of the bug in mind.

For LLMs, this is likely to be disproportionately effective as well: especially because they don't really build up a persistent view of the codebase, they're generally re-reading it each session, and they tend to be surprisingly good at predicting the behaviour of code.

(That said, knowing where and how to gather more evidence to make things clearer is a pretty core skill in troubleshooting, so it's generally good advice anyhow)

Also Claude Code is very good at writing small scripts/on-off test cases to confirm bugs, so I wouldn't even say the initial premise is correct.

That's AI getting an example of replicating the state that shows the bug which is exactly what I'm talking about. It does that far more than humans do, and it's ace. That's how you should be debugging a system - replicate the issue, understand why it breaks in that given state, and then make a code change to fix it.

Sometimes you can do that mentally and fix the code. Often your fix will be right especially in a relatively simple part of the code. However, equally often you'll fix a different problem (or something that wasn't a problem at all), and the original bug will remain but you'll believe you corrected the issue. This is why you should always replicate a bug to understand it, and why you should always add a test whenever you fix a bug to prove you actually fixed it as well as preventing future regressions.

> "You can't debug something just by reading the code."

This is exactly how I debug things. When I first started working I was forbidden from using the debugger “you won’t have a debugger in the field”. Being able to read the code, spot the bug and tie it to the symptoms; that’s a skill. A damned important skill.

100%. Debuggers are nice when you can use them, trace logs are nice when you have them, clean repros are nice.

But lots of bugs can and should be found and fixed by inspection of the code, inspired by a reasonable observation.

Depending on your process needs, maybe write a test to confirm the fix.

I call this a tautological mental model. You can read the code over and over again, but your second reading will be mostly an echo of the mental model you built up in your first.

> LLMs generally only debug systems by reading the code with whatever information you give them in a prompt.

Oh, I wish. "This call doesn't return, how strange! Let me fire off a bunch of small test script one after another, with a timeout of 10 minutes each, to narrow down where the error might be!" In terms of tokens, it's maybe even efficient. In terms of time it isn't!

Reply on news.ycombinator.com