[flagged]
Based on any reasonable mechanistic interpretability understanding of this model, what's preventing a circuit/feature with polysemanticity from representing a specific error in your code?
---
Do you actually understand ML? Or are you just parroting things you don't quite understand?
Quick question, do you know what "Mechanistic Interpretability Researcher" means? Because that would be a fairly bold statement if you were aware of that. Try skimming through this first: https://www.alignmentforum.org/posts/NfFST5Mio7BCAQHPA/an-ex...
> On the macro level, everyone can see simple logical flaws.
Your argument applies to humans as well. Or are you saying humans can't possibly understand bugs in code because they make simple logical flaws as well? Does that mean the existence of the Monty Hall Problem shows that humans cannot actually do math or logical reasoning?
> Your argument applies to humans as well
Yeah, I'm talking about obvious and trivial errors that reveal lack of representation of the code. But your question did make me think, cheers.
The mere existence of a research field is not proof of anything except "some people are interested in this". Its certainly doesn't imply that anyone truly understands how LLMs process information, "think", or "reason".
As with all research, people have questions, ideas, theories and some of them will be right but most of them are bound to be wrong.
Notice I didn't use vague terms like "think" or "reason" and instead used specific terms like "feature/circuit internal representation". You're trying to make a false equivalence of "the hard problem of gravity/reasoning/etc is not solved ... so therefore nobody understands anything" and that's obviously a false leap of logic if you've talked to any physicist or ML researcher or whatever.
That type of response is more typical GED holder who wants to feel intellectually superior so they pull out a "well you don't know anything either" to a scientist.
I'm not impressed. I've seen this before, from "biology is actually fake" or "the covid vaccine is fake, the FDA is using an 'emergency authorization' which means it's made up", or plenty of other examples. That's not a substantive objection, that's a thought-terminating cliche which is designed to dismiss any merits in the moment.
Imagine if someone in 1945 said "nuclear bombs cannot be real, even if the USA just dropped a nuke on Hiroshima, because it's just theory and it hasn't been peer reviewed yet. The Manhattan project is burning a lot of money". That would be hilarious. And yet if someone identifies an actual neuron or feature in a ML model that activates upon recognition of a software bug- WHICH IS LITERALLY WHAT YOU WOULD EXPECT IF A MODEL HAS AN INTERNAL REPRESENTATION OF SUCH A THING- it gets dismissed. If such an obvious signal is dismissed, what is even the end goal?
---
Way to go in showing you want a discussion, good job.
Now go read https://transformer-circuits.pub/2024/scaling-monosemanticit... or https://arxiv.org/abs/2506.19382 to see why that text is outdated. Or read any paper in the entire field of mechanistic interpretability (from the past year or two), really.
Hint: the first paper is titled "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet" and you can ctrl-f for "We find three different safety-relevant code features: an unsafe code feature 1M/570621 which activates on security vulnerabilities, a code error feature 1M/1013764 which activates on bugs and exceptions"
Who said I want a discussion? I want ignorant people to STOP talking, instead of talking as if they knew everything.
And apparently you think peer reviewed papers presented at NeurIPS or other conferences are considered pseudoscience. (For the people not versed in ML, NeurIPS is where the 2017 paper "Attention is All You Need" that started the modern ML revolution was presented)
https://neurips.cc/virtual/2023/poster/72666
https://jmlr.org/beta/papers/v26/23-0058.html