Not All Language Model Features Are Linear

I feel like un-supurvised methods like Anthropic's SAEs can be argued to find things we're not looking for (their most recent work is from a couple days ago: https://transformer-circuits.pub/2024/scaling-monosemanticit...). And we can get some sense of how "much" of the model they're recovering by looking at their downstream reconstruction loss.

It's a super interesting direction! That's one of the long term goals of interp research: deconstruct model behavior into circuits of features, and then turn those circuits into code (that we can maybe even formally verify!).