This is a really cool dive into the interpretability of intermediate-level neural network activations. I really like the way that they've identified a core debate in the ML conversation, and created a relatively simple test-space to better understand it and (hopefully) move the discussion forward.
Very nice. Was a bit confused by the jump to splines and fitting splines to butterflies, more explanation would be appreciated
They write about neurons having “meaning” without a strict definition. What’s the quantitative meaning of meaning?
They fulfill a task, if you want the short answer.
No idea how to define 'task' here
This isn't immediately clear on whether it's discussing biological neurons or artificial neurons. I suppose it's the latter.
All the authors are part of OpenAI, and the very first sentence talks about computer vision and a parallel with real neurons. So "This isn't immediately clear" should be read as "I only read the title and didn't even try to understand what the article is about".