> Instead, Olshausen and others argue that information in the brain is represented by the activity of numerous neurons. So the perception of a purple Volkswagen is not encoded as a single neuron’s actions, but as those of thousands of neurons. The same set of neurons, firing differently, could represent an entirely different concept (a pink Cadillac, perhaps).

Unless they’re talking about embeddings, this is how 99% of people think about representation.

And even if they are talking about embeddings, embeddings are, quit standardly, higher dimensional floating point vectors. Even the original Transformer paper used IIRC 512-dim vectors.

This article seems like yet another signal for the death of quantamagazine.