Cool experiment, but is this actually a practical path forward or just a dead end with a great headline? Someone convince me I'm wrong...
But sometimes you just have to let the academics cook for a few decades and then something fantastical pops out the other end. If we ever make something that is truely AGI, its architecture is probably going to look more like this SpiNNaker machine than anything we are currently using.
It doesn't have built-in storage, but that doesn't mean it can't connect to external storage, or that its memory cannot be retrieved from a front-end computer.
If you have two systems with opposite bottlenecks you can build a composite system with the bottlenecks reduced.
Yes, a different usage pattern could give you the worse of both worlds.
That's a very deep insight from queue theory and similar topics, I don't know if you wrote what you wrote with that in mind (I'm not being an ass, I just want to highlight this observation).
A system's behavior is highly dependent, to the point of it being fundamental, on how it is going to be used.
There's plenty to learn from endeavors like this, even if this particular approach isn't the one that e.g. achieves AGI.