Hacker News Reader

Comment by Lerc | Hacker News Reader

One of the things I have been wanting to try for a while now is something like this with a layer per MCU. I have some crazy ideas with RP2350's talking to each other with dedicated lines fed by PIO going through a combination of interpolators and dual multiply instructions.

PSRAM, Flash, and even SD cards may not have the best bandwidth individually, but they can reach quite impressive rates when you have a shitton of them running all at the same time.

The large scale dedicated hardware systems will still have the edge for performance per watt, but the low entry level and slow incline does make these things quite appealing.

View on HN

Run the numbers before you waste time here. I doubt this will work.

PIO to PIO between two rp2350s should be able to transfer as many bits per clock as you can spare pins for.

They have a single cycle double multiply per core, and the interpolators give you a heap of ability

The PIO can be awkward, but you can run a bunch of them at once. Going from MCU to MCU you don't even need to involve the CPU cores, PIO to PIO Comms via pins

You are obviously not going to get big TOPS from it because a Trillion is a ridiculous amount anyway. But never underestimate the power of controlling the whole pipeline.

Ultimately none of the other things I'm doing with MCUs are practical, why would this to be any different.

Running some quick numbers shows you should be able to get >1Gbps. But I seriously doubt you could get those speeds in reality. You would need to get them perfectly in sync, which would likely take a dedicated board and some great knowledge of the oscillator.

As someone who has done a reasonable amount with PIO, I do not think this is possible. However, that should not stop you. If you get it to work, please ping me.

Wouldn't a layer per MCU be heavily bandwidth constrained?

Depending on where you slice the model up, it can be not a whole lot of data. For instance each transformer block outputs a single vector in an embedding space.

I can see that being cheaper to bitbang with PIO than to actually compute.

There's certainly some latency stack up, but throughput should be remarkably good.

Reply on news.ycombinator.com