Makes me think of the demo from some time ago where someone got a ~29M parameter model running on an esp32. I wonder what kind of throughput this could get if a handful of esp32s were strung together...

Edit: I have a pile of d1 minis, but not much time.