Hacker News Reader

Comment by 8thcross | Hacker News Reader

whats the point of this? serious question - can someone provide usecases for this?

It is shown running on 2 or 4 raspberry pis; the point is that you can add more (ordinary, non GPU) hardware for faster inference. It's a distributed system. The sky is the limit.

ah, Thanks! but what can a distributed system like this do? is this a fun to do, for the sake of doing it project or does it have practical applications? just curious about applicability thats all.

I'm going to get downvoted for saying the B-word, but I imagine this growing up into some kind of blockchain thing where the AI has some goal and once there's consensus that some bit of data would further that goal it goes in a block on the chain (which is then referenced by humans who also have that goal and also is used to fine tune the AI for the next round of inference). Events in the real world are slow enough that gradually converging on the next move over the course of a few days would probably be fine.

The advantage over centralizing the compute is that you can just connect your node and start contributing to the cause (both by providing compute and by being its eyes and hands out there in the real world), there's no confusion over things like who is paying the cloud compute bill and nobody has invested overmuch in hardware.

got it. plan B for skynet. one baby transformer at a time.

Eh, something that's trapped in a blockchain and can only move forward when people vote to approve its next thought/block is a lot less scary to me than some centralized AI running behind closed doors and taking direction from some rich asshole who answers only to himself.

I think of it more like government 2.0.

It doesn't even scale linearly to 4 nodes. It's slower than a five year old gaming computer. There is definitely a hard limit on performance to be had from this approach.

In a distributed system, the overall performance and scalability are often constrained by the slowest component. This Distributed Llama is over Ethernet..

The raspberry pis aren't really the point, since the raspberry pi os is basically debian, this means you could do the same thing on four much more powerful but still very cheap ($250-300 a piece) x86-64 systems running debian (with 32, 64 or 128GB RAM each if you needed). Also opening up the possibility of relatively cheap pci-express 3.0 based 10 Gbps NICs and switch between them, which isn't possible with raspberry pi.

Reply on news.ycombinator.com