Distributed Inference and Fine-Tuning of Large Language Models over the Internet
browse.arxiv.org
browse.arxiv.org
(inevitably, someone will implement this with o.g. bitcoin and we can all cry.)
Yes, and that seems good
> (inevitably, someone will implement this with o.g. bitcoin and we can all cry.)
Why should we cry?
If the person providing the GPU in the distributed inference system can be "credited" on a ledger with some unit that will let them use other people's GPU to do their own inference in the future (and proportionally to what they themselves provided in the past), I see that as a way to arbitrate between present and future wants for GPU capacity.
You may have little use for your GPU 24/7, so you could let it chew numbers to fulfil other people AI needs, but tou may not want to do that if you're left with the electricity bill! Yet if in return you would be provided with some credit that lets you use other people GPU when you need it, you may be more willing to be cooperative instead of keeping your computer turned off ur using your GPU just for yourself.
There's also a mutual interest in cooperation: I wouldn't cry if I need in 1 second more than my GPU could do in 1 hour, and if this scheme existed, as it'd mean I wouldn't have to wait.
You shouldn't cry either: if you have contributed to the network for say 2 days in the past, you can "redeem" the equivalent of these computations immediately in 2 seconds, which is better than waiting or paying to use a company API.
The accounting may seem futile, but I think it ensures you can't waste more resources that what you provide: this is the proportionality between past/present/future wants for GPU (no leeching)
Can you explain me what you mean by "cry", or why you imply it would be a bad thing?
This makes GPU hosting far more competitive and ensures that i.e. participants choose the countries with the cheapest possible energy etc.
That seems good: I'd rather have GPUs next to where power is being wasted rather than have to build more nuclear power plants or water dams where there're people and wildlife.
If we can have say Arizona or even the Sahara covered with solar panels, I think it'd be better for everyone - humans, animal and plants included (there are not so many lifeforms in the desert)
It's completely unnecessary overhead: Ethereum's proof-of-stake seems to work now, or one could set up a centralized marketplace with no need for crypto whatsoever.
> One issue with that is it takes equally as long to verify a solution as it does to compute it.
Validate? If the "impossible" can be done on a group of nodes repeating it isn't hard, it just costs what it costs.
If you have a bunch of not very trust worthy nodes the challenge of repetition seems ideal. They can be made to do enough validation to earn enough trust to do their one nefarious act which resets their trustworthiness while the network keeps the fruits of their labor.
But as a consensus mechanism that doesn't work since we do have a global state we need to negotiate. Proof of work isn't about establishing trust, it's just about constraining the rate at which blocks can be added to the chain and for reconciling forks. The cost to verify the transactions in a block is minimal. Hashcash works because to solve an instance of it, m, a you have to find some n where H(m,n)<(1/difficulty), which takes a lot of time to try different values of n for a candidate payload m, but to verify it you just have to give me m and n and I only have to check the hash once. And the difficulty value is a dynamically adjusting parameter that keeps the network running at a consistent rate, indirectly being a spam prevention. It's not a mechanism to establish trust in (the blocks of) particular nodes.
NN inference doesn't work that way. To check that you ran an inference properly I have to do the inference myself too. (Ignoring advanced techniques that are more application-specific.). If we wanted to use it to establish trust, then a new peer that joins the network would have to redo all of the work that every peer they interact with has ever done, to determine the level of trust in it. Or if you wanted to build a consensus protocol with that trust primitive then you'd have to redo all the work everyone that's ever proposed a block has done. And that still doesn't do anything about ensuring the inference that these block producers are doing is actually useful to somebody.
Fwiw, I'm not trying to defend PoW here. I'm just trying to articulate why it's proved very difficult to find an alternative to hashcash that has the same set of properties it does, despite 10+ years of effort.
It’s an obvious move as open source models get bigger, really happy to see this out in the world - especially with a HF author attached.