Hacker News Reader

Comment by cuuupid | Hacker News Reader

Surprised no one has commented this but the latency requires the model to be trained in tiny fragments on each device which is currently a field of research that is being explored. As it stands now basically all of a model needs to be loaded into memory.

There’s a whole field here and people exploring this problem, colloquially solving this would enable Federated Learning and whoever figures this out will far eclipse OpenAI (if it’s ever solved).

View on HN

To piggy back this comment, Federated Learning actually has a lot of other uses beyond just crowdsourcing. It has benefits to data privacy also. The medical industry has struggled sharing their data with other institutions without compromising their information. They're using it to decentralize the model making process.

Intel is doing some work with Penn on the subject now, if people want to read further: https://www.intel.com/content/www/us/en/newsroom/news/intel-...

AFAIK Federated Learning is not a magic solution. This post explains how you can exploit FL if there's some untrusted server (http://www.cleverhans.io/2022/04/17/fl-privacy.html)

Well, that explains the lack of a BOINC client for ML tasks at the moment.

Where can I read more about this field of research?

https://arxiv.org/ has a ton of papers on it.

Do you have some good search terms to get started down the rabbit hole?

I too would like to go down this rabbit hole. I am going to poke around using the terms “distributed learning” and “federated learning” (They’re different areas, but somewhat related as far as I understand).

This one is a few years old, but seems interesting: https://arxiv.org/abs/1802.05799v2

Probably the biggest recent result: https://arxiv.org/abs/2209.04836 (author thread: https://twitter.com/SamuelAinsworth/status/15697194946455265...)

and more to OP's incentive structure: https://docs.bittensor.com/

Latter two intend to beat latency with Mixture-of-Expert models (MoEs). If the results of the former hold, it shows that with a simple algorithmic transformation you can merge two independently trained models in weight-space and have performance functionally equivalent to a model trained monolithically.

A broad covering term is MLOps (like DevOps).

https://en.wikipedia.org/wiki/MLOps

Armed with that term, we get (haven’t read):

Machine Learning Operations (MLOps): Overview, Definition, and Architecture

https://arxiv.org/abs/2205.02302

[+ps]

Better resource: https://ml-ops.org/

Based on my (limited) exposure to date, there is tremendous opportunity for software engineers and architects to make impact in ML systems. There is a pronounced lack of seasoned engineering talent (outside of big players like DataBricks, et al) and this knowledge gap sits behind an experience curve that mere IQ can’t jump over. Our experience as software architects and engineers is very valuable.

Know this and recognize the value you will bring to the table.

This initiative is getting some pretty good practical results https://arxiv.org/abs/2209.01188, taking a look at their citations should lead you to other examples in the field

bitorrent for GPUs?

isn't this already implemented in iPhones for their autocomplete suggestions?

Reply on news.ycombinator.com