Powerful AI Can Now Be Trained on a Single Computer

Lots of people are focusing on this being done on a particularly powerful workstation, but the computer described seems to have power at a similar order of magnitude to the many servers which would be clustered together in a more traditional large ML computation. Either those industrial research departments could massively cut costs/increase output by just “magically keeping things in ram,” or these researchers have actually found a way to reduce the computational power that is necessary.

I find the efforts of modern academics to do ML research on relatively underpowered hardware by being more clever about it to be reminiscent of soviet researchers who, lacking anything like the access to computation of their American counterparts, were forced to be much more thorough and clever in their analysis of problems in the hope of making them tractable.

The machine used is a 36-core + single gpu. So not quite a home computer yet but this is some serious progress!

Paper: https://arxiv.org/abs/2006.11751

Source: https://github.com/alex-petrenko/sample-factory

Kind of a weird headline, the vision-based RL this article dubs 'powerful AI' could already easily be trained on a single (pretty expensive) computer. They say as much in terms of the speed ups they provide:

"Using a single machine equipped with a 36-core CPU and one GPU, the researchers were able to process roughly 140,000 frames per second while training on Atari videogames and Doom, or double the next best approach. On the 3D training environment DeepMind Lab, they clocked 40,000 frames per second—about 15 percent better than second place."

So, not massive this-is-now-doable speedup.

May be my AI tasks are to simplistic, but I never had problem training AI on single machine and as many pointed out there always cloud services, otherwise I find it an impressive work, it takes some courage to say well there so many huge companies with there's frameworks, but we can outperform them all in specific problems.

Couple things that stood out from github page:

Currently we only support homogenous multi-agent envs (same observation/action space for all agents, same episode duration).

For simplicity we actually treat all environments as multi-agent environments with 1 agent.

My speculation is that this why they gained such dramatic performance improvement. (but I might be very wrong)

Is SLIDE being used anywhere, or were flaws discovered? It was supposed to massively accelerate training on CPUs.

https://www.hpcwire.com/off-the-wire/rice-researchers-algori...

I'm surprised that this is IEEE worthy and not just common sense. Of course there'll be huge speedups if, and only if, your dataset fits into main RAM and your model fits into the GPU RAM.

But for most state of the art models (think gpt with billions of parameters) that is far from being the case.

> His group took advantage of working on a single machine by simply cramming all the data to shared memory where all processes can access it instantaneously.

If you can get all your data into RAM on a single computer, you can have a huge speedup, even over a cluster that has in aggregate more resources.

Frank McSherry has some more about this, though not directly about ML training.

http://www.frankmcsherry.org/graph/scalability/cost/2015/01/...

It's very nice optimization and systems engineering work which shows what a computer can do if you use it properly.

One correction: it's not Doom, but ViZDoom, a simplified version designed for DRL.

Author of the paper here. Ask me anything!

I wish machine learning would have become mainstream on a language with more competent multithreading capabilities than python. During my machine learning course I knew I could squeeze more performance out of my code by parallelising data preprocessing and training (pytorch), but python cannot do proper multithreading. The multiprocessing module requires you to move data between processes, which is slow.

Out of the question, where should an indie .IO game developer start to build their AI players in their game?

For the record, I am a solo developer in progress of developing a to-be-online browser game. I must make intelligent bots and keep players busy until the time it has a lot of online players.

I had a look at Reinforcement Learning but I am not sure people are really using for this use case.

So this basically boils down to keeping your training data in memory? Is there something else I missed?

the most misleading article ever. Twice as fast as best known method ain't gonna cut it for training AlphaZero Go for example.

But still 140,000 images per second! That's nuts. Even 70,000 is nuts.

I can only play 1000 2048 games in one second. Damn! I am slow

There's a lot of research that you can get away with on smaller hardware. Much of my own I can do with a single 2080Ti. BUT at times it is extremely frustrating and I don't have the memory to tackle some problems with just that hardware.

If you want to see major improvements in the academic and home side of AI then NVIDIA and AMD need to bring more memory to consumer hardware. But there isn't much incentive because gamers don't need nearly the memory that researchers do.

I'm reminded of the following episode of Pinky and the Brain:

  Brain plans to use a "growing ray" (originally a "shrinking ray") to grow
  Pinky into super-size while dressed up as Gollyzilla, while Brain would turn
  himself gigantic and stop him, using the name Brainodo, in exchange for
  world domination. However, the real Gollyzilla emerges from the ocean and
  starts to rampage through the city, making Brain think that the dinosaur is
  Pinky. The episode ends with the ray going out of control and making
  everything on Earth grow, including the Earth itself, to the point that
  Pinky, the Brain, and even Gollyzilla are mouse-sized by comparison again.

https://en.wikipedia.org/wiki/List_of_Pinky_and_the_Brain_ep...

Being able to train deep RL models on commodity hardware is only an advantage if there isn't anyone that can train on more powerful hardware (or if somehow training on more powerful hardware fails to improve performance with respect to your model). Otherwise, you're still just a little mouse and they have all the compute.

Their results on lasertag are very poor. Looks like their network was incapable of solving it. Which probably means for apple-to-apples comparison to IMPALA they need a better network, which might require them to forego their throughput advantage.

Interesting article, i will check github at weekend, but assuming atari resolution 40x192 single channel and float size 4 byte and 140000 image you got 4300800000 bytes roughly 4gb data. I wonder if they talk about complete training time

A mid-range gaming laptop can do much more and better in 2020 than in 2018, thanks to novel available batch functions, a number of preliminary hacks and much hw-friendlier frameworks.

Is this an early warning indicator that the cloud-to-edge pendulum has begun swinging back in the direction of personal computing?

This is incredible, but won’t these techniques just make the gap wider as big tech firms create even larger models?

Depends on the size of a single computer. Cloud services have something 200 core, 4TB in a single machine too

It always could be trained on a single computer. It was just a matter of physical size versus time.

How much RAM did their test workstation have? I can't seem to spot it.

How many RPi 4’s would you need?

with a titan and a threadriper for instance what would be the results?

Apple’s in-house SOC is the future.

I suspect the cloud has a decade, maybe less, of hype to grift on.

Huge data sets on a personal computer and opt-in data sharing with business and healthcare, etc will be the new norm.

Further out, software as we know it will cease to exist as entirely custom chips per application are the norm. IN TIME.

New hardware wars to capture consumer attention incoming.