A Deep Attention-Based Reinforcement Learning Algorithm for Model-Based Control | Hacker News Reader

A Deep Attention-Based Reinforcement Learning Algorithm for Model-Based Control

arxiv.org

38 pointsctoth4 comments

If you look at the old Singularity Conference presentation of Demis Hassabis (founder of Deep Mind), his main argument for using attention, reinforcement learning and deep convolutional networks (and other known systems that were on his slides) for reaching AGI is to use only techniques that the brain uses to be able to decrease the search space for reaching AGI.

I believe his thinking was already proven, and going for emulating and integrating the known algorithms that the brain uses is the fastest way to reach AGI.

https://youtu.be/Qgd3OK5DZWI

> I believe his thinking was already proven, and going for emulating and integrating the known algorithms that the brain uses is the fastest way to reach AGI.

That's a stretch considering AGI has not yet been created by DeepMind or anyone else. Notable is that DeepMind's most prominent successes have relied heavily on MCTS, a classical planning method that doesn't have much relation to neuroscience without a lot of caveats. Their accomplishments on Atari lean a lot more on efficient computing than biological plausibility.

I think the strategy they're actually following (and I believe they've said this more recently) is to use what works and to look to neuroscience when other methods fail. This feels more solid than looking to the brain first to narrow the search space, which is the approach Numenta has taken, and does not scale as easily.

You're right with using MCTS, but ,,Their accomplishments on Atari lean a lot more on efficient computing than biological plausibility'' is a strange thing to write.

Efficient computing is of course needed for AGI, I think that was never a question. The question is what algorithms to use on the computers, and also what computing architectures should be created for those algorithms.

Those Atari simulations were the first ones putting together reinforcement learning and deep convolutional networks AFAIK, and yes, tree search was needed (which human brains are consciously doing, but extremely bad at compared to computers).

Just looking at what works is not enough. There was a strong reason why DeepMind didn't start with modelling language or logical reasoning, like many other people, and the background was based in biology (animal behaviour).

My point is that DQN is pretty far removed from the biological equivalent. It's impressive and useful but the main reason it succeeded was not because of some deep insight from neuroscience but because it scaled well (or at least better than alternatives at the time).

EDIT: Richard Sutton (largely credited as the grandfather of RL) has written about this recently: http://incompleteideas.net/IncIdeas/BitterLesson.html

Reply on news.ycombinator.com