DeepMind’s work in 2016: a round-up
deepmind.com
deepmind.com
https://deepmind.com/blog/deepmind-ai-reduces-google-data-ce...
> Our partnership with Google’s data centre team used AlphaGo-like techniques to discover creative new methods of managing cooling, leading to a remarkable 15% improvement in the buildings’ energy efficiency.
Which itself is an understatement of the achievement:
> Our machine learning system was able to consistently achieve a 40 percent reduction in the amount of energy used for cooling, which equates to a 15 percent reduction in overall PUE overhead after accounting for electrical losses and other non-cooling inefficiencies. It also produced the lowest PUE the site had ever seen.
Basically, they built an AI that was able to tune the "large industrial equipment such as pumps, chillers and cooling towers" to react to the dynamic, nonlinear interactions that vary within and between datacenters (weather, utilization, etc).
They also describe this AI as "General":
> Because the algorithm is a general-purpose framework to understand complex dynamics, we plan to apply this to other challenges in the data centre environment and beyond in the coming months.
They seem to imply that this technique could make almost any industrial process more efficient with minimal oversight/training/customization.
In the blog post in July 20 2016 they said "We are planning to roll out this system more broadly and will share how we did it in an upcoming publication" https://deepmind.com/blog/deepmind-ai-reduces-google-data-ce...
They're innovating in a problem space that only exists at Google, Microsoft, and Amazon.
This is the boringest of sauces.
In 2014, their data centers used 4,402,836 MWh. Even at 25$ per MWh, that's already $100B, so this little project basically saved them around $40B...
What I find fascinating is how different AlphaGo's impact was from the impact of early chess engines. Once chess engines became decent (not even good—before even Deep Blue), they identified tons of inaccuracies in published chess literature. These were missed tactics, hard-to-see moves, requiring only relatively shallow calculation but a computer's precision. These inaccuracies were found even in classical annotations of well-known chess games, as well as standard books about openings. I believe John Nunn was known for this kind of work.
AlphaGo hasn't achieved nearly the same impact. Have they even tried to identify the same types of inaccuracies in classical Go books? Can you imagine how absolutely cool it would be for a go engine to find errors in Invincible? Maybe they tried, but didn't find any inaccuracies, so now this negative result is sitting in one of their file drawers? I really wish they were more active with this sort of thing.
Within one year of their invention?
Interestingly, for the past 3 or 4 days there's been a mystery Go Bot playing on 2 servers calling itself "Master" which is currently about 50-0 (!) against professional opponents, many at the very highest level. Nobody knows its identity yet. It's stronger than the commercial engines, and its presence must be some sort of stunt or test by someone.
Diagrams with moves at https://tieba.baidu.com/p/4922688212?pn=150&
Google translate helps with the comments but they don't add much to the diagrams.
They are playing fast games with 3 time periods of 30 seconds per move. It means that one can exceed the 30 seconds limit and move on to the next period. When the limit is exceeded in the last period the game is over.
Fast games usually increase the chances of the stronger player if the difference is large.
Edit: 59-0 just after I posted this.
Strongest open source engine is Pachi, which has achieved an amateur 4 dan ranking on the KGS go server. Fuego is almost as good.
Since a week, I hear news about a bot named "Master" that may be an evolution (or a concurrent clone) of Alphago.
You can study the games: http://www.dgob.de/yabbse/index.php?topic=6381.msg208264#msg...
There are speculations that the next game between Alphago and best players would be an handicap game.
IMHO Alphago is starting a revolution in the fuseki theory that is bigger than fixing a couple of broken openings.
I don't mean to slight the game of Go, but it is a perfect information game which could make it non-central to AGI research.
My goal is audio books - I'd love to hear them read by my favorite movie characters.
What? Why don't they consider themselves part of Google?
The most clear sign I have seen is that Partnership on AI site has both Google logo and DeepMind logo.
I assume Google wants to have them doing their own thing instead of just borg-similating it. Also they have Google brain, which is doing similar things
Backgammon - AI usually beats the best humans
Ms. Pac Man - AI loses to almost any human
A game like go is hard because there are many possible moves every turn. But the games themselves are not very many turns, so it is easier to find the important moves, because in some sense all the moves were more important. A game like Ms. Pac Man is hard because, while you only have 5 moves each turn, the games themselves are many, many turns, and it is very difficult for the NNs to learn these long-term dependencies.
Otherwise we need 4 orders of magnitude more compute, and it's just hard to believe their is no better way.
Handling random ghost movements is only trivially confounding for AI learning. Low value, low yield.
I agree with the parent comment that some sort of new approach is probably needed. The main issue with reinforcement learning is just that the algorithm gets too little feedback. This ends up discarding an enormous amount of data on each run (or, more accurately, learning from that data extremely inefficiently). There's a huge amount of data present, but the learning from that data has to be unsupervised. A child can do it easily --- they only have to play Ms. Pac-Man once or twice to get the hang of it. Computers currently cannot. Getting them to do that will definitely require something different, but no one is quite sure what.
https://deepmind.com/research/dqn/
edit: Checking wikipedia looks like it isn't playing all games it can play at superhuman levels
> For most games (Space Invaders, Ms Pacman, Q*Bert for example), DeepMind plays below the current World Record
https://en.wikipedia.org/wiki/DeepMind#Deep_reinforcement_le...
DM scored less than 30,000. Human record is over 900,000.
Not true. Many people have also tried handcrafting as well and only scored around 30,000 which is a low score.
Multiple papers have been written on failed attempts. Dozens of other developers have tried.
It's an open problem.
"Learning" AI requires the broadest feasible search space for any solution and successful designs require clusters of machines. Clusters of machines have network latency and coordination overhead.
This is the same latency vs throughput design decision that every system must make, and its not impressive when a throughout oriented system struggles with being latency sensitive.
You are wrong to separate video games from generalized AI. If you read the paper I posted above it explains part of this active area of research,
http://gameinternals.com/post/2072558330/understanding-pac-m...
Many other attempts has been made targeted specifically against Ms. Pac Man and none have come anywhere near beating an average human player.
This paper gives an overview of the background: http://www.cse.unsw.edu.au/~mit/Papers/AAAI10a.pdf
One could conceivably perform a depth-limited search on the actual game state if it were available and then use an AlphaGo-like DNN to predict what a deeper search would find, no?
We are mostly in SF and Mountain View, but we also have people in a few other locations. Right now, SF and Mountain View are the largest.
Disclosure: I work for Google on the Brain team.