AI Learns to Play CS:Go via Large-Scale Behavioural Cloning
arxiv.org
arxiv.org
Finally, you just do dead-simple behavior cloning (predict "expert" action from current observation) on the large labeled dataset. It's still surprising to me how well this works! Behavior cloning has some theoretical issues since it doesn't address the sequential nature of decision problems at all, but apparently with enough scale it can still do fairly well - super cool.
Examples:
- Follows the wall contour using the crosshairs when moving around the corners. The only thing wrong here is that the AI is aiming at the wall instead of at the open space next to the wall where the enemy would be.
- After going through a door, it checks all corner to clear the room. But it's very slow in doing this, and would get shot instantly in a real game.
- Adjusts the crosshair constantly to not look at the floor when walking. The problem is that the crosshair is never at headshot level!
I am not bashing on this AI btw, just pointing out that it's actually getting close to real human behaviour! Fascinating!
Something I've noticed is going well is the fact that the AI doesn't spray n' pray but controls the spray pattern and drags the crosshair down. Shoots in bursts!
This is actually how well-practiced humans play, aiming instead where the enemies are likely to be (behind the wall) instead of trying to perfectly aim at the corner all the time.
They started with a system that lets players review games and try to understand if a player is a cheat or not. Games are anonymized, and you get to rewatch the replay to try to make a determination
They can then use that data to train their NN to help detect which players / games should be fed into this manual review process
I assume now the product is probably good enough that it's ranking reviewers too to determine how good reviewers are at detecting if there's cheats or not.
It's pure genius
Here's a GDC talk you may find interesting: > In this 2018 GDC session, Valve's John McDonald discusses how Valve has utilized Deep Learning to combat cheating in Counter-Strike: Global Offensive.
Things like moving around the map in a rational way, learning how to counterplay enemies, tactics when entering a room, all based on solely visual data, are extremely difficult. These skills are much more intuitive to humans, but much harder to learn than "there is a human head at {523,1021}"
I'd love to see the same method, but with audio inputs too.
You might not realize how bad that actually is. Built in bots on medium difficulty can't even challenge a complete beginner.
> Whilst RL research often aims to maximise reward, we emphasise that this is not the exclusive ob- jective of this paper – perfect aim can be achieved through simple geometry and accessing backend information about enemy locations (hacks and built-in bots exploit this). Rather, we aim to produce an agent that plays in a humanlike fashion, that is fun and challenging to play with and against.
love it!
https://github.com/TeaPearce/Counter-Strike_Behavioural_Clon...
Also so awesome, I've been wondering how people automate games. it looks super easy & simple, no longer a big mystery :D
In SQuAD 2.0 a model is given a question, and a passage of information. It has to select the start and end points of where the answer is contained in the passage.
Your problem is simpler in that there is no question - you just have to have a model that can identify the start and end positions that _you_ will select.
this game has much more going on than games like mario, and even dota 2. for one, audio matters! also, there are more inputs to train, such as walk, jump and crouch.
this is a great start, but a lot more needs to be done before it can be called a complete solution.
> Nah bro, let's teach it to shoot people in CS:Go!
Hopefully not too off-topic, but AI playing chess, doing this or that "better than humans" is not a 1st world problem.
I never followed the chess scene and was intrigued with the Han Niemann angle, but noticed the chess engines being overblown with a lot of it (play ping pong against a wall type of stuff).
The 1st thing I thought was a chess format called "turn the tables" where chess players perform the 1st 3-5 moves, then "turn the board" and compete with the oposing players setup (did not research and this may already exist).
A Deep Blue versus Garry Kasparov rematch using "turn the tables" gameplay would be cool.
I did laugh at Magnus Carlsen winning in 1 move when the opponent resigned after a crappy knight move by Magnus at the recent tournament. TtT chess takes a lot of the memorization strategies out of play for the natural GM's.
With much of this AI stuff, nobody is chasing AI because of its severe limitations, impediments and attribute deficiencies. Just play ping-pong against a wall.