What about this tweet from OpenAI?
"Our Dota 2 AI is undefeated against the world's best solo players" [1]
Also Musk called it more complex than Go,
"OpenAI first ever to defeat world's best players in competitive eSports. Vastly more complex than traditional board games like chess & Go." [2]
[1] https://twitter.com/OpenAI/status/896157788908290048
[2] https://twitter.com/elonmusk/status/896163163581825025?lang=...
I think there is a fair argument for Dota being vastly more complex than Go but there almost certainly isn't for 1v1 SF mid.
But that is not unfair. Humans receive plenty of curriculum training as well, we're not supposed to figure out the world by bumping into walls. Even in Dota2, the top players learned from observing each other how to deal with the bot. In fact, efficient retraining to include new strategies on the spot would be a very human-like learning ability.
I'm not a Dota2 player, but like SC2 for example is a game with LOTS of room for AI improvements. I've always thought that having some sort of APM limit might actually encourage AI authors to adopt new and unique approaches to macro-strats, but it doesn't seem to be on the horizon.
When it comes to do a small thing rapidly, I think bots are almost always going to win.
When it comes to do something large-scale with finesse, I think humans are going to have an advantage for a LONG time.
I think that part of what makes human agents so effective at certain tasks, especially in the context of being up against another human is that we can evaluate an event and better understand the WHY of it relative to the player that played it.
If I see a player pull back a bit, sometimes I think to myself that maybe they saw something they weren't expecting or something they weren't quite sure how to handle. When a computer sees the same move, a floating point number among millions changes slightly. I can try and figure out why they might be pulling back, if I did something weird or if I did something totally normal I might suspect it is bait, etc. I can think all these things in a short period of time and while large AIs might have better FLOPS than me, it doesn't understand what I'm doing, why I do it, etc.
Curriculum learning isn't as effective in bots as it is in humans is my contention, I guess.
Fair/unfair is a pointless observation when it comes to humans vs bots. The diversity of human-based problem solving is the perfect friction to train AIs against, imo.