Most of the games that machines can now challenge humans in are strategic, but slow: Chess, Go and poker, unless played in very specific settings, have no time constraints on player moves.
That is what has made the work of research group OpenAI, in online team brawler Dota 2 - which requires real-time decision-making between potentially dozens of choices in a single frame - so different.
OpenAI's bots, the OpenAI Five, went head-to-head against teams of professional players at Dota 2's annual championship, The International, this August. Although the bots lost, the matches provided an insight into how reinforcement learning is changing the game when it comes to artificial intelligence.
It's safe to say that AI has a reputation in gaming: many players consider a match to be an instant loss if they have to play with a bot, and a disconnect is often accompanied by "GG". However Mike Cook, an AI researcher at Falmouth University, told V3 that is a simplistic view:
"I think game AI is generally quite good, but people often don't appreciate how difficult the task of creating AI for games is. Game developers have a very difficult problem to solve: an AI can't be too good, because it'll beat the player all the time, but it can't be too bad either, or it won't be fun to play against.
"On top of this, game AI has to feel natural and contextual. If I'm playing a game against a bot I want it to feel like it's another human player, not like a piece of code with millisecond reaction times. I want to be able to trick them or distract them or surprise them, none of which make sense for an AI."
Thanks to its virtual years of training, OpenAI's system can act in a similar manner to a human. It utilises tactics like setting traps and focusing fire, and even creep-blocking (a technique whereby a player character slows down friendly minions by walking in front of them, to merge multiple groups into one). The bots that players are familiar with are much more limited, and mostly incapable of actions like these.
The machine threat
That isn't to say that today's bots are limited by their tech: if they wanted, the developers could make them play perfectly, to the point that it would be near-impossible to win against them - but that isn't what a game is about. "The difficulty comes in knowing how to restrict the AI without making it stupid, and how to make it seem intelligent without letting it unleash the perfect machine abilities it has," says Cook.
In a one-on-one situation, today's bots can perform capably; however, their inability to communicate with teammates exposes their weaknesses in co-operative play.
"Co-operative AI is much harder, because we have to guess what the player wants to do, and avoid getting in their way or becoming a liability."
OpenAI's bots have "a huge advantage" over their human counterparts. As well as split-second reaction times (these actually had to be artificially slowed down before the matches) and the fact that they will never fumble a command, they have an almost perfect knowledge of the entire battlefield.
Despite these strengths - which would see any human team dominate its opponents - the OpenAI Five still lost, which demonstrates the still-significant weaknesses of AI : so significant that their advantages don't make up for it yet.
Other changes - like switching to a single, mortal courier (a unit that ferries items to players) instead of five invulnerable ones - also certainly had an impact, forcing a change in OpenAI's typical strategy of getting a constant stream of healing items funneled to each hero.
One of those changes was picking heroes for the AI, in the interests of time, rather than allowing it to draft its own.
Even bots have 'comfort' heroes, just like humans do
"A lot of the bots' advantage came through [the draft] - we saw in earlier exhibition matches that the bots typically started a game with a 90+ per cent confidence they would win, simply based on the heroes they'd chosen to use… But this was a fair change I think: humans are used to playing with the standard 115 hero lineup, so restricting them to just 18 would have invalidated a lot of their knowledge and skill about the game."
Obviously, the difference between a hero pool of 18 and 115 has a significant impact on the balance of the metagame.
"If we remove most of these heroes, as OpenAI have, this balance is thrown off, and humans won't be used to the new relationships between each hero. I think some heroes play to OpenAI's strengths more than others… Gyrocopter and Sniper are two heroes the bots clearly love: heroes with big damaging ultimate spells and high damage. Other heroes, like Tidehunter, feel like they suit the bots' style less, even though they may do well playing against other bots… I think that even bots have 'comfort' heroes, just like humans do!"
Cotton seedling freezes to death as Chang'e-4 shuts down for the Moon's 14-day lunar night
Fortnite easily out-earns PUBG, Assassin's Creed Odyssey and Red Dead Redemption 2 in 2018
Meteor showers as a service will be visible for about 100 kilometres in all directions
Saturn's rings only formed in the past 100 million years, suggests analysis of Cassini space probe data
New findings contradict conventional belief that Saturn's rings were formed along with the planet about 4.5 billion years ago