Integrating GTA V into Universe | Hacker News Reader

Integrating GTA V into Universe

openai.com

805 pointsgdb165 comments

I love GTAV so much, it's the most real single player world I've ever experienced. I once rear-ended an NPC because they suddenly stopped in the middle of the street. I got out of my car (because it's GTA and you have to shoot everyone, duh) I saw the NPC had stopped because he hit a cat with his car!

I'd be lying if I said OpenAI Universe / GTAV wasn't one of the primary drivers in me trying to learn ML.

Anyone know what kind of hardware I will need to play with this? The AMI page on github mentions an AWS g2.2xlarge (iirc a GTX 1060 is slightly better than this?). And it seems like GTA is actually running on a different system you use VNC to share the screen/input? Any estimates about whether a 1060 or 1080 on a beefy gaming PC could handle both at once?

It's technically possible to run everything on one Windows machine (using VNC loopback mode), however we expect most people to run ML on Linux / Mac so that is the best supported configuration. Before Universe, I ran a modified version of Caffe in Windows along with the game on a GTX 980 and there were more than enough GPU resources for that. Running ML / ROS / other self-driving software in Linux is much more ideal though.

Thanks for this info. My gaming PC has a 7950, so I'd need to buy a Nvidia card anyways to do this. I was hoping to avoid the need to put together a second PC for ML, but I think I have enough spare parts lying around that I could do it cheaply, and I believe you about Linux being ideal over Windows.

Sounds like a 1060 should be a good start for me to play with Universe then.

Hmpth. Conversely, while playing GTAV in completely innocent explore-the-world mode, I accidentally bumped a police boat with my jetski. Apparently this warranted being shot in the head.

Most reviews I've read actually complain that GTAV is a step back in realism from GTA4 (for example, body-part specific injuries were dropped), but let's not beat about the bush - no GTA deserves the title of "realistic", because as you say "it's GTA and you have to shoot everyone, duh". They're made to feel realistic, but in fact they powerfully reward violence and punish being law-abiding, in precisely the opposite way that the real world does.

> no GTA deserves the title of "realistic"

depends on what you prioritize to mean "realistic." I think most people would agree that the world of GTA V (and IV) is incredibly realistic by video game standards, to the degree that it's the obvious choice for this AI project.

That is right in everything, but shooting and taking damage, which is mostly irrelevant for driving AI.

Well, if it had imported the body-part damage bits from GTAIV (which it didn't), at least those are easily fixed by tweaking some data points.

Of particular relevance here, another aspect where GTA V is supposed to be less realistic than IV is driving physics (i.e. handling). I wonder if OpenAI is using one of the mods designed to address that.

I enjoyed GTA V because it was so hilariously over the top.

It didn't try to be realistic. The whole point is that the police response is always massively disproportionate to the crime.

It's a pastiche of society, everything is exaggerated: they have a legalize medicinal cocaine campaign, you can buy supercars online, rocket launchers and machine guns at your local gun shop, you can launch your car at 100 miles an hour of the edge of the highway and survive.

They even reference the outrageousness of it in a scene where Michael goes to the therapist and comments that he may have even killed someone on the way there.

>it was so hilariously over the top

Every animal in GTA V will call the cops on you if they witness you doing a crime -_-

I think it tried to be super realistic in many ways and deserves some credit for it. Its like they made a realistic world simulation, and then tweaked a few parameters so it would be funner to play as a game.

Like obviously no real person would be able to survive the injuries my character gets every 10 seconds. But if I was dying all the time it wouldn't be fun to play.

The one thing that really bothers me, as far as realism goes, is lack of persistence. Leave a car somewhere, walk 10 feet away, turn around, and it vanishes. Cars and NPCs vanish when you aren't looking. Damage you do gets quickly reset as if it never happens. And a number of small details like this. It's not just that it's unrealistic, it really limits what you can do with an otherwise awesome sandbox.

Still, I think the game that deserves most credit for realism is dwarf fortress. It will be a long long time before AIs master that.

You have to consider the technical limitations as well. It was designed to play on the PS3 and Xbox 360. I imagine the next generation GTA will offer more of what you are looking for.

The accretion of player driven changes in an open world in an interesting thing I'd never thought about from a storage perspective.

If the game randomly JIT generates all local phenomena while keeping just enough coherent higher level data to suspend your disbelief, that's only the storage size of the high level information.

However, when a player begins to specifically perturb that generated phenomena at the detail level, how do you prevent leaving an ever-storage-requirement-increasing trail back through the player's space-timeline? Reconvergence to normal over a reasonable timespan?

I assume this is a thought about problem, so links greatly appreciated!

Most video games do stuff like that. It is necessary to some extent for the reasons you say. My issue with GTA is they do it excessively. A car that I just parked, 30 feet away, should not disappear when I turn around. If it disappeared after an hour, or after travelling a mile away, that would be much less restrictive.

I'm pretty sure the reason why they prune the memory tree so aggressively is that the storage requirement is proportional to the "memory radius in spacetime" cubed (or worse).

Skyrim does it better in my opinion. It resets a block after 2 game days (144 minutes playtime, and waiting, sleeping and faster travelling make time pass faster). So you can leave the loot that you can't carry in the dungeon, and after you get back to it, it would still be there. So it feels more "realistic".

As far as I remember, Mercenaries 2[1] maintains your world destruction.

[1] https://en.wikipedia.org/wiki/Mercenaries_2:_World_in_Flames

One of the limitations of GTA that I've found most frustrating is that the game is seemingly programmed to prefer red lights in the streets, I think this is to force you to slow down a bit and let the loading occur more seamlessly, but it's still annoying to just run into traffic jam after traffic jam.

You adhere to the traffic regulations in GTA? :)

No but the NPC cars do, creating little hard-to-pass blobs of traffic

"seemingly"

Look into confirmation bias

Men in suits also correct damage in Westworld. Sorry, I am obsessed by this new series! :D

When they killed Mark Zuckerberg was also interesting. Reality is GTA is a parody of American Culture. The developers aren't even American, at least not originally. They were all from Europe, that's why Vice City is basically a Scarface spoof. Then you got San Andreas which looked and felt like I was watching Boyz N Da Hood / Friday, and other movies.

This might be getting way off topic, but I noticed that one of the corrupt government official at one point in the game said "I'm glad I don't pay tax", which is something an American would never say, we say "pay taxes".

I think a lot of time outside observers are better at parodying something than someone who is inside of it. They just nail the ridiculous culture in the US. I remember GTAIV had a television you could watch, and "Republic Space Rangers" was one of the funniest shows I've seen anywhere.

I agree. I interpreted GTA V as a bitter satire of American society and culture, set in a technologically marvelous video game (which is pretty 'meta').

> They're made to feel realistic, but in fact they powerfully reward violence and punish being law-abiding, in precisely the opposite way that the real world does.

You're right, realistic maybe wasn't the best word. I think "believable" would have gotten my point across better. It's true that GTA V mimics the real world pretty terribly, but it's game world and AI is so incredibly internally consistent that it feels less like a game and more like a another world.

> I accidentally bumped a police boat with my jetski. Apparently this warranted being shot in the head.

Were you playing as Franklin? The GTA V devs have denied this, but from personal experience and plenty of online reports, the police in the game are racist. More realistically, it's probably because they're unable to pathfind to a state where they can arrest you (because you're both on boats on water), so they just go guns blazing.

I saw a video not too long ago that determined that the police in different parts of the game's city were more/less aggressive to each of the three player characters.

>they powerfully reward violence and punish being law-abiding, in precisely the opposite way that the real world does.

Except if you're into politics.

> Hmpth. Conversely, while playing GTAV in completely innocent explore-the-world mode, I accidentally bumped a police boat with my jetski. Apparently this warranted being shot in the head.

I don't know--these day's I'd believe it.

Once I was driving normally, and a cop car drove into me. So of course the cops immediately got out and started shooting at me, even though it was 100% their fault. Definitely broke the realism.

(I had right of way and they didn't have lights/sirens on)

Physics, particularly vehicle physics in GTA IV was considerably more realistic. I enjoyed it more, but GTA V physics are probably more fun for the vast majority of people and honestly, I don't mind it either.

Were you playing as a black character?

Great point about it being not realistic, although I think it is no doubt this is a game, so it promotes people to go wild in the virtual world. I rarely hear people say GTA brings realism, instead it is a virtual world where players can do things they can't do in real life.

Sounds like Westworld! :D

I agree with you. When i was younger (until 25 years old), i played a lot (GTA, WOW, Counter Strike ...) and i have this inclination to think that games always have some realism inside then.

In fact it's quite the oppose. What makes games fun is taking what we call reality and make it "closer" and "unreal" at the same time. Like ... "hey, look, i'm shooting a RPG ...". But it's does nothing to do with shooting a RPG, you are just pressing a key and seeing something shining in a block of glass.

Problem with games are that they trick you mind to think something is real. But it's far, very far away from the real. It's just you, sitting at there, imagining things.

Exactly, reality is boring -- my favorite shooter always was Unreal Tournament. :-)

These days I play an undead thief in Dark Souls 3. (But maybe it is realistic, I have no idea how professional thieves live.)

I don't remember if GTA V had this same feature as GTA IV, but I remember being impressed in GTA IV by the cell phone interference through the radio when you had an incoming call and the radio was playing. I just absolutely loved the attention to detail that game had. It was truly the small things that added up that made you want to figure out what other details they put in the game.

Ok, I am so glad that we're not training AI to "win" at GTA V. That is not an android I want to meet on the street :-)

The transitive property of teaching an AI to drive given a world simulation, makes me wonder if you can train a network to recognize speech by feeding War and Peace through a text to speech program and then sending the audio data generated into an RNN. Where is the ground truth in such a system which you can calibrate your results by?

> Where is the ground truth in such a system which you can calibrate your results by?

Answering standard reading comprehension test questions? - "Why did Pierre meet Natasha at the church?"

I'm not sure I would like to live in a world with autonomous vehicles that were trained in Grand Theft Auto.

I guess trianing/testing the training might work well though

This is a modified version of GTA where the behavior of people is non-violent.

Maybe the most unrealistic part about the simulator, then.

Why?

They're implying that lack of road rage makes it unrealistic.

Sounds a bit crazy until you look at it from the perspective of being able to visually observe how an AI driver would react to erratic behavior like pedestrians suddenly running onto the road or conditions like gunfire, accidents, etc. I think this is just a small step in the positive direction of AI development.

It's surprising how scriptable the game is. For example, the car I train the AI with is set to the least aggressive driving style which stops for rabbits crossing the road :) Also, all violence against the player is disabled.

Don't you want your self driving car to be able to get away from the cops?

Mul-ti-pass?

I don't think anyone is thinking of training on GTA then reusing it on a real car. It's just a convenient way to work out the algorithms and architecture in a controlled way that might then work out with the real world data.

But I do think GTA's unrealistic physics engine[1] will hurt it for simulating the cases we care most about (accidents, lost of traction, etc)

[1] GTA is a regular submission on /r/GamePhysics which collects funny game physics glitches: https://www.reddit.com/r/GamePhysics/search?q=GTA&sort=top&r... https://www.reddit.com/r/GamePhysics/search?q=GTAV&sort=top&... https://www.youtube.com/channel/UCy332GE6lcMVP612IUs7ODw/vid...

exactly, i saw a similar complaint as to the gp's in a thread about working with classical computing to simulate quantum computing

my argument then was 'imagine prototyping on a machine that needs to be kept at 17millikelvin'

here the argument is 'imagine prototyping on a two ton missile traveling at lethal speeds through highly populated areas'

Wildly unrealistic vehicle physics is core DNA for GTA and related games. I remember playing True Crime: Streets of LA back in the days, where you could do doughnuts in a city bus. It was amazing.

If GTA's physics model is realistic enough, you could train a model in that world "safely", then use the model with transfer learning (perhaps with another dataset from the "real world" - either homegrown or pre-existing) to "fine tune" to the real world. At least, that's one possibility.

There might not be a better (virtual) alternative. Remember billions get sunk into these games. Don't think you'll get the same $$ for dedicated training environments.

I am sure and I do not.

This is cool.

Next step is obviously teaching it how to kill people in the game and get away from police.

I'd like to see the AI go on a rampage and win :).

That should be enough to wipe off a lot of smiles off our idealistic faces :)

It's all fun and games until the Second Amendment is used to protect the ownership of armed drones that can be used to assassinate anyone anonymously simply by scratching the serial numbers off.

yeah that's definitely a scary possibility, though I would think explosives would be the more realistic threat

anyways, electronic actuation of a firearm is already illegal in the US. if you're firing a gun in any way besides actually pulling a trigger with a finger, chances are good it's illegal (with some small leeway such as turning a handcrank)

If you're going to murder someone, you're probably not too worried about arms-control laws. You're already planning on doing something way more illegal.

But if you're going for some sort of obscure criminal achievement by only breaking one law at a time, then just make your lethal device something that does not legally qualify as a firearm. A black-powder muzzle-loader or a Bulgarian ricin-umbrella would do the trick, depending on whether you care about noise or not.

Or, like the parent mentions, make a grenade, and deliver it via remote-piloted aircraft.

Realistically, though, very few people on this planet would bother with any of that when they can often just kick down the door of your home, kill you with a tool purchased from a hardware or sporting goods store, and just leave before police can respond. The clearance rate for violent crimes in the US is already low enough, even for people who give no thought whatsoever to covering their tracks.

I'll continue to assume that unmanned aircraft are carrying cameras, and assume that if someone wants to murder me, they need a compelling reason to have a more complicated plan than to just bash my skull in with a big rock when I'm not looking.

Granted that paintball guns don't meet the definition of firearms, but I wonder if you've seen the software driven turrets that some folks have rigged up?

https://www.youtube.com/watch?v=nTs7VRFV36c (and others)

Substituting other devices to be actuated seems... trivial, and well within the reach of even hobbyists.

>anyways, electronic actuation of a firearm is already illegal in the US. if you're firing a gun in any way besides actually pulling a trigger with a finger, chances are good it's illegal (with some small leeway such as turning a handcrank)

Huh, that explains why Mythbusters always used a string rather than any sort of servo system. Neat.

It's only illegal when it happens to violate the definition of a machine gun in federal law (multiple rounds fired per trigger pull); there's no specific law about electronic triggers. The ATF has specified that electronic firing is acceptable as long as a single physical action by a human results in a single round being fired from a single barrel. A drone with a gun that fires a shot in response to a button press from a wireless controller would be legal (ignoring any FAA restrictions on guns mixed with aircraft). Electronically "storing" human trigger pulls for later use is not allowed, so it would never be able to fire on its own.

That would be really awesome!

Who's ready for some "Deep Pimping"?

http://gta.wikia.com/wiki/Pimping

ABSTRACT: We explore the adage "pimpin' ain't easy" in the context of deep reinforcement learning in the OpenAI environment GTAV. We train a Deep Q-Network (DQN) to optimize player cash flows. Over the course of training, our network learns to keep these hoes in line, and discipline misbehaving customers.

This is incredible! Huge fan of GTA V / Rockstar as well as Open.AI

Will be super interesting to see how all of the randomness in GTA V driving influences autonomous driving AIs.

I could see them being a bit more "cautious" in order to deal with all of the crazy drivers (and events) sharing the roads of Los Santos.

I'd be super interested in seeing how this reacts to having a single star wanted level. At that level police cars will come to a stop almost perpendicular in front of your car. Will the AI be able to avoid an aggressive obstacle.

Also it'd be interesting to see how it reacts to getting shot at

I'm missing (and the article doesn't exactly state) what the fitness function is here.

It might just as well don't care about being shot at.

We currently don't look at being shot at (although it's an interesting suggestion!) - the current reward function is basically this:

  distance_reward - off_road_penalty - speeding_penalty - slow_penalty - discomfort_penalty;

Plus if you collide or drive against traffic, the episode terminates.

https://github.com/openai/universe-windows-envs/blob/f5aad96...

Well that part was mostly a bit of a joke - It could add in a few interesting variables such as deflated tyres causing the car to drift more than the AI is expecting.

Thanks! Yes, the game is highly scriptable[1], so you can test the types of scenarios you'd want. We've disabled violence against the driver, but could enable[1] it on-demand if it's useful for training.

[1] http://www.dev-c.com/nativedb/ [2] http://gtaforums.com/topic/822314-guide-driving-styles/

What happens when a car hits a pedestrian at speed? Is it still a bloody mess as in the normal game?

I wonder if AI is where games can make use of many cores. If the game logic can not be reasonibly parallelized anymore, the unused computing capacity can be used to add "intelligent" agents into the game. They don't need to do revolutionary things, just being more complex, more "reasonable" and less predictable than the usual agents would be enough. I think that has the potential to make some games more interesting.

Imagine a game like Warcraft 3 where the units actually seem to have a personality and follow "their guts" in battles rather than following simple patterns like attacking the nearest enemy.

Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?

Mr Anderson...

Interested in this as well. Applying machine learning in this way could create some interesting game play.

I was told (but didn't verify it myself, nor have any sources on this) that in Half Life 2 (2004), your enemies would "learn" from your attack patterns and adapt to it. At the same time, as to make the game enjoyable/chellenging and not frustratingly difficult, the agents would adjust their difficulty level to your skill. Take it as an anecdote.

That was actually Left 4 Dead.

https://en.wikipedia.org/wiki/Left_4_Dead#AI_Director

The director in L4D was had a very simple logic with inputs like number of alive special infecteds and number of zombies to control spawns. It's like seeing a health kit when your health is low.

And the director in L4D2 actually changed layouts of levels to suit difficulty.

Have to start somewhere, though.

IIRC it tried to be slightly more clever (might've been L4D2) in that it also kept track of the recent history of the game and attempted to balance the pace of the action.

If it thought players had had it easy for a while it might throw a larger waves of monsters and specifically deny spawning health for a few minutes.

There's a good presentation somewhere about how it would try to balance the rising action with periods of "stress" and "relief" over the course of a level.

I've played an unhealthy amount of HL2 and spent a while modding it, and I don't believe this is true. People say things like this a lot about games with decent AI. For example, the old quake 3 hoax about bots "learning" to become peaceful on a server that was left running for 4 years. It's fun to imagine the AI as more than just a set of scripts.

The Valve Developer Community has some okay documents about the AI [0], moddb has some nice articles too (no links offhand). And most of the source for combine guards is available in the Source SDK [1], you can see tons of if's and hard coded sequences of actions.

I've read that the AI was changed after the Orange Box release, so maybe there was some learning features to them originally. Or maybe those features do exist within HL2's private source files, though it doesn't "feel" like it.

[0] https://developer.valvesoftware.com/wiki/AI_Learning:_Combin...

[1] https://github.com/ValveSoftware/source-sdk-2013/blob/master...

I've heard a similar anecdote about a racing game (I think it might have been in Forza) where the developer had a AI where individual racers could hold grudges and that would affect how aggressively they treated you (bump someone in lap 1 and they'll be more aggressive in lap 3). Eventually they had to take it out for various reasons, but didn't advertise that fact for obvious reasons. When the game was finally released several reviewers brought up the feature as a plus, not realising that they were just reading into the actions of the standard AI.

As a side note, the writeup of F.E.A.R's AI is a fascinating read. By combining very simple goals and actions, they created AI that is still remembered as incredibly good. http://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear....

I don't know about current games (I suspect they have enough to load the CPU with gfx/physics to devote much to AI), but state of the art AI software can definitely be many core. It's very useful for anything doing Monte Carlo simulations, tree search, planning.

Games haven't really been pushing CPUs very hard for the past few years. All the mucking about with overclocking has been purely a matter of culture recently, as you can play top tier games in 4K with a mid-range CPU, as long as it can handle enough PCIe lanes for your monster GPU setup.

Depends on the game, really. That may be true for some, but far from all.

Take a look at the game sections from the recent Kaby Lake reviews. Here's Anandtech's for example: http://www.anandtech.com/show/10968/the-intel-core-i7-7700k-...

Most of them look pretty much like that, with very little difference until you start getting to really low-end or old chips. Games are mostly GPU limited these days and those that are CPU limited are mostly leaning on one or two cores and thus haven't really gained much from recent generations.

You're arguing that benchmarks of recent top of the line chips (which at most tend to have a few tens of percent difference) not showing large performance difference is evidence that games aren't CPU limited, whereas the same test shows drops in single-thread performance immediately cause large performance differences (see AMD chips).

If anything, this shows that games aren't strongly multi-threaded (which comes with severe development and debugging impact) or aren't made to scale beyond the highest end chips consumers are likely to have.

Lastly, consider that while the graphics effects and physics effects of a game can be scaled down, scaling down the AI in many instances would cause balancing issues.

Since when is a $168 i3 chip "top of the line"?

Imagine an RTS or something similar where you play with an AI and he learns your behaviours. He tries to help you. At first you deny what he does, he learns, do something else. A bit like the beast in black & white!

pretty sure Rival Theory was working on something similar to that

There's a recent game called Hello Neighbour where the eponymous neighbor learns from the patterns you use to sneak into his house. The one video I saw of the game looked freaky. 10/10 would play.

> Imagine a game like Warcraft 3 where the units actually seem to have a personality and follow "their guts" in battles rather than following simple patterns like attacking the nearest enemy.

> Maybe I'm ignorant about the current state of the art and games already do that. Do you know examples?

It's a really tough trade-off between fun and realism. It's cool that your units have personality for a bit "Oh, look, those 2 dwarves are fleeing from the troll fight early!", but not fun once they disobey your intentions "That fucking troll has 3 hit points left. If my 2 dwarves didn't run away, it would be dead! Now it's destroying my base!"

One game I remember striking a good middle ground with this was the original Battle For Middle Earth. It changed animations, sounds and other aesthetic things based on AI, while the game remained the same mechanically. If your units ran up against a mountain troll, you would see the ones in front of it cowering and holding up shields, while units behind attack furiously, but every unit was really doing the same damage. Or if you left troops of Orcs just standing around too long, they'd break out into fights and kill other orcs, but the animation is done as a trick where as one orc model "dies", a new one spawns in the group. So you didn't really ever lose Orcs, even though there would be orc corpses strewn wherever you had been camping.

I think the main reason though is just that AI doesn't really sell. Graphics, physics and voice actors sell. You can blame a failure on AI, but very rarely is success attributed to good AI in a game.

Bad AI often winds up panned in a game's review scores, which have an impact on sales.

For instance, bad pathfinding in an RTS game can make the game almost unplayable. Halo is just about legendary at this point for teammate AI that is worse than worthless [1]. Any one of these things can destroy immersion at a critical point.

AI is to video games as IT is to a modern enterprise. If it's doing its job, nobody notices, but if it's bad, it sticks out like a sore thumb.

[1]: https://www.youtube.com/watch?v=vdbg8h3d3uM

Or, if it does its job really well, it gets accused of cheating.

My former company had a backgammon game with a really good AI. We constantly got accused of cheating, even though the dice-roll algorithm was identical (it didn't have a parameter to specify who the roll was for). What people missed is that a good backgammon player will position themselves to benefit from a wider set of possible dice outcomes.

I was under the impression that backgammon was a solved game (i.e. there is a known optimal move for every possible position), can I ask why you used an AI instead?

Probably because an AI that always plays optimally wouldn't be very fun to play against.

Cannot find it right now, but I saw a video of Civilization 's Sid Meier saying that about Civ's playtests.

> Bad AI often winds up panned in a game's review scores, which have an impact on sales.

This is what I said though. A game's failures can be blamed on bad AI. But once you reach the point of okay AI, you stop hearing about it. There's very little reason to put in whatever effort is required to make your AI great, because the only people who will take notice are hardcore players who put in tons of hours. And that's nice and all, but the game costs $60 whether you play it for 10 hours or 100. So why spend millions on a feature that's hard to make a trailer for, and is only noticeable to your most hardcore fans?

>Graphics, physics and voice actors sell.

This is hardly arguable, but personally, after some time playing I begin to recognize animation/sound/timing (i.e. behavioral) patterns of non-plot entities and that ruins the entire experience. That's what retains me on the storyline and prevents free exploration -- there is nothing really interesting besides nice graphics. I think there is a room to evolve from [hundreds of] programmed random encounters to purely independent world.

Actually, I'm fine with GTA3/GTASA-like graphics, like I was with roguelikes and physical book-and-dice quests, so idk how that applies to wide public.

ps. Another conclusion, it may appear too frustrating. We probably play games to win and achieve. If there is no end, no goal and no perfect winning strategy, that understanding alone may cause depression, like IRL.

Not necessarily, we play games for play sometimes. Because it is fun. Take Minecraft for example. No serious win-state (until fairly recently) but it still has tremendous appeal.

I remember that in Close Combat https://en.wikipedia.org/wiki/Close_Combat_(series) the units have some small independent thinking and refused to follow very stupid orders and wanted to take cover. It probably has a few simple rules, like an expert system.

As discussed in other comment, it was a pain to manage them, because they didn't follow all the (stupid/suicidal) orders.

>I think the main reason though is just that AI doesn't really sell. Graphics, physics and voice actors sell. You can blame a failure on AI, but very rarely is success attributed to good AI in a game.

The main reason that I avoided buying Civilization 6 is that the AI was notoriously bad. For any kind of turn-based strategy game, I want the AI to be superb.

I'll note that this actually makes the original point, since despite your avoidance of the game, Civ VI is estimated to have been the top grossing game on Steam in 2016. So, even in this case, the game has sold gangbusters despite lackluster* AI.

*I didn't intentionally rhyme lackluster with gangbuster. Rather these are the words that sprang initially to mind, and it's a kind of satisfyingly weird combo, so I'm leaving it in.

Source: https://galyonk.in/steam-sales-in-2016-def2a8ab15f2#.nxpnuko...

> I think the main reason though is just that AI doesn't really sell.

I'm not sure that's the case. Last of Us and MGS:V both had intelligent AI where they would change their behaviour based on what you've done in the past and work together as a team to flush you out of a hiding spot. This was widely covered and praised in reviews and pre-launch interviews.

Game logic has been parallelized since about 2005-2010.

In ARMA opponent AI is a parameter. When you have a lot of players on one server, you turn down enemy AI so you don't bog down the entire server with pathfinding calculations.

It looks like it's not just using the camera/vnc images as input, but also uses depth data to simulate LIDAR input (http://deepdrive.io/).

I wonder, what makes it hard to rely on solely the 2 cameras to calculate/guess the depth, similarly to the way us non-robotic drivers do it, when it comes to machine sensing.

EDIT: found this: https://web.stanford.edu/class/ee368/Project_Autumn_1516/Rep... on the topic. Is there anything else I should read up on?

EDIT: found lots of articles: https://www.google.com/webhp?sourceid=chrome-instant&ion=1&e...

Open world games seem like a great place to work with AI. What has traditionally prevented more intelligence, in my mind, was the ability of AI to understand and integrate with its environment. For example, if I say I threw a baseball, you can visualize that. In a virtual world, the AI can "see" and potentially interact with every object to increase understanding. Great experiment and I look forward to seeing how far you can take it!

Off-topic, but does anyone know whether it's the GTA engine used to record these videos:

https://www.youtube.com/watch?v=T5kooJeXGiI

There's a ton of them on YouTube, but I can't figure out how they make them.

Yep, you can tell by the way the helicopter sounds and the walking animation for spiderman. The motion blur also looks very similar. The terrible looking ENB mod makes it difficult to notice, tough.

It's just a normal stock game or some mod added.

Some use the GTA V. director: http://kotaku.com/on-youtube-millions-watch-shows-for-childr...

The creepy thing is that this video is targeted at 0-3 year olds who randomly tap on mom's iPad. If you think like that now it's clear why the title is just keyword stuffing, the lame music and the spiderman outfit.

80 million views in a month is a lot of money: https://www.youtube.com/watch?v=ipTJNNvW-Gw

Just checked their channel: 4,316,098,805 views

"80 million views in a month is a lot of money"

What would 80 million views translate to in dollar terms?

I have seen $2,000 / million views, so $160,000 / month?

>Just checked their channel: 4,316,098,805 views

Where do you see that count?

https://www.youtube.com/channel/UC0gNKhFMg-bKyNNZ_MB3D9Q/abo...

be careful, those videos melt your brain in an instant

This is super cool. Are there any good resources/how-to modify GTA V for things like this? I'm looking in to GTA main website and wikies but info for developers looks thin.

Here's few links I found so far:

https://en.wikipedia.org/wiki/Modding_in_Grand_Theft_Auto

http://www.dev-c.com/gtav/scripthookv/

https://www.youtube.com/watch?v=wS8oJTHqf8Q

https://www.reddit.com/r/GrandTheftAutoV_PC/comments/38wct0/...

https://steamcommunity.com/sharedfiles/filedetails/?id=55807...

http://gtaforums.com/forum/109-tutorials/

Also I'm reading that GTA folks went out of their way to prevent modding. So may be everything you do with GTA is a hack and there is no supported APIs for developers?

Rockstar's policy is that you're free to mod the single-player game, but any modding whatsoever of the online game is forbidden, no matter how miniscule. They detect it and it's a potential ban.

That said, they don't publish anything about modding and don't have an official API available. Everything has been done third-party.

The modding has caused real problems in the online play - player in godmode, handing out in-game money, etc. No matter the efforts of R*, they seem to be unable to eradicate it for good. The group of players I previously played the game with, including myself, have all become extremely disenchanted with the game by the online modders.

It ultimately boils down to a cat and mouse game. What you end up with are extremely intrusive anticheat tools that do all sorts of nasty Trojan-like things; for example, injecting DLLs into a player's host machine to determine if they have modified the memory of the game.

The anticheat process itself will hide and run in kernelmode. I don't know about you, but I don't feel terribly comfortable having a closed source binary running in ring0.

Nor do I, though it should be noted that most of the games with intrusive anti-cheat are Windows-only so the entire kernel is already closed-source blobs…

I'm more comfortable with an anti-cheat blob running in ring0 than I am game-mod code running in any ring.

But, I run on a machine that was purpose-built for running games and if it all goes to hell, I just wipe and reload. All the games are from Steam, so they're easy to get back on there (and I only play a handful of those, so ...).

This feels like an important step in the democratization of AI.

We have algorithms, data, and computing power which are unevenly distributed. Providing easy-to-access training data for a self-driving car feels like a huge step in democratizing the data aspect of car AI.

With all respect I think "easy-to-access" is a bit of an understatement when you are comparing a 3D game to real-world sensor data. They are barely comparable.

The fact that you can even consider training an NN for driving after under an hour of setup is absurd.

Did you see the "self driving mario kart"? [1] That's a lot of complexity just to try out some algorithms for driving.

Is GTA V the same as setting up a rig which controls I/O for a real car? No. Of course not. You're not being fed data from LIDAR, which Waymo [Google X driving] seems quite invested in. [2]

However, much of Waymo's self-driving training is simulated [of course it is, how else would you get enough data?]. This is filling out a gap that is currently otherwise only filled by proprietary solutions. That's a big deal.

[1] http://kevinhughes.ca/blog/tensor-kart [2] https://news.ycombinator.com/item?id=13352385

> Is GTA V the same as setting up a rig which controls I/O for a real car? No. Of course not. You're not being fed data from LIDAR, which Waymo [Google X driving] seems quite invested in.

I've been coming around to the conclusion that LIDAR isn't going to be the way forward, unless prices come down greatly and mechanical scanning is done away with. Perhaps something like Flash LIDAR, if it could be made 360 degree FOV and cost under $500.00 in quantity - that might be worthwhile for manufacturing a vehicle with, but currently even the cheapest 3D LIDAR is still way too expensive.

I honestly think we'll see camera-based solutions becoming the norm; high-speed, high-resolution machine-vision cameras are already an established technology (though mainly black-and-white; pair those with lower-res color cameras and fuse the output as needed). Set up multiple stereo-camera pairs for overlapping 360 coverage - it would likely be a fairly cheap solution.

At that point - the frame-buffer view data from GTA (or some other simulator) could be considered close to "real world" for data gathering and model training.

I think LIDAR is essential - at least for any cars to achieve L5 in the next 4 years.

[which I also think will happen]

I think it will become cheaper - probably to ~$1-2k in the next two years.

And even if it doesn't, we aren't expecting self-driving cars to replace the fleet of cars on the road, we're expecting to increase utilization % of cars. So we won't need as many vehicles, and the cost per mile will still be much cheaper.

But GTA V is scriptable. We'll be able to simulate LIDAR in-game within a few months, if desired. So... all-told, I'm pretty excited about the possibilities here.

If it is highly scriptable, we may see people simulating sensor data within the GTA V environment. Data streams from cameras, lidar, and other types of sensors could be simulated with as much noise as you want. This could be used as input to train an AI using data that looks much closer to what you'd see in the real world.

This is way easier said than done. Even with added noise, the sensor data from the real world is nothing like simulation.

Probably Uber should practice in GTAV, rather then on real roads:)

It would be really great to see OpenAI sponsor AI contests, the # of people contributing / progressing on specific goals would be enormous.

This is really awesome, great work to those involved.

So imagine I'd like to start doing cool stuff like this by myself. Where do I start?

too bad GTA V is not open-source for researchers. I don't know how customisable it is, but e.g. experiencing snow/ice or a failure like spontaneous pressure loss in one of the tires would be interesting to simulate.

Yes, it would be great to have source! The modding community does make up for this in some ways. For example, mods provide access to things like finding memory addresses for things like steering angle that weren't available in the API for game scriptors working on GTAV:

https://github.com/crosire/scripthookvdotnet/blob/dev_v3/sou...

re snow: It is possible to lay fresh snow anywhere in addition to drive through this area with snow burms, etc..

https://www.youtube.com/watch?v=rOo2mtuPkKQ&feature=youtu.be

Traction is definitely affected in snowy areas.

re tires bursting: it seems possible

https://github.com/crosire/scripthookvdotnet/blob/2b14a22ad3...

> artificially slowed to 8FPS

i'm sure this is to offer relief to the nn but does this further simulate 'reality'?

are our autonomous vehicles seeing at 8 frames per second?

As the average human reaction time is 0.19 seconds, I don't see 8FPS too bad for a start. More FPS is always nice, of course.

interesting comparison but it is a bit disingenuous

there is more going on than just input in that reaction time figure

8fps fails to assume a reaction time of 0.125s for the same reason that the human eye's ~220fps.. now this is a mental stretch of the true underlying mechanisms but i'll make it for sake of consistency of argument(o).. fails to imply a reaction time of .00454545s

i guess i assumed autonomous vehicles sensor data would be measured in hertz, but now am intrigued by what framerate the computer vision system works

(o) http://www.100fps.com/how_many_frames_can_humans_see.htm

Reaction times and flicker fusion rates[0] are orthogonal. 0 ms latency at 8 FPS is different than 190 ms latency at 120 FPS. E.g. If there is an event that takes less than 125 ms, at 8 FPS there is a probability that you would miss it entirely! At 190 ms latency with 120 FPS, you would be guaranteed of seeing a 125 ms event, but processing of that information would take 190 ms longer.

[0] https://en.wikipedia.org/wiki/Flicker_fusion_threshold

Just a heads up to anybody who tries to get this running, you may have to request permission from Amazon to get the required instance going (g2.2xlarge). Apparently they are making sure people aren't trying to set up botnets and bitcoin mining. Just sent in my request, we'll see what happens.

The video shown is a bit underwhelming; the car moves at 1 mph with pretty much just one car behind it the whole time.

How much real-time is needed to get 1k (or 1M) iterations, to improve it so that it moves as fast as the speed limit allows while obeying all the laws?

Convergence on target speeds and steering is actually very quick (a matter of hours on a modern GPU / AlexNet / 227x227 RGB) once you have the data. Generating training data with the correct speed limit and providing speed limit based rewards is the missing step. The training car already obeys traffic laws.

Cool! You've proved my point :)

So the training car has a max speed of 10 meters per second or 22mph which the net matches. Future datasets will contain the road's speed limit as well as throttle / brake targets required to reach that speed limit.

This would make an interesting plot for a movie. Someday hacks GTAV ai into Google self driving cars. Suddenly "do no evil" company becomes "fuck the world up" and man tries to fight cars.

Anything that has wheels is dangerous to humans!

Imagine using this to train and to "unit" test a self driving car. With realistic graphics and the ability to control events, I wonder how useful this could be. (e.g. simulate near accident situations)

It looks like it has driving support but I read the post and can I make a pedestrian agent? It doesn't appear that I could. This looks like something I would like to play with and an awesome project.

Damn, OpenAI is a house

Did anyone try to feed this AI with actual dashcam images and see if the driving learnt in GTA would be applicable in real life situations ?

Is it possible to return observations that a well equipped driverless car would receive (e.g. lidar, sonar, multiple cameras, IMU, etc)?

It seems like it would almost be easier to hire some of rockstar's game designers and build a realistic self driving car environment from the ground up, given the amount of funding OpenAI has

Nah. This is second-system syndrome thinking. Definitely easier to modify the existing huge working system.

GTAV is closed source

This could have been done completely open source, but you wouldn't have gotten all the extras that GTA V gives you (basically the world and all of its objects, npcs, etc - which is a huge plus).

For instance, they could have gone with TORCS:

http://torcs.sourceforge.net/

...or Stunt Rally:

http://stuntrally.tuxfamily.org/

...or Trigger Rally:

http://trigger-rally.sourceforge.net/

(i'm not sure - but Stunt Rally and Trigger Rally may be from the same Trigger codebase)

Trigger Rally also has a webgl version:

https://github.com/CodeArtemis/TriggerRally

All of this is just to say that if it were wanted, a completely open-source driving AI simulator could be easily made.

Next Step: Build Street View map of GTA V.

I know this is totally unrelated, but i'd love to see a ML framework have a go at NetHack!

This is super intriguing! Didn't have GTA V for PC but now I do!

I'm not sure how anti cheat methods will work now.

looks like the blog post got deleted? why???? Sad face :(

This is actually pretty exciting!!!!

I live in the future.

What could go wrong

now the interesting part: put it in front of ethical problem (take one from http://moralmachine.mit.edu/) and lets see what it does

I'm probably one of the most critical people of the new AI-safety craze, but even I wonder if training AI on games like GTA is a good thing to do... if it is something I'd be hesitant to let my kids play, should we really be training our mechanical offspring on it?

Surely you're not that naive.

What is naïve? OpenAI is not a domain specific company working on some limited application like self driving cars. They are interested in developing general intelligence. Unlike a racing game, GTA has non player characters (agents) that can be manipulated to accomplish goals. But the lessons learned in doing so are not things I would want to generalize or carry forward into future experiments.

E.g. Maybe it learns that pedestrians made braking instruments in a tight turn.

It is not playing the game, it is using the game as a driving simulator.

It's using the game as a driving simulator by playing it.

Reply on news.ycombinator.com