Learning Machine Learning for Game Development

I'd be curious to see ML take this problem. I imagine the examples are limitless, but one that comes to mind might be procedural worlds. Rather than tuning procedural worlds around seed values, what would it look like if we try to get an ML to craft the worlds? Procedurally generated worlds often have that feeling. Things don't make sense, etc. I imagine ML could make some far more diverse patterns that are both interesting and fun.

I'd like to see this pattern of "informed RNG" in a lot of things. Skyrim with NPC patterns to feel a bit more authentic. Etc.

I'm less interested in the classic AI dreams inside games, as it seems far out and less gamey. But our current games with informed RNG? That sounds.. neat.

This means that rather focusing on what is merely broken one is focusing on both ends, what is broken and what is useless, attempting to push both towards an average of usability.

Personally it astonishes me that groups like WOTC (who produce MTG) don't appear to be using such models given the quantity of mistakes they seem to continue to make in game design.

In Magic: the Gathering this balancing act is achieved by a set of constantly changing "environments" (sets of cards) that are updated every few months, when a new set of cards is published. Sets "rotate" in and out of the various environments and there's a few where it's legal to play with all cards ever printed (excluding some that are banned or restricted for being OP- Over Powered).

When a new set rotates (its cards become legal) in a given environment, there will be a reshuffle of the balance of power between already-established decks, that now acquire new cards or lose cards they needed, and some new strategies become available resulting in new decks being designed. Eventually the dust settles and the "tier one" decks (the ones that win the most) are found. Then tournament play in particular becomes a rock-paper-scissors affair, until the next update.

In some of the environments were all cards from all sets are legal (again, minus the ones in the banned or restricted lists) games indeed often come down to a coin toss: the player who goes first wins (in one turn, through some OP combo, typically).

I suspect that a perfectly balanced game is impossible to have and would probably be boring even if it was possible to have it. An element of creativity with constantly updated design parts, like in M:tG is probably the best one can do.

It means that the AI that generated the data has 30% chance of winning, or at least that's what it's trying to estimate.

The neural net is a well-researched cnn variant - the activations are quite interpretable these days.

The metric’s interpretability is independent from the neural net used for gameplay, so the black box comment doesn’t make sense (unless I’m misunderstanding the comment...).

The metric is also very human friendly: think of it as 30% of players in the given position, at the given game’s stage and state, successfully continued on to a victory.

Dota added a win prediction graph a while back that's been valuable to look at. It's not really a black box, because you can see the win % change over time and get a feel for what it's weighting. For example, it takes character selection into account, and the initial prediction can get to 60/40 before the game even starts.

The bit that I found the most interesting is when the prediction doesn't line up with my intuition of the game state. I've had a couple of comebacks from a predicted 1% win chance, and all those felt more like 20-30% comebacks, where we were down but definitely not out. A map is not the territory, and that's easiest to see when you fall off the edge of the map.

This is fairly common but more simulation replaying in networked games with fake bots based on that gameplay. Previously I have done similar in networked games where the way points and reactions are reused in AI agents that mimic players. The networked players that we captured, if another player dropped, we replayed captured ghost plays essentially with more interactivity that seem more real than just AI. This allowed us to have networking game experience of matchup, that can drop a player, but seemingly did not but still have that player act human not just bot-like. This is now used in all our matchup/networking games because it makes the matchup/networked experience better for the player even if they end up playing a sim and not knowing it.

Taking that further if you could train the machine learning models from the existing player base for an additional level of focus it could really make the game fun and you could use it to even drive player behavior.

Machine learning definitely has a place in game testing and progression tuning. However, even more it would be nice if it can learn on the fly and not end up in predictable AI, one possible way would be capturing actual human player behavior and modeling that.

The GIF above "An example game state representation used to train the neural network" makes it seem like it's the latter.

Some of the variables have an unexpected representation. For example to represent "chimera health" with value H, it seems like they just use the 3 rows at the top, with the first H pixels "on" (green) and the remaining "off" (black). Same thing for "Chimera Power" and "Link Energy".

I guess this might make sense since they are using a CNN, but I wonder why that works better than using a different architecture and passing each of these values as a single input.