Hacker News Reader

Comment by discardorama | Hacker News Reader

So lets say I take the trained network, and flip the color of the pixels, and make some other cosmetic changes (keeping the game intact). Will the network then solve it in the first try?

View on HN

The ALE agents are almost always using grey-scale pixels in the first place (to save on processing and NN space requirements), so flipping the color makes no difference.

Is it learning the specific level layout or can it adapt when the screens are different and order swapped?

DQN is stateless, so at every point it's reacting to just the current frame (technically, the average of the last 4 frames IIRC) being fed into a CNN and outputing a motion. It doesn't care about ordering all that much because it has no memory. In this case, it's probably learning stuff like 'if there's a key, go towards it' and 'if you have a key, go towards a barrier'. If each screen keeps working the same way, then I guess it would achieve about the same performance as it does now. New screens with new layouts/patterns of enemies will test how well its learned heuristics generalize, though.

Personally, I think that someone should be trying a DQN with an RNN rather than CNN in it to see if that helps on the harder levels. Or better yet, combine it with some of the memory mechanisms and see if it can start doing some real long-term planning.

I believe the input state is the past 4 frames stacked or concatenated.

I think in general these types of neural networks are flexible enough to be resistant to trivial changes like this- Whether that specific change for this specific NN would affect results would likely need to be checked empirically.

Reply on news.ycombinator.com