Hacker News Reader

I think at least some of the structure is down to the update equation, roughly 'new = sign(gate * old + ...)'. Since 'gate' is always positive, this means that even when the network is randomly initialized, the state is more likely to stay the same between two time steps than to change.

I tried "deep transitions" which repeats this equation a couple of times, and the results were indeed noisier.

It's mostly that these are very narrow and shallow networks -- if I remember right, 5 layers and 30 units wide?

Not OP, but I imagine you can set maximum limits on the rate change of the RGB vector so it doesn't create a static but a more fluidic appearing image.