Looks like the internal state of the numbers that actually decide what can be merged is changed when you press a key. Also works if you hold it down, which is a bug I think.

Just hold down one direction to observe.