I think what's actually happening is that the game only counts your swipe as an input once you lift your finger off the touch screen.

You can kind of get away with that if you're implementing, like, 2048 (although the official version at play2048.co doesn't do this), but it feels way too unresponsive to be usable in an action game.