This video (https://www.youtube.com/watch?v=W4tGaoSz14g) at 3:20 has a good summary of how the concept works in cricket. With a ball that is equally worn on both sides it is the seam (like the latex band) breaking up the air flow that causes the ball to curve the wrong way ('reverse swing' in cricket).

edit: and the position of the seam is obviously important in directing the force, which is why you will see this more often in football free kicks when the player can position the ball before taking the shot.