Mario's speed was limited via "speed < MAX_SPEED", but it didn't have any boundary on negative speed. So you can build up effectively infinite negative speed by chaining long-jumps backward, without turning Mario around. Get enough (negative) speed, and the boundary checks for walls get bypassed, because you move all the way through them in one frame.

You can combine that with another trick: chaining long jumps works as long as you're close enough to the ground to kick off the ground, so if you hit the jump button rapidly enough, you'll jump repeatedly without leaving the ground. Notice in the linked video that you'll hear many jumping sounds rapidly, but occasionally the player will miss the timing and do a full-height jump.

That same trick also works to bypass the "infinite" staircase without having 70 stars (because you move past the looping portion in one frame).

Human players can successfully use this to warp through walls and bypass star/key requirements, to complete the game with 50, 16, 1, or 0 stars. Tool-assisted speedruns also use this to move rapidly through levels, but that requires a series of frame-perfect inputs.

My assumption was some sort of flaw or limitation in hit detection but it would be awesome if someone who actually worked on it or similar could comment further :)