This turns out not to be the history of the game designers' creation of the level, but rather of speedrunners' efforts to squeeze milliseconds out of their time to reach the Warp Zone. A pretty remarkable story of dedication and obsession.
The way some runners are slicing ms off is reverse-engineering their creation: that is some obsession.