It's not hard to see 100 bushel per acre swings in corn yield between growing years even within normal growing parameters, never mind times of crop catastrophes. While everyone hopes for a good year every year, you can't plan for a good year. As such, there will always be times where there is way more corn than we can handle.
Which is what happened in 2006. A lot of corn, having busted the bins and having no home, was left out to rot. Ethanol subsidies were introduced in 2007 to build out more ethanol plants to provide a greater buffer in excess times when the regular corn market can't absorb it all so that we aren't both burning gasoline and letting the corn rot away to nothing.
I don't recall ethanol ever being considered better for the climate in a vacuum. It was only ever thought to be when you take reality into account.