The amount of construction needed for that kind of project that would only benefit a small number of people... It's just absurd. Rural people would rather that money go into more important projects.
And did it seem absurd 100 years ago to expect a nation-wide system of 55mph highways?
I'm pretty sure that it did seem absurd, and yet we're all better off for it in the end. Transportation infrastructure has a huge long tail of benefits.