If there was true HS service on the NEC between Boston and NYC you'd easily get a far larger share of the BOS-NYC pax trips made. Estimates are about 15MM trips/yr with rail being about a third of that. Is getting an extra 5 million car trips off the road worth inconveniencing some of the most affluent communities in the US?
BOS NYC, driven is about 2 micromorts, the drive is about 140kg CO2. So, napkin math, if half the people traveling by car and plane switched to train, you save 5 lives per year and 500,000 tons of CO2 emission. That's a (shittily calculated, admittedly) estimate of the cost of inaction.
Right of way seems to be the biggest driver of cost and seems to be a major blocker for transportation progress.