This idea is always met with extreme incredulousness about it because it violates their intuition. "You have to go as fast as possible or it won't compete well with air travel! If I have to stop in Merced for 5 minutes to let a handful of people on or off, I'd rather just fly".
The problem is that those handfuls of people, the result of the potential travel between the cartesian product of dozens of cities, adds up. There are number of trade publications that have analyzed passenger patterns on successful systems and have found that the ridership exclusively between major terminal stations typically accounted for ~50% of total ridership, and less than 50% of total revenue. Smaller hops have higher prices and better market penetration because they aren't served by air travel and therefore have no competition. Larger hops between major cities have lower prices simply because they have to compete with air travel. Even in a "perfect" world with non-stop trains, HSR is still going to lose some traffic to air travel, and they won't have all of that massive supplementary revenue from intermediate destination travel.
Unfortunately, the idea that we should build a system that isn't optimized for them just doesn't resonate with people. And since high speed rail is now a partisan issue, and those same parties have a sharp urban rural divide, we get two camps: 1) democratic voters that want high speed rail, but lose interest when experts say that there need to be lots of intermediate stops, and 2) rural voters who don't want high speed rail, even though they would benefit from it greatly, because "socialism". And because of it, I believe American HSR is doomed. It will either get built and lose massive amounts of money, or it won't get built at all.