My understanding is that in north America the big problem with passenger rail is less cost and more the fact that there's a bias towards freight in track right of ways. Even if you're willing and happy to pay a lot to travel by train, scheduling is still likely to be screwed up somewhere by a big ass freight train in the way and there's nothing you can do about it.
Edit to add after a double check: I guess in the US it's technically the law that passengers be given priority but it's poorly (or just not) enforced[1].
[1] https://www.amtrak.com/content/dam/projects/dotcom/english/p...