Another reason is that the freight companies own a vast majority of the rail tracks in the US which means freight trains get priority over passenger trains..
This only happened after the airlines stole most of the passenger traffic. By the 70s the rail vs flight curves had crossed and most passengers were on planes, after deregulation of the airlines the market for long-haul passenger carriage was owned by the airlines; the only thing left for railroads was freight and short-haul passenger traffic in populated corridors. When passengers were the bulk of the rail revenue stream the system was optimized in their favor, and when fright took over the conditions were inverted.
The airlines did not "steal" the passenger traffic... they offered something people wanted... a journey lasting hours instead of days.
I did not mean to imply that rail had any "right" to the passengers, just being a bit informal in the description. Faster transit was definitely the initial draw, but the implied status and luxury that early air travel provided was also important. The speed difference at the time was significant, but not as large as it is today -- NYC to Chicago was an overnight trip and you arrived blocks from the loop and in much better shape than you do today after enduring a similar trip.