You could, of course, run the bus in a dedicated, separate right-of-way to solve that problem. But like you said, just build rail at that point
That's just nonsense or specific to some areas. I spent a couple of hours on the Stagecoach buses today travelling from Paignton to Kinsgwear and then Dartmouth to Totnes and back to Paignton. The same journey by car would probably have been more expensive because of the cost of parking the car in busy tourist towns and it wouldn't have been much quicker. Some of the roads are barely wide enough for two cars to pass each other yet the buses still keep to their timetable.