In the long term when the space transport infrastructure is as well developed as the terrestrial one, I can see the benefit of having different systems for different purposes, so you get the specialized benefits. But I'm not sure we are there yet.
To use an analogy from history, we aren't at the Golden Spike moment yet where California was sufficiently settled to warrant a continental rail road connecting the East and West Coasts. We're still in the exploration and settlement stage, and while trains may be more energy efficient, what we need is a Conestoga Wagon that can be operated and maintained by the settlers themselves while they live off the land and develop the first homesteads.
This is roughly the difference between an 1881 steamliner from London to Syndey, and a 747.
https://www.nasa.gov/centers/marshall/news/news/releases/201...
I’d guess a 10%-30% chance SpaceX is already Skunkworks-ing some form of atomic rocket, either fusion or fission.
Edit: and yes, that estimate is despite the public comments.