I don't think anyone thinks it's a particularly practical idea these days - for the reasons you identify.

However, it seems a bit of a shame that for all of the 2000+ nuclear weapons tests and tens of thousands of nuclear weapons built we never did anything as constructive as flying an Orion.