In order to get approval to launch nuclear material, the launch vehicle must be nuclear-rated (like human-rating, it ensures safety). With the recent retirement of the Delta II, only the Atlas V is currently nuclear-rated, but eventually others may be. Additionally, the fuel is encased in boxes designed to withstand launch failure and impact into the ocean without releasing the contents.

In the case of RTGs, the fuel is always releasing heat and is always radioactive. For fission reactors, the fuel is relatively inert and non-radioactive until it is turned on.

The casing works pretty well. I believe there's even an instance where there WAS a launch failure with an RTG (Atlas V has never had a legitimate launch failure and it has flown almost 100 times), and they fished it out and reused the fuel for another mission.

Many reactors are envisioned to be launched by chemical means and then turned on remotely. Before a nuclear reactor turns on and starts splitting atoms, you can hold its fuel in your hand no problem.

Not necessarily a problem. You don’t have to launch it with the fuel rods in, if it would be safer to store the fuel rods a different way for launch. Ultimately you do need high levels of safety, but only about the same level that’s needed for ships at sea, since many of them have nuclear reactors.

This, basically: https://en.wikipedia.org/wiki/Kosmos_954

In this case, it was reentry of a reactor that had been running for a few months. If a rocket carrying a reactor blew up, it would probably cause less damage since the reactor would have fewer (if any?) hours on it.