Echoing your sentiments, it's worth noting that we've already launched nuclear power generators on a number of NASA missions. Not the usual type of fission reactor, but radioactive material nonetheless. So the risk certainly hasn't been considered intolerable so far.

https://en.wikipedia.org/wiki/Radioisotope_thermoelectric_ge...