> Average is the worst case, as everyone has to deal with it.
That's true if:
(1) No one in the worst case receives a kind of damage that is not compensable, and
(2) You have systems in place so that all the people who experience better than average results compensate all those who receive worse than average results.
> The chance of anything going wrong with a modern nuclear facility is astronomically low.
The chance of anything going wrong in a modern design, implemented without cutting corners, and operated without cutting corners is, yes.
Whether you trust the poltiico-economic system in your country (and that of whoever is providing design/build support, for countries where that isn't purely domestic) is an open question.
Social factors are part of the risk calculus. Nuclear disasters have generally not been unavoidable consequences of the technical knowledge at the time they were constructed and operated occuring at their irreducible rate, they've been symptoms of failures of the social systems involved in planning the deployment and use of the technology. You can't assess the real risk of nuclear from the risk posed by the technology when properly applied in an ideal social context alone.