It's not that much better with Throrium either.
Nuclear is oddly politicized, both pro- and anti-, whereas I think the truth is somewhere in the middle. It's not the worst power source but it's also not the best. It's not worth continuing to invest in for the future because better sources are already coming online that don't have the associated fallout risk, hazardous materials disposal issue, and intensive mining/refining processes.
For example, solar panel production requires a lot of electricity. That electricity is mostly generated from fossil fuels. But if you supplied that electricity with solar panels instead, it would be way cleaner. Which is correct? We should probably present both numbers, if possible.
It starts to sounds complicated, and to be a lot of information to digest for a decision, but another way of looking at it is that correctly assessing and planning for energy needs in the future is so important that ignoring information like that when making an assessment is irresponsible. We need more nytimes.com style widgets that allow you to tweak the values to easily digest data like this, and that clearly reference where the data and assumptions come from.
Alas, if we really compare to solar, then keep in mind that for a solar cell you can use 100% of the refined silicon, whereas with uranium you end up throwing away a large fraction (over 95%) with low enriched fuel, and an even larger fraction (over 99%) with high enriched fuel.
Add to that that uranium is not among the most abundant elements on Earth (about 1ppm), whereas silicon is the second most abundant element in the Earth's crust (about 27%). That alone gives silicon a huge advantage in energy/chemical impact on the environment compared to uranium.
Oh, and maybe I should point out that the chemistry to work with uranium is also a lot more nastier than with silicon. Uranium is a heavy metal, so it all happens through complexes and acidic chemistry, which limits the options on chemical pathways. Silicon OTOH is very similar to carbon in its chemistry, so there are vastly more options to process silicon, and that alone allows for far more efficient processes.
How is that even a problem? Nuclear power plants do not require much fissionable material at all. Compare that with any fossil fueled power plant.
For making solar cells you don't have to be picky in which isotope you make them from.
Also there's only about 1ppm of uranium in the Earth's crust, whereas silicon is the second most abundant material (27%). So you don't even have to spend a that much energy just to separate the non-silicon stuff from the silicon-stuff, whereas a huge amount of energy in uranium production is preoccupied with doing just that.
If you go outside and take any rock, you're holding in your hands mostly silicon and oxygen. Strip away the oxygen and you get pure silicon.