How is that compared to mining for battery and solar material? It's a double standard to hold nuclear accountable for manufacturing when the same is not applied to solar/wind.
Alas, if we really compare to solar, then keep in mind that for a solar cell you can use 100% of the refined silicon, whereas with uranium you end up throwing away a large fraction (over 95%) with low enriched fuel, and an even larger fraction (over 99%) with high enriched fuel.
Add to that that uranium is not among the most abundant elements on Earth (about 1ppm), whereas silicon is the second most abundant element in the Earth's crust (about 27%). That alone gives silicon a huge advantage in energy/chemical impact on the environment compared to uranium.
Oh, and maybe I should point out that the chemistry to work with uranium is also a lot more nastier than with silicon. Uranium is a heavy metal, so it all happens through complexes and acidic chemistry, which limits the options on chemical pathways. Silicon OTOH is very similar to carbon in its chemistry, so there are vastly more options to process silicon, and that alone allows for far more efficient processes.
For example, solar panel production requires a lot of electricity. That electricity is mostly generated from fossil fuels. But if you supplied that electricity with solar panels instead, it would be way cleaner. Which is correct? We should probably present both numbers, if possible.
It starts to sounds complicated, and to be a lot of information to digest for a decision, but another way of looking at it is that correctly assessing and planning for energy needs in the future is so important that ignoring information like that when making an assessment is irresponsible. We need more nytimes.com style widgets that allow you to tweak the values to easily digest data like this, and that clearly reference where the data and assumptions come from.