I know that a few thousand years still feels like a long time, but with nuclear fission, it is truly the most non-renewable resource, being formed only in stellar explosions. Let's put aside the science fiction of asteroid mining nuclear fuel for now; I think a reasonable argument could be made to not spend nuclear fuel for our primary energy needs. If we have dreams of ever being able to do mass interstellar travel, nuclear material is the only fuel we have so far with reasonable energy density for the speeds and distances involved. (Of course some will say "nuclear fusion" or "light sails", to which I say, we should defer mass consumption of fissile material until we have a reasonable expectation that those methods are viable).