Lithium and other metals will be recycled and it is trivial to do so. Currently the volume is too low: except for manufacturing/design defects, no automotive lithium battery has reached EOL yet.
That doesn’t really matter much though. Uranium isn’t dangerous by itself, and has the highest energy density of any existing source.
Well.. it's not a huge deal but don't eat it. I have a bit of uranium ore on my bookshelf, but I avoid handling it and wash my hands after I do. Besides the alpha radiation, which isn't good for your insides, it's also toxic.