An EV essentially is a car with a very expensive fuel tank (the battery), made out of specific materials and properties. While it is efficient in energy use and can make use of fungible electricity, lithium is rare, limited to specific areas of concentration, and can (as any material) only be partially recycled.
There's lithium elsewhere on Earth, or more precisely, in seawater, but processing seawater for minerals (uranium is another which might be extracted) is expensive in the sense of requiring a lot of energy (arguably the ultimate definition of cost is energy requirements).
At US rates of auto ownership scaled to the world and future populations, known lithium reserves would quickly be exhausted -- a few decades, if that. With recycling, yes, some of that would be re-used, but even at 90% material recoverability (and references I have suggest 30% is far more common), you'll lose ~50% of your original material in 7 generations. (The formula: remaining material = (portion recoverable)^generations. So: 0.47 ~= 0.9^7.)
Other options include other forms of energy storage (including possibly synfuels), other battery components, or we all just start walking a lot more.