Each of these problems needs to be considered on its own merits, not lumped together in some sort of misleading generalization.
I was worried about the phosphate thing for a while since, unlike resources like copper and rare earth metals, the used phosphate mostly ends up in the oceans, not conveniently concentrated in landfills. Getting it back out of the oceans is pretty difficult.
But then I looked into it a few years back. It turns out that the absolute amount of phosphate in crustal phosphate rocks is staggeringly huge: about 0.1% of the crust is phosphorus, almost all as phosphate, conventionally measured as P₂O₅, which would thus be 142/62 of that, or about 0.2%. This works out to be on the order of 10¹⁹ kg of phosphate. World phosphate production is 53 million tonnes (https://www.fao.org/3/i6895e/i6895e.pdf), less than 10¹¹ kg, so at current consumption rates we will run out of phosphate in about 10 million years.
Given this, you might wonder why "phosphate reserves" are stated as only a few tens of billions of tonnes! The answer is that currently most phosphate minerals are unprofitable to mine because they can't compete on price with the fairly pure concentrated apatite deposits in the US, Morocco, and China. When those deposits are exhausted (probably sometime in the next few centuries), the price of phosphate will rise, unless we're mining it from the Moon or the asteroids by then. That will make it profitable to extract phosphate from less concentrated resources.
So, while it is technically true that phosphate rock is a "nonrenewable resource", the implication that we will run out before the Pyramids have crumbled into dust is not.
The bigger problem with phosphate mining is phosphogypsum, not energy consumption or running out of phosphate minerals. Detoxifying phosphogypsum is not a difficult chemistry problem, but there is a lot of it and nobody is paying to detoxify it, so it sits in waste dumps. I wouldn't be surprised if we saw a red-mud-style industrial disaster or two with phosphogypsum in the next few decades.