Now, if on top of the solar energy, we assuem that we will have fusion energy available within a few 100 years, energy available may easily grow way more than 100x of today's production.
Also, if efficiency continues to increase, for instance due to AI development, nanotech, biotech, etc, we may be able to build almost any amount of infrastructure given the atoms we have available in the Earth's crust. Including multi-story farms, feeding electric lighting to power photosynthesis.
In fact, I'm not aware of any hard limits anywhere near this side of 1 billion people. As far as I know, there are billions of billions of kg of most critical elements available on earth, or in the order of 1+ billion kg per person, even if we're 1 billion people. Our ability to organize these atoms into whatever combination we want (including human bodies) is primarily limited by energy.
And even with 1 billion people, it will take a while before we run out of hydrogen for fusion power.