I don’t think there is any danger of hordes of people starting to live in vans. Try not to let such concerns stress you out too much.
In my case there's a popular area for climbers on a fire road near a river close to me. Many of these folks just shit arbitrarily in the woods, and don't even bury it. It stinks, attracts animals, and makes it impossible to go on hikes. A friend's dog even ate some which had THC in it from someone who ate a marijuana edible. And let's not get started about the risk of forest fires in the dry season.
There are some that are better about managing their waste and overall footprint, but there is an outsized impact from the less considerate.
There are places where these concerns are real and are having actual and increasing impact. Dismissing these concerns is not helpful.