Curious to understand how AI will continue to grow if this is the trend. Assuming most valuable data is behind such firewalls. And whatever is public has been harvested, trained on top of whatever has been acquired illegally (this is a grey area).
Will it become a closed ecosystem without outside input?!
There probably is a point of “peak data” where the amount of new data will start decreasing, but that’s likely a 22nd or 24rd century problem.
Unless we’re producing data on the order of an entire new internet every couple of years, then it’s hard to see how LLMs can achieve further huge leaps in capability compared to training on effectively 0% of the internet vs 100% of the internet.
And individuals will loose motivation to share, because it wont be that pro-social activity anymore anyway.