The difference is that folks had an incentive to make their pages easily indexable: drive page views.

With becoming training data the incentive to the creators is a little less clear