Or a company might decide to try to solve the problem by gathering and indexing all of these documents so they are more accessible than if they were spattered over thousands of locations, unsearchable, and cost time and money to find.
I would also like to be given an example of something that you found hard to find via normal means but was easy to find via one of these document repositories. I am still wildly unconvinced of your general claim. Google is pretty good at the whole text indexing thing.
Do you use Google video search before you use Youtube?
Also, if you use sitemaps correctly Google will update your content in its main index pretty quickly.