>> Google can arbitrarily and capriciously exclude them from their index.

As a web developer on SEO forums, I hear of these cases all the time, but when you analyze these sites, usually 1 of 3 things is happening:

- Web developer(s) made a mistake, causing a (search engine) accessibility problem.

- The site is in violation of the Google webmaster guidelines.

- The site lacks (unique) content, or otherwise doesn't contribute at all to a healthy search engine index.

I never read of a case of Google arbitrarily or capriciously excluding sites from their index, offering them no way to appeal. In general, I also think the advice Google gives is far from inconsistent.

This could be a popular position for a web developer who got a site de-indexed, but maybe apply Occam's razor first. A mistake? A trend? An algo change? Worthless content? Blackhat SEO? Bad architecture? Got hacked?

Or do you want to jump immediately to Google arbitrarily removing well-intentioned sites from their index? I guess then you can blame bad luck of the draw.