> Specifically: why do so many people ban web crawlers other than the "large" ones, and then wonder why the quality of other search engines are lacking?

Not to be difficult but is this a real problem? Do a lot of sites ban DDG's crawler and then complain about the low quality experience? Can you name a single example?