Why is there a juxtaposition between what people want and between what people are willing to give?
Specifically: why do so many people ban web crawlers other than the "large" ones, and then wonder why the quality of other search engines are lacking?
Not to be difficult but is this a real problem? Do a lot of sites ban DDG's crawler and then complain about the low quality experience? Can you name a single example?
edit: OK, I was wrong - https://duckduckgo.com/duckduckbot
Maybe StartPage still displays Google's ads? They display some ads.
Never heard of this. Third party crawlers usually get banned because they are poorly made (get stuck) or disobey robots.txt.