Heh, if only Yandex and Baidu respected robots.txt.
Not tested. :)
THIS is evil. You could use this argument for banning any new search engine.
This is the reason they're blocked, not because they're new or non-English.
This site coolshell.cn is boycotting baidu, so it tells Baiduspider that it doesn't want to be indexed by baidu.
Baidu respects this and doesn't index anything from this website.
http://help.yandex.com/webmaster/controlling-robot/robots-tx...
Yandex's online tool to check specific URL on allowness in robots.txt: