Good performance matters if you have decent networking infrastructure or your server has limited resources.

Bandwidth and IP limits are the most common bottle necks, but these can be solved using multiple proxies and ssh tunnels. Colly has built in support for switching proxies [1].

[1] http://go-colly.org/docs/best_practices/distributed/

Tell that to the project I migrated from scrapy to go six months back. Granted, scrapy might be doing other "fun" things to eat into performance, but it was really night and day. Immediately went from CPU bottleneck to network.

It has always been scheduling and anti-blocking