If bandwidth cost is a concern the maybe you should reconsider how you publish your site.
Like, what if you actually post something that gains traction, is it going to bankrupt you or something?
Like, what if you actually post something that gains traction, is it going to bankrupt you or something?
It's not just some light bump in traffic. It's a headache that shouldn't need to be dealt with if they would respect ROBOTS.txt. Quite simple really.