Urgh! They already ignore robots.txt.
This is because we need to write some weird sh!t on all our pages. Now they tell us it doesn't even matter if we do...
This is because we need to write some weird sh!t on all our pages. Now they tell us it doesn't even matter if we do...
If you want a page out of the index, you must allow crawling in robots.txt and use noindex on the page.
It used to be that you could use robots.txt to stop indexing but google changed the rules a whole back.
Nofollow links were stated as "My link to this page should not benefit that page's rank", but apparently some people found a loophole: "I can exclude pages from Google's index of my site by nofollow'ing all links to them".
Loophole closed.