I wonder how long before/whether a major player will simply decides google et al freeloading off their hard-produced data and displaying it in search results makes being listed a net loss and goes for a social-only model of traffic sourcing with a block-all robots.txt Between Google pulling out prices and reviews & Google News auto-generating summaries and snippets of on-the-ground reporters it feels like a matter of time. Maybe a halfway house would be starting to de-index hot news/exclusives for the first few hours.