Google is exploiting copyrighted content to make profit and create audience.
If the search index of Google was empty, the people wouldn't use Google, it's that simple.
So the newspapers are asking for royalties for feeding that search index.
Granted, I have `disallow: /` in mine because I don't want my stuff scraped, but I still see Googlebot sometimes, and others, try to crawl my site.
It's not a very effective opt-out, because it requires the 'attacker' to honor the file's settings.
Feel free to enable it on your own server and watch the logs for a few days after sharing some links.
I hope daddy government treats you well.
Count on Big Brother to save you, and they will beat you down when it's time as well.
Noting that something exists and including it in reference material should by no means incur royalties.