There's an inconvenient fact that Google prefers to ignore:

-- Frequently, the headline and snippet is all you need. "Inflation still at 3%" is often enough for a lot of readers. An awful lot of them aren't going to follow the link.

Yes, they are trying to preserve a business model that doesn't work, but that doesn't mean they have to provide free services to any and all. If Google just had a link labelled "News on Inflation" the paper would probably be happy.

"You can just use robots.txt if you don't want to be crawled" is a canard. You can just not have a website, too.