Incognito no more: Publishers close loopholes as paywall blockers emerge | Hacker News Reader

Incognito no more: Publishers close loopholes as paywall blockers emerge

digiday.com

55 pointsjoegahona88 comments

Wait, what? Is content shadowing now ok with Google?

I always thought that content presented to Google bot should be very very close to what user see when they click link. And if not, Google penalize website hard. Has that changed?

I mean, if rule still stands, than news websites will show up in search results with keywords "subscription" and "article". Instead of "USA" and "brexit".

If content shadowing is ok now, that gives spammers a free ride.

> Wait, what? Is content shadowing now ok with Google?

It goes by a different name, but yes - the policy ended according to the article:

> Google’s ending of first-click free now means that subscription publishers can still appear in Google search results without offering access.

Kind of a shame. I remember my frustrations in the mid-2000's of search results riddled with this dark pattern in the form of ExpertsExchange and Quora results.

Kind of an aside, but on ExpertsExchange you simply had to scroll down past the blurred div to see the full content.

That was their adaptation to Google's rules. Initially answers were behind a paywall.

This hasn't been the case for years if you're a big name. Small fry lives by different rules than the Washington Post or the San Francisco Chronicle.

Springer (the science publisher, not the German trash media company) has done that since at least 10 years. They'd send the full text PDF to Google and the abstract & buy option as HTML to users. Google doesn't care.

I think you need to think through objectives and motivations rather than approach policies as if they were if-then-else conditions.

Objective of Google is to maximize their search revenue. The biggest driver of revenue is users. More users use Google search more often if it shows them relevant high quality content more often and they're satisfied after clicking through and reading.

Big name famous publishers usually produce high quality content. Try want traffic and they also want to monetize via subscriptions. So naturally there's a win win for Google to show high quality results in spite of paywall and find a way to work together.

I’m not sure it’s a win-win though. Most users don’t want to pay for anything. If they get used to not being able to access major news search results they’ll simply stop searching for news or at least not click on the publishers they remember having a paywall.

It is a constant annoyance to me when my mobile google app suggests "interesting stories" to me which end up being paywall locked.

Google's just trying to help their customers succeed.

Maybe google doesn’t like their search market dominance

Tangentially related, but I find it pretty frustrating that websites can detect private browsing to begin with, when all I really wanted is to not have a browsing session leave a trace on my phone/computer when I'm done.

It doesn't help that on iOS, deleting browser data is 4 menus deep, and there's a bug that keeps surfacing allows this to happen: https://video.twimg.com/ext_tw_video/1128887578541797376/pu/...

Worse, deleting browser data turns off private browsing if you previously had it enabled.

I had no idea that this was the case; thank you for highlighting it. I also have no idea what would posses the Safari team to think settings I have enabled aren't distinct from data foisted onto me by websites I visit but that is a question for keener intellects than mine.

I wish mobile browsing was as good as Desktop. It's such a shame that various restrictions don't afford the granularity of laptop browsing.

I guess for test on iOS would be to see if you could store something to localStorage as that doesn't work in safari would be one way to tell of your in incognito mode.

It would be nice if there were an app like CookieAutoDelete [0] for mobile Safari (or desktop Safari if Mozilla keep misbehaving!). You can disable cookies but that's trivially detectable. I guess it's trivial to detect, but it needn't be; something akin to Mozilla's Containers [1] on mobile Safari would be a step in the right direction.

[0] - https://addons.mozilla.org/en-US/firefox/addon/cookie-autode...

[1] - https://addons.mozilla.org/en-US/firefox/addon/multi-account...

The only way to have free content is if the user is the product being sold. That's detrimental to both the user and to the society that they live in if it affects how they make decisions. (Which is an integral part of news)

I simply don't understand how anyone could be frustrated with a publication making it harder to get content for free. It's what we must accept if we don't want the government to provide the news.

It's a foolish strategy for mass media but makes sense for topic focused publications that usually are not mainstream or elite publication that try to exclude.

You were always the product. Newspapers, magazines, year books, church bulletins have ads directed at you or a group you are part of. Certain people buy a publication for the ads.

Free isn't the problem. Taking free away means less people read their paper, less talk about it, less get invited on cnn, less word of mouth. The need to move to clickbait titles increases.

Remember these media sites do not create the news but present it in a digestable format and provide commentary.

Media sites behaving foolish and dying faster imo is a net win for the average folk, so all in all this is good news.

This is because of the modern idea that we should be most interested in ourselves and what profit we can generate rather than what is good for our community (i.e. open, independent and shared journalism).

Also, if the user is not to be the product, then the company must demonstrate the value of the product that we should be paying for. If the perceived value is not there, why would anyone want to pay?

the point is not free Vs paid, the point is the bait and switch of presenting the content as free to crawlers and then wall it to users

that's the annoying part.

We had lots of free content during the Internet's early years; people simply made things and gave them away for the pleasure of sharing. Generosity is as much a part of human nature as greed.

There's nothing stopping people from continuing this either. There are myriad ways to promote your own creations.

In fact there was a recently promoted site called https://1mb.site/ which allowed a small free website. If you hotlink your images from your own https://imgur.com account, you can have a relatively easy site.

> There's nothing stopping people from continuing this either.

Yes, there is. If you make it easier to monetize, people who would otherwise have shared their creations freely put them behind paywalls or force their visitors to subject themselves to advertisement.

I disagree. Technical ease has nothing to do with monetization and it's extremely easy to set up.

Technical ease of monetization has nothing to do with monetization?

Also, something technical can be extremely easy for the average HN reader and practically impossible for most people.

Do you realize how easy it is to set up a facebook page, wordpress or blogger blog? Tons of non-technical people have done and continue to do this. This isn't about your average HN reader but the general public and they're competent enough to do these things by themselves, which has already been proven.

I did not realize you can monetize a facebook page.

You seem to be stuck on the idea that it's about money, and it's not.

My entire argument isn't that it can be monetized easily (https://www.facebook.com/help/publisher/321041698514182 it can) but that there are dozens of EASY ways to publish things and do so freely.

It's literally dead simple.

Now you're claiming that monetization has nothing to do with money. Ridiculous.

If you make monetization hard, people will freely share their works. If you make it easy, they will monetize them. Monetizing is easier than it used to be, stopping people from sharing freely. That shouldn't be hard to understand.

No, go back and read my comments again. Stop debating something I didn't bring up and trying to push an irrelevant point of view.

anoncake presented a relevant related argument, that's kind of what people do on discussion sites.

No, they didn't. There is nothing preventing someone from easily sharing content in spite of how "easy" it is to monetize it.

There's also the Guardians way: Ask, but don't block. Despite all the cries that this could never work Guardian had a positive revenue in their last yearly report. It depends on people doing the right thing, if they can afford it and if not cross-finance it with the people who can.

> Guardian had a positive revenue in their last yearly report.

Positive revenue for the first time since 1998.

While admirable and noble, I find that this strategy is fraught with its own risks. Your company/product becomes fragile to financial headwinds in absence of a steady revenue.

Positive revenue, for the first time since 1998. I think there are more factors at play there than just the fact that they had an interstitial asking users to contribute or subscribe.

You can have free content as a complement to another business, such as selling a product or service, directly to users.

> The only way to have free content is if the user is the product being sold.

Or we could provide robust public funding, like the BBC

> It's what we must accept if we don't want the government to provide the news.

Public funding of media does not make it "government news". In fact, it's subject to far more scrutiny than private media. You would rather Jeff Bezos and friends control the news rather than a transparent & highly regulated independent publicly funded organization?

Public funding makes it "government news" in Germany. And very expensive at that.

Both of these sentences are factually wrong. There is no government news in Germany. And the Öffentlich-rechtlich (which program is, again, not controlled by the government) are not really expensive (The price is 17,50 for everything together: TV programs, radio programs, internet presence).

It is controlled by CDU and SPD who together form the government. Secondly afaik it is one of the best funded public broadcast worldwide.

Please stop spreading misinformation. The Federal Constitutional Court decided in 2014 that no more than a third of the members of each Rundfunkrat can be chosen by the state or by parties. Also, not all party representatives are representatives of a governing party. The ZDF Fernsehrat has representatives of each party present in the Bundestag. Two thirds belong to other groups such as churches and unions. For the state-level Rundfunkrats, the federal government is obviously irrelevant.

Name one Intendant (head of broadcaster) who is not member of a ruling party.

I don't take orders from you. But there are a few.

There is not one single way to publicly fund media. Hell you could just leave the whole system as is as provide no strings attached grants or tax credits to all publishers. The point is, there is no technical solution to this, we need policy change.

I came here to make the point that there was too much friction in subscribing and logins across devices. To prove it I checked a major publisher and - wow, ApplePay, TouchID, done. Five seconds and I'm a subscriber, $8 a month. And iCloud sent the password from my MBP to my iPhone, probably Google-users get the same or better.

I checked a second publisher and it has slightly more friction, asking for my name, but still makes payment much easier than before.

There was a time where digital was about upselling print, but to my surprise publishers seem to be doing things right. I just now got two subscriptions, it feels good to support journalism.

Washington Post and NYT are what the President of the United States says is fake news. At the same time, they're limiting the audience of their coverage to subscribers only beyond a few free articles a month. I wonder how ordinary people who can't/won't pay for subscriptions will receive their information to make up their minds after this retreat.

> I wonder how ordinary people who can't/won't pay for subscriptions will receive their information to make up their minds after this retreat.

Don’t we already know the answer to that? They’ll get their “news” from their “friends” on Facebook.

The way it was done historically? Pay for one or two publications which instead of arriving at your doorstep every morning will be delivered to your phone every morning. The people who couldn't or wouldn't pay for good journalism would not receive any - at that point you'd be deriving your information from discourse with other people.

Most people read title to make up their minds. Unfortunately.

A lot of freebie newspapers summarize the original stories.

Some ~large~ sites are still quite far behind tech wise in stopping free reading. What do these bigger papers use?

My national site just had a `class="premium-content"` that you needed to delete to bypass it.

They upgraded, and now all you have to do is take the `class="QUnWjUZnTonf"` for each paragraph, and remove it.

(e.g. https://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&object... )

I wonder who is in charge of these sorts of things, you would think that if you are going to block people off you would put more effort into things.

I've worked in a few NZ dev shops and rigor isn't in my experience, high on the agenda! I also had no idea NZ Herald even had a subscription model, so thank you.

It's good enough that for the average consumer, they'd never decide to look in to the source code, edit it and then reload the page with the changes. However this type of stuff just allows someone to easily come along with a counterforce app and scale the paywall.

Is there a way to remove publications from Google News sources? If I can't read the article for free, then I don't want to see their links ever.

"Sources and topics you see less of" in Settings can block sources.

You could use the uBlacklist browser extension.

nytimes routinely publishes filler articles on improving privacy, security etc. And yet, if you open nytimes in something like firefox focus/klar, it will block your view with a giant "You are in private mode; sign in" banner. Yeah, no shit nytimes. If I wanted to sign in, I wouldn't be in private mode.

To anyone who still wants to persist on mobile, firefox with ublock origin works. It runs like a dog though, partly because of firefox on mobile and partly because of nytimes' bloat.

This is also why amp isn't going away anywhere. Real news just fails on many levels. For a paid product, they really punish their users in so many ways.

Firefox Mobile with uBlock origin runs fast on my cellphone, faster than Chrome without adblock.

The problem and the solution look obvious: there are many news sources, everybody wants to get paid, people don’t want to pay for all of them, yet still not content from getting the news from a single source.

We have solved this problem for music and movies already, Spotify and Netflix are Silicon Valley poster children. Now we just need to build a news / text media aggregator along the same lines.

>there are many news sources, everybody wants to get paid, people don’t want to pay for all of them, yet still not content from getting the news from a single source.

Don't pay for all of them and BE CONTENT with getting news/information from a small set of outlets you want. Abundance of choice doesn't mean that you have to try out everything under the sun. This is consumer expectations being irrational after an era of free ad-sponsored internet articles. Given a median US income of 60k, most people within this group can definitely budget for 3-4 publications annually. Go lower and people can pay for 2-3 publications. Go lower and you probably don't have time for reading news and have more pressing matters in life. We are no longer primed to pay for content so even paying 10 USD a month is jarring experience for most.

The problem is less of the market and more of the consumer. We went from a lack of information to information overabundance with the advent of internet, google and free articles. Now we want to read/watch/listen to everything which is physically impossible. We think we want to read a couple of articles from every other site but most of us (HN/reddit may be vocal exceptions) end up reading a decent chunk of articles from a small number of sites. I am not pulling these numbers out of thin hair. I am building a much more powerful alternative to pocket/instapaper and these trends are coming from the first set of beta users (albeit small but for which I have no reason to believe also doesn't pan out at significant numbers) from finance, tech and other industries. Even I was surprised when I ran my own set of links and saw that 95% of them were coming from a set of 5-6 publishers at best whose combined annual subscriptions were north of just 200 USD - something I'm happy to and already do pay for.

It is very inconvenient to watch over these 5-6 individual subscriptions, especially when I am not interested in 90% of their content (but the content I _am_ interested in can come from anywhere).

It is not about money, it is about friction.

I'm not convinced the problem will remain solved for music, and video certainly isn't solved. Gone are the days when everything was on Netflix; now it's fragmented, split amongst the big players in the space. I can see Spotify losing their dominance just as easily.

There are issues, but it is still more convenient than it was before Netflix and Spotify. In fact, I don’t use Spotify, I use Apple Music (as I travel a lot and live in two countries, and they are less picky about that) — the same music is still available there.

> We have solved this problem for music and movies already

Even in the US, arguably the golden land when it comes to the library size of the services you talk about, people are frustrated about walled gardens forming in these services.

Companies producing content (or companies at all) want to "diversify", especially into things they already have some content for (as is the case for publishing companies). The next question is how to pull people to your platform and the answer is simple: make your content available nowhere else. Thus, another walled garden is born. This is the pattern you see in the wild now that streaming has become popular.

This pattern does not solve the aggregation problem at all, in fact it stands exactly in the way of it working.

Paying subscription for aggregated news sources (even if there are several major aggregators) is still infinitely more convenient than managing 5-6 individual subscriptions.

If you move the goalposts, then any argument will eventually converge on the target of sounding reasonable. You started out by claiming that unifying subscriptions was "a solved problem". I responded by saying that it clearly is not, and that the intent of the industry as it is now will converge on least convenience. Whether or not 4 subscriptions are more convenient than 40 is fundamentally immaterial, as that is not what we were discussing.

Incognito mode is still working fine for WaPo, in my hands (chromium/ubuntu 16.04.) There are people in the HN thread she links to substantiate the claim, which also report it's working fine for them.

This trend has been great for my productivity. Any time one of these comes up, I just close the tab. So far I haven't felt like I've missed out on anything important!

If you manage to find a trustworthy news source, support it with cash to make sure it sticks around. If they break your trust, revoke your cash (and be sure to let them know why). That's the only way things will improve.

It's an interesting development because the amount of people who actively seek to bypass paywalls must be negligible. While it's mostly second nature to a large swathe of the HN crowd, devtools/private browsing/Google-refe[r]rer plays aren't exactly typical browsing tricks for the average person. I can only extrapolate that it's either it's either a) a bid to be seen to be doing something to higher-ups or b) an attempt to garner subscriptions from the tech crowd with the nous to bypass a paywall ue to low numbers [0]. The latter seems unlikely on account of actual conversions and the former seems myopic. I'd love to be a fly on the wall at some of these meetings!

[0] - https://www.nytimes.com/2018/11/01/business/media/new-york-t...

News content should be bundled like streaming services. I am never going to pay $1 to each news site i read. I would pay $8 to read any news site.

Apple News kinda does this, and they have a Free and + version.

Next stage.. Piracy prevention as one copies articles and republished.. Every incapable Industrie from movies to music dies the same insane way.

Anyone know how to bypass the "Medium" pay wall?? Help!

Publishing is in a similar position to streaming, but I find that I can be satisfied with one publication subscription as opposed to streaming where I have 3-4 subscriptions.

You can remain very much in the loop by getting a subscription at one of the major 'centrist' general newspapers (NYT, WaPo).

Are we going to end up in a situation where the only news sites without a paywall are those filled with sensationalist and/or inaccurate articles and Z-list celeb gossip like the Daily Mail [1]?

[1] https://www.dailymail.co.uk/

I think the business model associated with these paywalls is what is flawed. I'm not going to subscribe to a site just because I viewed a number of articles if I didn't perceive them to be high value.

They must demonstrate value for money, and this problem seems to present evidence that the perceived value isn't there.

I have not read a Washington Post article in a long time since they made it harder to bypass their paywall. Anytime it comes up, I just close the tab.

I don’t mind supporting journalism (and I pay for two local papers) but I do not like the heavy handed tactics of this bigger sites.

ok I do the same thing but it isn't because if they let me read the articles for free then I would pay them.

I tipped for the guardian in the past and their sales are up[1].

[1] https://news.ycombinator.com/item?id=19799893

What are your favourite paywall blocking software and extensions?

I wonder if paywall bypass tools will one day be treated as burglary tools.

Why? The webserver chose to provide the browser with content. If they're comfortable serving the article to any browser that whispers the secret passphrase "I'm visiting you from Google.com" - well, that's their prerogative.

If a guy can be convicted for traversing a URL ... https://en.wikipedia.org/wiki/Weev ... then just about anything can be prosecuted if the "victim" and prosecutor are determined enough and the perpetrator is unlikable enough.

His conviction was overturned on incorrect venue, not because of a problem with the charge.

Yep, I know of Weev's case and think it was a heinous miscarriage of justice. Companies outsourcing the consequences of their misconfigured servers as externalities to the legal system is how we wind up with massive data breaches that firms simply brush off and consider business as usual. But that's a whole 'nother soapbox.

https://ansuz.sooke.bc.ca/entry/23

Probably no different than a crack.

This is just getting worse. The internet has been sold out and now it's just pathetic.

I imagine in a not-so-far away future, it will be illegal to bypass paywalls and fines will apply.

If I can’t use incognito mode to read to NYT, NYT shouldn’t use incognito mode sources.

Reply on news.ycombinator.com