Hacker News Reader

Comment by misterbwong | Hacker News Reader

I'm not quite understanding the logic here. Just because another customer (google) was given access, does that mean everyone should have it? Would you rather publishers give google private direct feeds to index?

View on HN

> Just because another customer (google) was given access, does that mean everyone should have it?

They didn't provide access to just another customer or Google -- they send the article text to EVERYONE in the HTML response. They are thus making that text publicly available. If they want to make content only available to certain individuals, that is literally what paywalls and authentication/authorization are for.

Instead, they want Google to index the site AS IF the content is publicly available but then use javascript to restrict that content from users client-side on a case by case basis. The golden rule of authorization is never rely on client-side authorization as by then you've already willingly given the data/information to the client.

And the client has no obligation to not look at that information/data. They didn't do anything nefarious to get it -- you literally gave it to them.

> Would you rather publishers give google private direct feeds to index?

Absolutely I would. If we could enhance the indexing and search engine system so that (a) I could tell google which services I subscribe to and (b) google knows which articles are behind paywalls, then we could have a world in which we could intentionally filter To or Away from paywall content.

I know that I would often filter TO paywall content, especially if: - I could also tell google to avoid content farm content via a filter or content full of ads (not in their interests currently but it might become so as more of us get frustrated with shittier and shittier search results) - If google could surface the price to access each article on the search results (wherein articles on sites I already pay/subscribe to would be free and articles on sites with individual pay per access would be transparent and upfront).

This is the “but she was wearing a low cut top” argument.

In a way these publishers are abusing a public service (search engines) to get a higher ranking, giving the illusion of publicly available content, but then breaking the promise. In practice for their personal private gain they are damaging the system for everyone else, which is much worse imho.

Stealing from them just because they are trying to gain search ranking in a shady manner is still stealing. Two wrongs don't make a right.

Please define how this is theft? They are making the article publicly available.

There aren't making it publicly available. If they made it publicly available there would be no paywall. It just so happens that people can utilize some code to bypass the paywall.

This is a false analogy for several reasons: 1. If, in your example, "but she was wearing a low cut top" was used as a justification by someone to physically sexually assault someone, then obviously that is wrong as we've established that rights to one's body belong to the person with said body and you need their consent. If on the other hand, you are simply looking at someone who is dressed provocatively, then while crass, that definitely is not and should not be illegal. In a public space, we each have a right to look wherever we please. We can be judged for where we look (and people should be judged for oogling at someone), but it shouldn't be illegal. There is no right to privacy in a public space for very good reasons. 2. But even if you meant it with the later case (the oogling), there is a huge difference in this case from a moral perspective, namely these are companies who are selling/monetizing their content. Thus, (a) they want you to look at their content and (b) they are ok with you paying to do so. So then your example would be much more akin to an art gallery putting their art on the public sidewalk to entice people to come in and see the art -- only to find that the people are happy to look at the free art on the sidewalk and never pay the entrance fee. No theft happened in that case. The art gallery simply had a flawed economic model where they gave away their product and then were angry no one paid for it.

If you put something in the public sphere, it is public. Simple as that. You have no right to turn your intention of how I "ought" to consume your product into forcing me to do so if you have given me your product for free.

Use a paywall, Use proper authorization.

I solely use firefox for getting around paywalls. I have it automatically clear all cookies upon exit. That way any website that gives a few free articles per month will always be free for me.

From an economics perspective, they are trying to using old school pricing tricks (e.g. bundling, price discrimination, variable unit pricing to enable 2 article users == free, 10 articles == paid) as a tool for managing their customer funnel. But the golden rule of price discrimination is it only works if you can actually prevent your customers from engaging in arbitrage and can keep them in their bucket. A store can have a policy like "limit 1 per customer" but that doesn't prevent me from getting 100 of my friends from buying the thing. If I want to clear my cookies and look like some anonymous new customer each time, there is absolutely nothing illegal OR immoral about that. That's capitalism and they have designed a pricing model that can be gamed.

> A store can have a policy like "limit 1 per customer" but that doesn't prevent me from getting 100 of my friends from buying the thing.

Sure, and if there's urban chaos, there's nothing preventing me from looting a store. Or sneaking stuff in my backpack. It's still stealing.

As to your example about the art, it's more like if one piece of art is in the window, but the rest of the gallery you can only see if you buy the ticket. But there's a door in the alley you can sneak in to and nobody will notice. So you sneak in the back, avoid paying the price, and get to see the whole thing for free. That doesn't sit right with me ethically.

I hear you and I understand the technical aspects of it but I think you're making the false assumption that Google indexing == content is public. That may be the user expectation, but that doesn't mean it's true. Am I off base here?

I'm having a hard time envisioning a scenario where giving Google exclusive data will be in any way good for content creators and site owners in the long term.

If we took it a step further and gave google private direct feeds from publishers, would you then be OK with the automatic moat around Google's services that would then be built? How about the one that formed from the private subscription data that you needed to feed to them for them to tailor their searches to your preferences? What if they started charging for search? Would you then go to another search engine that did not have the private content feeds and could not, by design, find those relevant paywalled results?

edit: Just to reiterate, I'm not defending the paywall practice (or breaking it, for that matter). I just see this practice pushing paywalls to be hard rather than soft.

Many of these paywalls simply have either the full body in the response or will return it if you ask with the right user agent (e.g. the one that google uses). These plugins then either remove some overlays or impersonate the google bot.

Obviously that’s not how it should be implemented and that‘s also not what google suggests you to do as a publisher. Instead you should verify that the bot is actually google https://developers.google.com/search/docs/advanced/crawling/...

Reply on news.ycombinator.com