Bypass Paywalls Clean for Chrome/Firefox | Hacker News Reader

Bypass Paywalls Clean for Chrome/Firefox

github.com

35 pointsjoker76520 comments

Open articleView on HN

There is currently a really interesting paywall-bypass bug on NYTimes.com:

- Take a paywall'ed URL: https://www.nytimes.com/2013/05/05/magazine/y-combinator-sil...

- Add a dot after the domain: https://www.nytimes.com./2013/05/05/magazine/y-combinator-si...

voila, no paywall. Would love to hear an inside story about why this bug appeared.

Unfortunately this bug will probably get fixed from your post

I really feel like it's just a matter of time where they require an account to read the articles and only provide snippets to those who are beyond there free limit.

Thats the difference between a domain name and a fully qualified domain name.

If you've ever administered BIND, you'll note that you need to add a ending . to A records. Thats the FQDN.

Now, our browsers link cookies to domain. And interestingly enough (example.com)!=(example.com.)

That's why sites like nytimes.com. gets around the paywall, because you don't have a cookie history with that site.

I can't say for certain why this bug is exposed by NYTimes; however, I saw a comment on HN about trailing dots on domain names[1] being a browser implementation quirk.

The author of that article suggests that NYTimes may have misconfigured their Apache VirtualHost.

[1] http://www.dns-sd.org/TrailingDotsInDomainNames.html

It's simpler than that. I'd disagree that it's a site config bug. I think this is intended operation with respect to browsers.

Your browser won't allow access to cookies of "nytimes.com" to the "nytimes.com." domain, as because they are different. "nytimes.com" is ambiguous.

If you were on the "example.com." network , and you typed in nytimes.com , it could be nytimes.com. OR nytimes.com.example.com.

Based on what you are saying though, you lead me to believe that the NYTimes wants this behavior to occur, which I doubt.

If they want to paywall access to their site but adding a dot to the domain causes it to be bypassed, then it is misconfigured in some way, regardless of cookies.

> Based on what you are saying though, you lead me to believe that the NYTimes wants this behavior to occur, which I doubt.

Not at all. We wouldn't expect reddit.com. to share cookies with news.ycombinator.com. The browser enforces that hard separation between the 2 domains with respect to access to cookies and such.

Thats the same split with nytimes.com and nytimes.com. . The browser sees them as 2 distinct domains, and will absolutely not let cookies cross.

> If they want to paywall access to their site but adding a dot to the domain causes it to be bypassed, then it is misconfigured in some way, regardless of cookies.

It's their choice (nytimes) to not do a paywall for a 'new potential customer'. And that data is stored in cookies. You can simulate that by purging all nytimes cookies and you can access as no initial paywall.

Going to their FQDN is a 'different domain' with respect to the browser and its cookie store. So its only logical that you start 'fresh' (aka: no cookie history). And there's nothing that I know of that can allow cookie sharing... Then again I do systems administration.

>It's their choice (nytimes) to not do a paywall for a 'new potential customer'. And that data is stored in cookies. You can simulate that by purging all nytimes cookies and you can access as no initial paywall.

Thank you for clarifying. I hadn't considered that NYTimes may not show paywall for a 'new potential customer' since I am all to familiar with their paywall.

How does this work?

It's a variety of techniques depending on the site. See https://github.com/magnolia1234-new/bypass-paywalls-chrome-c... and https://github.com/magnolia1234-new/bypass-paywalls-chrome-c...

Techniques include blocking cookies, specific JS scripts, redirecting to a different URL, changing the user-agent to pretend to be Google's indexer, changing the referer to make it appear the user is coming from a Google search, and removing or hiding elements from the page that get in the way of reading the article.

There's so much content on the internet that, if I stub my toe on a paywall, I move on to the next link.

Did this get manually bumped off the homepage?

Repo moved to BitBucket: https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-cl... https://bitbucket.org/magnolia1234/bypass-paywalls-fixed-cle...

Repo moved to BitBucket (ff typo):

https://bitbucket.org/magnolia1234/bypass-paywalls-chrome-cl...

https://bitbucket.org/magnolia1234/bypass-paywalls-firefox-c...

Refactored extension/add-on with lots of new sites, bug-fixes, add custom sites and update-notification. https://github.com/magnolia1234-new/bypass-paywalls-chrome-c... (no google analytics)

https://github.com/magnolia1234-new/bypass-paywalls-firefox-...

Has the link been updated? It's what the story is pointing to.

Anyway, nice work.

Old account was suspended after complaints of iamadamdev of spamming his repo (while he copied commits), I guess.

How is this related to the existing bypass-paywalls extensions from https://github.com/iamadamdev?

Hard fork/mirror after very slow progress ... Lots of new sites and features.

Reply on news.ycombinator.com