Hacker News Reader

Comment by jashkenas | Hacker News Reader

Source code: https://gist.github.com/jashkenas/9a4d9e648eb11aab394e

Could you give more information on how do some websites react to a google referer?

The design intent is almost certainly using Google's "first click free" policy, which primarily applies to newspapers like the NYT or WSJ. Google back in the day struck a deal with the print media: they would remain in Google's index if and only if clicking from a Google search to their articles circumvented their paywall on the first click. If they put only partial content or a full paywall, they'd be delisted. It was not exactly a secret that this would mean that Google would rank scrapers higher for NYT articles than they'd rank the NYT.

Thus, when someone posts a paywalled link to HN, someone inevitably posts the advice "Google the title and click through." (Other than, say, "If you're a working professional, just buy a WSJ and NYT subscription.")

I am a working professional but I have always resisted buying a subscription because I don't want one single publication to become my de facto news source.

I would love if I could pay $20 to $30 per month to some aggregator that lets me read articles from Al Jazeera, NY Times, WSJ, BBC, and a selection of other diverse sources.

I like Circa News (take my money please!) because they aggregate news from a variety of sources and present news items in small bit sized chunks listing facts or statements. They also link to the appropriate sources. edit: and you can follow news stories to get updates when new information is available.

They're either shutting down or being acquired soon: http://fortune.com/2015/04/30/news-app-circa-buyer/

That's a damn shame. Their (android) app has unfortunately become significantly less useful due to technical reasons lately. I always did wonder how they would monetize.

That's unfortunate. I'd pay for it even if it were free otherwise because I like the service so much.

Never heard of Circa News before, thanks for mentioning it.

https://circanews.com/ for the interested. I use the app exclusively though and haven't spent much time on the website.

If you get a chance you should check out BriefMe -- pretty much that, except free, with a very clean UI. http://www.getbriefme.com

Oh my, that's a terrible site. So I have to watch a full page video to find out what I might want to install?

So it's not so much that they intended to have a "first click free" policy in the first place but that they are forced to do an exception in order to remain in Google's index. That's interesting, I always wondered why didn't they apply the same policy to being linked in Reddit or HN.

Actually, some do. NYT for example allowed (maybe they still do) incoming links from Twitter & FB to bypass paywall

One issue with the subscriptions, at least with the NYT, is that their pricing is a little crazy. The phone version is $15, the tablet version is $20, and both together are $30. It seems... excessive. WSJ at least (used to?) offer a single subscription that gives you access on all devices.

Not to defend the NYT craziness, but that plan somewhat exists in the weekend print edition and should save you money, too. Subscribe, check the box in your account to have them donate your paper copy to a school or library, and enjoy access on all of your devices.

I didn't see that option when I signed up a few years ago. When I look at my account now, the only place I see that option is when requesting Vacation Suspension. Do you happen to know if there's a place on the website to set this indefinitely? I guess I can call.

I'm on permanent vacation as far as the NYT knows

Gotcha, and it sounds like that doesn't affect your digital access. Thanks for the tip. At least 50% of time, we just throw away our physical Sunday paper without ever looking at it. It's ridiculous, but I couldn't find any friends who wanted it.

Weren't they the same publishers that were suing google for linking directly to their articles?

No. They weren't. Some mis-guided ones tried.

Many news websites will pop up a paywall if you visit a link directly, but won't if you arrive from a Google search.

The reason for this being that Google results penalize sites that show content to the crawler and then hide it from people who visit the page (ie non-subscribers). If they want it in the index, it has to be actually viewable.

The alternative would be to not show paywalled content to Google's crawlers, and have them entirely left out of search results.

I've seen some websites highlight your search terms for you. It's kinda useful, but it's also kinda creepy at the same time.

Did someone make a Firefox version yet?

You can achieve the same thing with the RefControl add-on, which allows you to set a custom referrer for a particular site:

https://addons.mozilla.org/en-US/firefox/addon/refcontrol

Fantastic tip. For anyone interested in trying this out, use http://news.google.com as the "custom" referer.

Screenshots and help here: http://www.renjusblog.com/2010/01/hack-read-wall-street-onli...

Thanks, this worked perfectly

Perfect! Thank you!

For sites that has x free articles/month, I just discovered Private Tab[1] which makes it easy to turn an existing tab private or force a bookmark to always open in a private tab.

[1] https://addons.mozilla.org/en-us/firefox/addon/private-tab/

Self-Destructing cookies extension also keeps the read count at 0 after you close the tab. As an added bonus the extension should also help prevent you from being tracked across the web by various advertising/state agencies.

here's a port i cobbled together from some firefox addon demos in 15 minutes. tested to work on wsj

https://github.com/zethraeus/wgsm-ff/blob/master/gglsntm.xpi...

(consider reading the source instead of just trusting it)

I sent you a pull request improving your code – only making the change apply to the current tab, and changing the Icon Name ;)

Reply on news.ycombinator.com