Tell HN: Bypass Paywalls repository is gone | Hacker News Reader

Tell HN: Bypass Paywalls repository is gone

343 pointssogen125 comments

Repository seems to be gone:

https://gitlab.com/magnolia1234/bypass-paywalls-chrome-clean...

Visiting the user profile says it has been blocked:

Seems like Gitlab seems to be even more ridiculous with their responses to large businesses and potentially dubious copyright claims than GitHub is. It's not the only time this month I've seen them take down repositories related to projects that some large company wanted removed despite it not necessarily being illegal in any way.

Yeah this is not a DMCA thing, there is nothing copyrighted in the code.

The core problem is the newspapers wanting their cake and eat it. They want those sweet google hits but they don't want to give their articles to readers without payment. So there's always a way around it if you can manage to fake a google spider.

For me the current model is so broken. I end up on lots of different newspaper sites. But no, I'm not going to sign up for a subscription to the washington post or whatever to view 1 or 2 articles per month. That's just ridiculous, I don't even live in America. I subscribe to the local paper because it has content I read every day.

If I'd subscribe for every article I get referred to by google or here on HN I would spend hundreds in monthly subscriptions :P It's just not a reasonable ask.

Yeah that's the thing isn't it? When people use a search engine, they want to be able to click through to the result and read it, not have to subscribe to some paid service to do so.

If your business model doesn't let you offer that, then the answer is to accept you're not gonna be able to rank in search engines. That's it. Maybe a search engine for paid content could exist as its own thing, but that would be because people using it would expect they'd have to pay for what they want to read there.

The attempts to get both Google hits and paywall content feel like some author trying to make everyone pay them money to borrow their book from a library; completely counter to the point of the institution to begin with.

Yeah... some services offer a couple of articles per month and that would be nice though... there is again a way to circumvent it. I'd sign up for a free account with like 1-3 articles views per month because literally I don't read more in majority of the sites (apart from links on HN or reddit)... I do pay sub for the sites I visit often (at least 1-3 times a week)...

The trouble is that the ideal model (newspapers on the whole used to cost, like, 0.25c) is extremely tricky in the online space, because payments are high friction and small payments especially are not really economically viable. I'm not sure what solution could possibly exist that wouldn't involve some hyper-common payment provider building out a service to solve this specific thing, and then getting all the online news sites to agree to use it, and then getting most visitors to accept "I need to spend less than a dollar if I want to read this article" as normal and okay.

It's ... kinda crazy when I type it out.

But that's basically what advertisements do for publishers today, and if we're moving away from that model (and we *really* should), what do we replace it with?

The ideal model IMO would be P2P micropayments (hopefully a web standard implemented as a browser API), where the payment processor middleman is cut out, so small transactions become viable. Patreon allowing you to bundle and only be charged once for your handful of monthly subscriptions was a good half-measure and a great way to reduce the middleman's cut, but since its elimination, we're back to square one. It doesn't seem realistic to entrust an omnipresent payment provider with this, since they'll always collect a fixed amount of overhead per transaction, which will intrinsicially disincentivize smaller individual payments. But maybe I'm being overly paranoid in some aspects and naïve in others.

The primary purpose of the DMCA was (and, I suppose, still is) to criminalize technology that circumvents access control protecting copyrighted works. There's also the part about takedowns, but circumvention was definitely the big issue when it was made into law.

Remember DeCSS? https://cs.stanford.edu/people/eroberts/cs201/projects/1999-...

Thanks to the Streisand Effect, I've installed this. Seems to work pretty well on the few sites I've tried. Hopefully the maintainer can find a new home and let people know on Twitter.

https://twitter.com/Magnolia1234B

Streisand Effect indeed... I wasn't aware this existed but now I've saved a copy.

Wow, Gitlab has gotten ridiculous. They're making me sign up for an account, verify email, go through two captchas, and fill out a survey before they'll let me view the repository.

. . . and now that I've done all that the repository is 404ing.

Phone number verification also fails for Google Voice.

Loads of services have that issue, not just GitLab. Some are even blocking Twilio numbers.

Twilio is registered on the PSTN as an IPES, if your making the effort to block non-wireless OCNs, then your gonna block Twilio.

You are*

https://youryoure.com/

They really had an opportunity to capture significant market share and they ruined it. Their interface also doesn’t work without javascript. What a shame.

I suspect it was a conscious decision, they went for large, paying, enterprise customers instead of fighting for the free, public, and open source projects.

GitHub isn't usable without JS either anymore.

(A long time ago, I clearly remember it did. There's nothing about browsing files or participating in the effective forum that's called "Issues" which fundamentally requires JS.)

It’s now usable without JS once again. It was not long ago that repositories stopped being viewable without JS enabled, but they are now again.

I hadn’t tried interacting in Issues without JS enabled in years, but I just tried to out of curiosity and successfully replied to an issue that way.

I must admit to finding myself quietly impressed that any place would be capable of reversing itself on a decision like that.

Why would they have to make their interface work without JavaScript? The few people who block JavaScript for sure know how to enable JavaScript again. And if they block JavaScript for security and trust reasons, well, they are going to download software from gitlab and run it anyway.

This is the attitude that has ruined the entire web. The real question should be: why is JavaScript necessary and what does it add? (In the majority of applications, absolutely nothing.)

It adds interactivity and dynamic features that would be pretty slow or very challenging to implement in pure HTML/CSS.

That’s nice I guess, but these should be progressive enhancements and the page should still work without JavaScript enabled.

> pretty slow or very challenging to implement in pure HTML/CSS

That's the heart of it, isn't it? It's challenging.

At the core of modern web development is the attitude that developer convenience trumps all.

The software is free after all. (The thousands you pay in bandwidth and hardware upgrade costs to keep running their bloatware doesn't count because none of that goes to the web developer, and is therefore irrelevant in their it's-all-about-me worldview.)

Agreed. Sadly it's due to the constant churn of tech these days. Developers should just take notes from the books on the shelf: don't change without consent unless you explicitly some change.

Slightly off-topic: I've been viewing YouTube on a old YouTube frontend that works back to (at least iirc) IE6 and Win98 for some time now [1] [2] [3]. The frontend feels mad snappy as hell and it loads fast compared to YouTube today since the frontend is not heavily JavaScript reliant, it just uses it to enhance the site.

[1] https://i.ibb.co/YpmFP91/yt2009-34.png [2] https://i.ibb.co/THK0wk0/yt2009-35.png [3] https://i.ibb.co/nk7nxZQ/yt2009-36.png

    Abuse of animation and skip of best-effort loading:
    This is probably more relevant to the sluggishness and bad interoperability, rather than JavaScript.
    [ Those abusing JavaScript for nefarious purposes: probably not the JavaScript to blame. ]

    Standards established don't necessarily imply being well-designed:
    Blindly following without thinking shall regardless trap.

    Many current infrastructures are fundamentally flawed and difficult to fix:
    1 step wrong, all steps wrong.

Delivering STATIC DOCUMENTS via dynamic javascript apps running on top of http is the quintessential caricature of overengineering.

I would argue that gitlab.com is a bit more than a collection of static documents that could be delivered over gopher too.

    Over-engineering indeed, but it does address actual problems.
    Primarily: Addressing what most browsers haven't well managed to handle: interaction with big files.
    E.g. huge database-like Plain Text: browsers typically fail with OOM (Out of Memory).

    I'll write a review of GitHub's delivery infrastructure when I have the time.

I think the sign up page is because the repo is blocked. It probably redirects to that for non-logged-in users.

warning: gitlab link is taken down too, dont waste time

Another extension that deserves some fame is "Behind the overlay". it allows you to kill any unskipable full page overlay informing you about the benefits of a subscription and the excellent quality of the article beneath it.

Coupled with the Archive.ph button they make up my holy trinity of paywall bypass.

curious, what's the 3rd part of the trinity now that Bypass Paywalls is gone ?

It's not going anywhere, it just needs to find a more reliable home.

Thanks for the twitter link, the author has now posted a wetransfer link and is thinking about another home.

Uhm... maybe codeberg.org? Or self-hosted forgejo/gitea?

This extension works remarkably well. But it requires frequent updates to stay working as sites change. I sure hope there's some statement on what's up soon.

I've ran into this issue as well.

Is there an alternate "store" for open source chrome extensions?

Something akin to F-droid that keeps them up to date?

You can only install from the Chrome Web Store. All other methods are annoying and you have to manually whitelist extension ids. I don't understand why the Chrome Web Store does not fall under the Digital Markets Act in the EU. Google is clearly a Gatekeeper here.

Archived copy that's newer than others mentioned:

https://archive.softwareheritage.org/browse/origin/directory...

  Tip revision: 1c8f71997c1fb3b3f60957514ec2bce1591a1bfc authored by magnolia1234 on 01 April 2024, 07:35:12 UTC

also the Firefox-specific repo: https://archive.softwareheritage.org/browse/origin/directory...

also the bpc-uploads (XPI, CRX): https://archive.softwareheritage.org/browse/origin/directory...

I updated the archive.org copy with this one. Oh man, what a great site.

I think you need the "chrome" repository too.

I'll put all 3 you linked up.

Is it possible to clone it instead of only downloading current version or viewing history?

Previously (yesterday): https://news.ycombinator.com/item?id=40007670

Do bypasses like this, which I assume just do things like deleting "uncloseable" client-side modals and spoofing UA/referrer headers, count as copy-protection circumvention technology under the DMCA?

Does the thing being bypassed prevent you from saving the page, or from making in any other way a copy of any content you've already fetched?

If not, how could it be classed as copy-protection?

Edit: One other thing: if spoofing UA headers was a problem, all the browsers in the world would fall foul of it - because non-Mozilla browsers all say that they're Mozilla, and Opera said it was IE, and Chrome said it was Safari, and now a bunch of browsers say they're Chrome - at the same time as saying they're Mozilla. Except it's more complicated than that.

I guess it's less copy-protection and more access-control.

this is basically the "computer scientist" view of the world in https://ansuz.sooke.bc.ca/entry/23, especially the Monolith bit. spoofing UA could be legal in general or when done for interoperability but illegal when done with intent to bypass access control.

Right, the UA spoofing browsers perform is historically done to bypass sites not wanting to serve them content.

Well the UA spoofing I perform is done to bypass sites not wanting to serve me content.

Those looking into the history or the user agent, and why it happens to advertise itself as Firefox, Safari and Chrome ate the same time, can look into this article: https://webaim.org/blog/user-agent-string-history/comment-pa...

Maybe it's fraud? The user claims to be reading an article via a facebook reference but in fact does not use facebook.

Hard to see how to police that from the web site's perspective though.

To be clear I'm not advocating for anything here.

I'd also worry about a CFAA violation too.

https://www.nacdl.org/Landing/ComputerFraudandAbuseAct#:~:te....

The question is: do you actually want to spend time dealing with that or just nuke anything that gives you a legal headache?

When big corpos tell you to jump you jump.

Per magnolia's Twitter:

magnolia1234_bpc @Magnolia1234B · 22m Update: tomorrow a new Firefox-release v3.6.4.0 (signed = security check) ! Manual update required with upload on X (new version will update automatically)

For now no more public source code or request issues (use DM on X) https://twitter.com/Magnolia1234B/status/1779050596181438791

WeTransfer already expired (should last 7 days), but new upload here:: https://t.co/8TADPNhiYW

That's quite fishy.

I'm grateful for the extension. I used to see a steady stream of updates. I hope the author find another host to host the repository.

I have a six day old copy of the repository at 'https://drive.google.com/file/d/1cS0uXgXdBjkI1ExmUr4ma23FGCw...' and 'https://ibiza.seedhost.eu/smugchicken/filebrowser/api/public...'.

git log:

commit 1ec176a328c072114148b9359de4927ca4a1a095 (HEAD -> master) Author: magnolia1234 <7676006-magnolia1234@users.noreply.gitlab.com> Date: Sat Apr 6 07:23:50 2024 +0200

    Add Nordwest Zeitung

    Fix Capital.fr (videos)

Apparently this page works! You can download latest version from here: https://gitlab.com/magnolia1234/bpc-uploads/

EDIT: magnet link with all btc-uploads content magnet:?xt=urn:btih:7839e845f7965bed1035dc5d4635811d1ede73d2&dn=magnolia1234-bypass-paywalls-uploads-master

also: https://archive.org/details/bypass_paywalls_clean-3.6.3.0

gitlab link does not work

Seeding for Streishand ™

The last couple of versions I tried to download gave TROJAN warnings from MS Defender. I had created an issue on the project notifying the developer, and I understood he was changing things to not give a "false positive". Perhaps GitLab didn't think they were false positives?

The archive linked in the developer's Twitter profile (https://twitter.com/Magnolia1234B) lists a new GitHub repo: https://github.com/bpc-clone/bpc_updates/

Download latest release from here: https://github.com/bpc-clone/bpc_updates/releases/tag/latest

It should auto-update (as long as the new GitHub repo isn't taken down...)

Why was this hosted on gitlab.com ? If you're not on github, the discoverablity is pretty low. You are not self hosting. There is no upside for a project like this to be on gitlab.com. If you're already taking the hit of not being mainstream, why not self host in a sane country ?

I don't know about this project, but some projects fled GitHub on principle, when they sold out to MS.

If this project were, say, an adtech company, or someone publishing open source solely for resume-boosting reasons, then principles might not matter.

i dont understand this principle at all, mind explaining what is the problem with MS owning github?

https://jacquesmattheij.com/what-is-wrong-with-microsoft-buy...

1) There are lots of reasons to avoid Github and they are all spelled Microsoft.

2) Putting something on Gitlab means that there is one level of activation energy between idiots trying to pad out some corporate "social goodwill" number by "fixing" things on your project.

If I ever had to open-source anything, I would host it somewhere else and mirror a read-only repo to github.

I've never experienced the latter but if somebody wants to waste their time sending me unwanted pull requests I can just ignore them.

Discoverability is solved by being on the Mozilla AddOn store.

Besides, not everybody cares about discoverability.

But it was removed from AMO. That was an issue as people that installed it from AMO just stopped receiving updates silently. You had to uninstall it and re-install the gitlab version for the auto-update to work again.

Hmm, didn't catch that. I see there's still _a_ Bypass add-on on Mozilla. There always were a few, and close forks last I bothered to check. Is this different?

There are some but with very few users and limited to a small number of sites. The only one still on AMO that comes close is the one by iamadamdev (https://addons.mozilla.org/en-US/firefox/addon/bypass-paywal...) which is discussed here : https://news.ycombinator.com/item?id=40016918

It was removed from AMO after it was already on the site. Maybe they didn't want to move it.

Weird, someone said something about that earlier today but then it got restored and now it's gone again?

Use DIllo, Lynx or Links on the URL's. If that fails, head to gemini://gemi.dev, the section News Waffle, and paste your own link. Use Lagrange, Gplaces or whatever Gemini client you like.

Does Gemini have any news about being vegan?

The Gemini 'News Waffle' service it's just an HTML decrapifier, you can put almost any web source as the input. Copy and paste the URL from a browser and that's it.

It's hard to know how up to date this is but:

https://github.com/eugenesvk/bypass-paywalls-chrome-clean

(It does seem to be more up-to-date than other mirrors I've found)

Edit: It's about 300 commits old.

https://web.archive.org/web/20240410093750/https://gitlab.co...

These appears to be the most recent forks for the archivers. ;)

https://gitlab.com/Fernos/bypass-paywalls-firefox-clean

latest: https://gitlab.com/expat.russia/bypass-paywalls-firefox-clea...

https://gitlab.com/cjkpfit/bypass-paywalls-firefox-clean

It's good that there are forks, but I was under the impression that the head person running the project was the one keeping all the paywall bypasses up to date. Even if these forks work, if no one is actively keeping track of each site that is being bypassed as it breaks or changes, they won't be working for long.

I am looking into uploading it on archive.org as well. Fingers crossed someone takes the torch. Bless open source.

Looks like the official store one was last updated on March 1, so the forked links from mid March are only slightly ahead.

https://addons.mozilla.org/en-US/firefox/addon/bypass-paywal...

Hopefully the original user migrates to a new platform, but if this store version gets removed I think we can all assume they got struck with a lawsuit threat.

edit - done: https://archive.org/details/bypass-paywalls-firefox-clean-ma...

What did you mean by "Item details is no longer available via the wayback machine"?

The Wayback Machine seems unable to archive the JavaScript-based GitLab UI in general; it wasn't removed. For example, another major repo: https://web.archive.org/web/20240228225845/https://gitlab.co...

Ah, that explains it! Will edit it (also with the updated April 1 copy - https://archive.softwareheritage.org/browse/origin/directory...). Cheers.

Newest repo location https://github.com/bpc-clone/bpc_updates/releases

I happen to have downloaded the master branch at 9am this morning. Here's a torrent: magnet:?xt=urn:btih:92042e65d1b38cd5d97c29baa4d0c9e2af46f355&dn=bypass-paywalls-chrome-clean-master.zip

The easiest way to bypass paywalls is to just disable JavaScript for the site. It works pretty much everywhere I've tried it except for the WSJ, which I suspect this extension doesn't work for either.

Edit: or, maybe it does. I see WSJ.com in the config, with a referrer as drugereport and user agent as Google not to get around it.

I remember there is a big media site, which even use Archive.is can't check its paywalled content part, do you know that site?

Can this plugin work on it? I doubt.

it works on WSJ, I use it for that site. for some bypasses, it doesn't "just load the page" but rather pops up a link to archive.ph or similar to pull the article, still convenient

I also point out there is "bypass paywalls" and "bypass paywalls clean", kind of similar to the uBlock family with "uBlock origin" being the "good guy" edition

Some WSJ articles work and some don't for me. I think the difference may be opinion pieces.

Not sure about WSJ, but in some cases the extension fetches the content from archive today and inserts it into the displayed website tab.

If there is no mirror on archive today, the extension cannot insert it.

magnolia1234_bpc @Magnolia1234B · 22m Update: tomorrow a new Firefox-release v3.6.4.0 (signed = security check) ! Manual update required with upload on X (new version will update automatically)

For now no more public source code or request issues (use DM on X) https://twitter.com/Magnolia1234B/status/1779050596181438791

WeTransfer already expired (should last 7 days), but new upload here: https://t.co/8TADPNhiYW

What's the relation between the Gitlab repository and the Github one? https://github.com/iamadamdev/bypass-paywalls-chrome

The Gitlab one is actually maintained. They haven't been using GitHub for a while now. (Plenty of updates to the extension have occurred within the past 6 months, the date of the last commit on GitHub)

The gitlab one is "clean". But i don't remember the exact details regarding what it means. Just not to use the "dirty" one.

The clean one is not “clean” in that it still does all kinds of tracking

What do you mean with this?

https://github.com/iamadamdev/bypass-paywalls-chrome/blob/c6...

If you look at the source code in any of the mirrors people have linked, you'll find that the "clean" one that was taken down today does not have Google Analytics.

That’s not the issue. It’s that it tracks in other ways.

Okay, for some reason they took down that one but the original (but not maintained) fork is gone. Fascinating...

They have a point, I guess. It pings home in bpc_count_daily_users.js.

https://archive.softwareheritage.org/browse/content/sha1_git...

The "clean" version is a fork made by someone else, non-clean version has/had Google Analytics in it, and the fork is more actively maintained

The github repository isn't under active development. The gitlab repository exists because a contributor was frustrated with the slow updates, forked it, fixed many bugs and removed google analytics, and actively maintained it.

https://web.archive.org/web/20240114042043/https://gitlab.co...

I think a good idea would be forking Telegram's quick article view feature source code and developing on it.

Now I wish I had bookmarked the distributed (peer to peer?) github replacement I’ve seen trend on HN a couple of times. It seems like a good place to host something like this. Anyone remember which tool I’m talking about?

https://radicle.xyz/ This?

https://radicle.xyz/ most likely.

So... any mirror out there?

There's also a version still in the Firefox catalog (the official one has been banned from there too): --link removed, it was a fake, the one I referred to is gone too :( --

This is a help in particular for mobile firefox because they refuse to let us sideload addons.

Pretty sure that's the version that kept calling Google Analytics, and why we got a clean fork "Bypass Paywalls Clean".

EDIT: Actually, never mind, this isn't even that, this is a fake according to the reviews.

Oh yes the one I mean is Bypass Paywalls Clean (D). But this also seems to have been kicked out of the Mozilla store. It was just a resubmit by someone else after the original got kicked out when a French newspaper complained.

We really need sideloading capability in mobile firefox :(

You can sideload in Firefox Android beta branch.

Settings> about Firefox> tap on the fox logo a few times> debug menu enabled> now you see "install ad on from file" under advanced in the settings menu.

Wow they finally added this?? Wow. I will try that thanks!

In Firefox for Android Nightly, you can (since recently) sideload extensions via the hidden debug menu (tap the Firefox logo in the About section a dozen times). This will add an "Install extension from file" option to the main settings menu.

I don't know whether this also works in the standard version of Firefox for Android.

It is good they reenabled this feature. Since it is/was very short-sighted of Mozilla to make their centralized addon store the single gatekeeper for extension access on user devices. As if they never thought about power dynamics in software.

Edit: * Has been taken offline now. *

They didn't purge the xpi download repository, so you can still download the latest XPI package signed by Mozilla:

https://gitlab.com/magnolia1234/bpc-uploads/-/raw/master/byp...

There are other xpi packages available in the same repo:

https://gitlab.com/magnolia1234/bpc-uploads/

I get 404s now following those links.

how powerful do you have to be in the US to be able to take down mirrors within an hour? this is disturbing to see

Specifically for a tool for bypassing paywalls. I'd expect this behavior if we were talking about CSAM or a stream of terrorists beheading people.

You just need a lot of money and some lawyers, everyone just folds as soon as lawyers and legal threats get involved.

checkout https://brisk.news

Orthogonal question: Does bypassing of paywall somehow work for Substack? Can a solution like RealDebrid / SciHub, but for Substack exist?

https://web.archive.org/web/20231223094834if_/https://gitlab...

Reply on news.ycombinator.com