Ask HN: Legality of archiving/re-hosting Reddit content | Hacker News Reader

Ask HN: Legality of archiving/re-hosting Reddit content

25 pointsdom9652 comments

For content hosted publicly on reddit.com, what is the legality of downloading/scraping that content and re-hosting it on a separate website?

I am aware that the Archive Team is currently archiving Reddit[1]. As far as I understand what they do is legal. But I would like some reassurance.

Are there any good articles on this topic? Contact details for lawyers specialising in this area of the law also welcome.

1 - https://news.ycombinator.com/item?id=36254172

View on HN

It depends entirely on what nation you're operating in. (And additionally state/territory, if the answer is the United States)

Personally I'm most interested in the United Kingdom (since that is where I reside). But I would guess HN would be mostly interested in California (and I would be too since that is where Reddit is based).

Isn't it true that most implementations of copyright laws (irrespective of country/region) will likely have fair use exceptions for things like parody, journalistic reporting and archiving?

Well... no.

Some jurisdictions will have fair use exceptions for some scenarios. I don't think we can assume "most", "likely", or "irrespective of country/region", although it can feel like that if you happen to live in an area where such things are commonly allowed.

This is probably bad legal advice (above and following...). The EULA clearly states reddit content is owned by reddit. If you break their agreement you break it - and the Governing Law section says you would be put in Federal or state court in California, even if you are a foreign national.

Does a California court have jurisdiction over a foreign national. Probably not - which is entirely possible that Reddit would take a lawsuit to whatever court is closest to the person.

As I stated above, this isn't legal advice either, just my layman's view.

I'm also not a lawyer.

My reading of the user agreement[0] is that it distinguishes between "content that belongs to Reddit Inc." (things like the CSS and up/down vote icons) and "content licensed by Reddit" (what users post).

But yeah, lawyering is hard, don't just trust random internet comments like this :)

[0] https://www.redditinc.com/policies/user-agreement

Are EULA even enforcible? Also, AFAIK you can never completely revoke your rights to content you yourself created in the EU.

When someone posts content to Reddit under the Reddit terms of use they grant Reddit a license to use and distribute the content but ownership rights of the content remains with the poster. If you scrape Reddit and post the content in theory you open yourself up to copyright violation claims from the original poster of the content. The odds that someone is going to sue you for redistributing without permission content they posted to Reddit is likely extremely small but it is not zero.

From https://www.redditinc.com/policies/user-agreement

> Except and solely to the extent such a restriction is impermissible under applicable law, you may not, without our written agreement:

> - license, sell, transfer, assign, distribute, host, or otherwise commercially exploit the Services or Content;

Even without being a lawyer that seems pretty clear that it is not legal. Unless you have Reddits written permission, which I guess the Archive Team has

on the contrary, I read this as "it is allowed, unless your local laws allow us to restrict you from doing it" ;-)

That "except and solely" is what makes it very much not clear at all. They're basically saying "we make these demands iff applicable law allows us to do so".

I would guess that Archive Team don't ask for permission before acting, and certainly don't wait. The sad truth is that copyright law is not on their side. They preserve cultural memory because they believe in it, and they get away with it only because they're a disorganised collective of volunteers, so there's little to gain from suing them.

Given the users own the content, it'd presumably be up to them whether it can be rehosted or not. Reddit gets a license to display the content, but they don't really have any control over what third parties can do with it.

Personally I don't care if anyone reuses stuff I've posted on Reddit or other forms of social media (forums like Hacker News included), but there's always the possibility that someone might. And if you remove their posts when asked, I doubt most of them will take it any further than that.

There are separate terms for the API, which seem to indicate to me it's legal to use their API to download user content: https://www.reddit.com/wiki/api-terms

According to these terms, the content is owned by the users, and you're not to modify the content. However, if the content is owned by the users, then IMO Reddit cannot really say what you do or don't do with the content, as long as you're not building an application that acts as a proxy to Reddit.

The license is revocable to accessing their API, but they're not licensing you the user content, only the ability to download it. What you can do with that content is likely up to the laws in your jurisdiction. I'd say most content would qualify as public domain, though obviously some content will have copyright protection.

I would do the downloading now before they start charging for the API if you're serious about the project.

> as long as you're not building an application that acts as a proxy to Reddit.

Presumably you mean an app which displays Reddit.com as-is (i.e. with their logos/icons/UX). But what about a service which acts as a proxy to the Reddit API?

If content is not owned by Reddit and API design cannot be copyrighted then this seems legal, right?

IMO, the API terms look like they're written by a teenager, and you can do just about whatever you want. They admit the content is owned by the users, so not sure what they could actually enforce other than disallowing you to use their API. Once you have the content, it's none of their business at that point.

For what it’s worth, Reddit’s TOS reads:

You may not: Access, search, or collect data from the Services by any means (automated or otherwise) except as permitted in these Terms or in a separate agreement with Reddit (we conditionally grant permission to crawl the Services in accordance with the parameters set forth in our robots.txt file, but scraping the Services without Reddit’s prior written consent is prohibited)

TOS is not a law

But in some (not all!) cases it is a lawful contract, so it's worth consideration.

What if I crawl reddit comments without ever clicking on „agree“/signing up?

Then the same law would apply as if you recorded a movie in a theater and distributed it without signing anything.

How do you figure? In a movie you bought a ticket, that ticket is your contract with the theater, whereas scraping content while logged out is more like filming a police beating.

If you’re record a sporting event off broadcast TV, is it your impression that the law allows you to redistribute the recording freely?

Only if you agree to it.

If you don't agree to the licensing terms, then you have no legal right to access, archive, or redistribute the content.

Is it true that people have no right to access any of the content on a website based in the USA without agreeing to that website's TOS if:

* the majority of the content is publically available to users who have not agreed to the TOS, such as when arriving on the website for the first time from a search engine result?

Linkedin lost the case on this AFAIK. It is allowed to scrape linkedin if you haven't accepted TOS.

They could TOS-wall the site, but if they allow to access without TOS then you could access it without agreeing. Also Google definitely access every site and even caches it without accepting any TOS.

What's with the bunch of reddit crap on the front page recently?

https://old.reddit.com/r/OutOfTheLoop/comments/147fcdf/whats...

Why are we still posting links to claims they are "raising the price of access to their API from being free to a level that will kill every third party app" when they've made completely clear that over 90% of reddit apps won't have to pay a penny, and neither will the remaining 10% if they fix their inefficiency problems?

https://www.reddit.com/r/reddit/comments/145bram/addressing_...

> when they've made completely clear that over 90% of reddit apps won't have to pay a penny

Because the distribution of Reddit apps is not linear and if 90% of the long tail of Reddit apps (that nobody uses) are not affected that doesn't really change things.

Besides the difference between "all" and "10%", there's also the difference between "kill" and "require to fix gross inefficiencies".

These apps aren't grossly inefficient though, they've been substantially lower than the 84k per user per day API limit that reddit has held in the past for these applications, typically being below 500 requests per user per day. Even reddit's official app uses more API requests per day.

86,400 requests per day is Reddit’s historic per-user limit, but the number was picked as a maximum, not an average or a number any significant fraction of users would be hitting any significant fraction of the time. An app aggregating many users, if reasonably efficient, would be expected to have an average per-user rate over time much much lower than the single-user, single-moment absolute limit.

> not an average or a number any significant fraction of users would be hitting any significant fraction of the time

Mhm and no major third party reddit app is going anywhere near that with their 500 requests per day per user.

> An app aggregating many users, if reasonably efficient, would be expected to have an average per-user rate over time much much lower than the single-user, single-moment absolute limit.

The per user rate of these apps is 500 a day, which is reasonably efficient. Your point of them being "grossly inefficient" is just false.

Do you have a source for this claim other than spez himself, who has proven to be willing to, uh, let's say bend the truth?

Who would you consider a sufficiently authoritative source?

The developers themselves are every bit as authoritative on this topic while also not having been caught out lying. Given that, you'll forgive me if I choose not to put my chips with Reddit management.

I choose to trust the guy[0] who's been open in his communication and break down exactly why the new reddit API changes are forcing him to shut down his 3rd party app.

[0] https://web.archive.org/web/20230613060536/https://old.reddi...

Those two claims aren’t mutually exclusive.

Could you elaborate?

I'm not going to claim anything either way about the efficiency of these apps nor the potential (or lack thereof) to improve; but if they're shutting down rather than complying, this suggests that any efficiency improvements are not low hanging fruit.

Of course, now I'm thinking of that time a coworker insisted that $feature couldn't possibly be made more efficient and it was absolutely impossible to optimise it to take less than 20 minutes to complete, and I spent a few hours looking at what it was doing and got it down to 200 milliseconds…

…but I can't project random success anecdotes like that onto projects I'm not being paid to work on.

I’m interested specifically in Apollo, one of the most popular iOS Reddit apps, and the one that I use. Conveniently, its backend was recently open sourced [1] so how it uses the API is public.

Please cite one example of an “inefficiency problem” that Apollo could fix to put it on the path toward not “pay[ing] a penny.”

[1] https://github.com/christianselig/apollo-backend

Just at a glance, and maybe I'm reading it wrong, but this appears to be issuing six requests every five seconds:

https://github.com/christianselig/apollo-backend/blob/main/i...