The 451 status code is now supported | Hacker News Reader

The 451 status code is now supported

developer.github.com

421 pointscujanovic77 comments

Besides the reference to Fahrenheit 451 the referece to Life of Brian at the example from the RFC made my day:

https://tools.ietf.org/html/rfc7725#section-3

"Unavailable For Legal Reasons

This request may not be serviced in the Roman Province of Judea due to the Lex Julia Majestatis, which disallows access to resources hosted on servers deemed to be operated by the People's Front of Judea."

I wonder if China would respond favorably to the code 451 (which would be ironic) or if they would take offense at it. If something is to be censored, wouldn't the censor want to deny its existence?

Imagine trying to watch a movie critical of China in China--status code 451.

China doesn't try to hide the fact that they censor a huge chunk of the Internet. In fact, they seem rather proud of it.

Perhaps they could even use 451 as an easy way to detect potential "terrorists". Sort citizens by the number of 451s they generate each month, send a cop to the top 1% of them, and publish this fact to scare the other 99%. They're already using TCP resets as a core component of their censorship regime, so I wouldn't be surprised if this Internet standard got co-opted, too.

So instead of:

    You are not in the sudoers file. This incident will be reported.

They'll get:

    HTTP 451. This incident will be reported.

This could be really useful. If this was done by other big content sites (Youtube for example) then a search bot could build up an index of banned resources. A repository of burned books.

YouTube also bans content based on your location (e.g. In Germany), so the index would have to take that into account.

That actually makes for a nice usecase: correlate bans between geographic regions. You just need to access the same content from multiple locations to build a better index.

I really wanted to use it when our German customer try to view product pages for games that are illegal in Germany, but the business people said no.

You probably meant an advertising ban to under 18 year olds (no, a form asking for your birthday isn't enough). Only very few games are actually banned, KZ manager (https://en.wikipedia.org/wiki/KZ_Manager) for instance. For those only distribution is banned, possession is legal.

>You probably meant an advertising ban to under 18 year olds (no, a form asking for your birthday isn't enough).

For a webshop that effectively a ban isn't it?

No?

PayPal, EC Cards, and the eID system can do 18yo verification online.

> but the business people said no

Did they not want to be associated with blatant censorship? That sounds pejorative but I mean it literally.

I think the issue was that you basically can't even allow the Germans know that the games exist. We can't even show the product page and just not all them to buy it.

It is blatant censorship, but it's either that or the German government closing down you operations in Germany.

Can you really not tell them about it, or can you just not advertise it? A product page would probably count as advertisement, but an error code is obviously not.

We've been asked to remove everything, the games can't even show up in a search result.

But you might be right that it's about advertising the product.

If I correctly guessed what you are talking about (https://en.wikipedia.org/wiki/Federal_Department_for_Media_H...) then yes, it's about advertising. That law is about protecting minors from getting in contact with material that is deemed highly dangerous for them (e.g. the game "Manhunt" is on that list). Just telling them in the search results might be enough for them to seek out this game and eventually get it.

Edit: The law actually clearly states that a shop must not have the game on display anywhere a minor could see it

You are by the way the way totally allowed to sell these games. Without any advertising that is. In a classic shop that meant you keep it under the counter and if someone over 18 asks for it by name, you sell it. No idea how that would translate to an online shop.

But long story short: It seems very hard to interpret returning 451 to a url the user already has to know in advance as advertising.

I am pretty sure that if a minor asks for that game in a store you are also allowed to say "no, you are too young for this" and are not required to lie "what, no, never heard of that" ^^.

> In a classic shop that meant you keep it under the counter and if someone over 18 asks for it by name, you sell it. No idea how that would translate to an online shop.

Wouldn't allowing users to search for the game by name be equivalent?

I think the search is more like the shelve.

> No idea how that would translate to an online shop.

That would translate to using the age verification of the eID ;)

> We've been asked to remove everything, the games can't even show up in a search result.

Makes me wonder if we should have a robots.txt per locale.

That's a good point, I didn't think of that. I suppose there is a semantic difference between content that has been taken down globally for DMCA infringement, and content that is blocked in a particular region.

There is some difference, but error codes are made broad to allow for different cases. "Resource is unavailable due to an external legal request" seems to cover both of these cases.

Which for Germany, would be nearly anything with music on it.

Fucking GEMA.

"Responses using this status code SHOULD include an explanation, in the response body, of the details of the legal demand: the party making it, the applicable legislation or regulation, and what classes of person and resource it applies to."

So in the articles example, GitHub should really include who is requesting the DCMA in the response.

While it isn't part of the 451 response, you can see the takedown notice submitted to GitHub. Load the blocked content in a browser and you will see a message with a link to the takedown notice. Here's an example: https://github.com/popcorn-official/popcorn-app

Their link to the DMCA should be included in the response. It's weird that it's not

It is in the body of the response. How do you make out that is is not?

Let me edit my post.

The link to the DMCA should really be included in the api response

That's a really good point. I'll see if I can fix that in the response (I'm hoping I can just easily pluck the DMCA link from the repo).

thanks!

https://en.wikipedia.org/wiki/Fahrenheit_451

Coincidence? I think not!

ha, was thinking the same thing - sure enough, it's totally named after that

I like the Farenheit 451 reference. Is that intentional, or does destiny have a sense of humor?

Edit: Wikipedia knows it all, as always. [1]

[1] https://en.wikipedia.org/wiki/HTTP_451

Original discussion on HN from 4 years ago https://news.ycombinator.com/item?id=4099751

I thought this was an interesting comment[1] in the context of AWS:

>If the .ru site sent 404s for nonexistent users and 451s for real ones, you'd be able to gather potentially useful information. It's like if I go to bad-porn.com and type your email into "forgot my password", it should neither confirm nor deny the existence of your account, simply tell me the request was received. In any event if delivery of the requested resource is legally prohibited, why would I go to the trouble to determine whether the resource exists?

>A final analogy: 10 year old enters US gas station: "Have you Marlboro 100s, menthol?" gas station attendant (without checking whether or not he has this particular brand/style of cigarette): "get out of here, kid. [HTTP/1.1 451 Unavailable For Legal Reasons]."

Amazon will similarly return 404 for S3 assets which exist but which you don't have the right to see. Annoyingly, they will also do the opposite of the cigarette analogy in the AWS console: they will gladly let you walk though the whole process of configuring and launching an EC2 server and only reject you at the very end because you don't have permission. Hence my "AWS bar joke":

http://blog.tyrannyofthemouse.com/2016/02/some-of-my-geeky-t...

[1] https://news.ycombinator.com/item?id=4100526

I was under the impression that the 451 status code should be used for requests blocked by proxies, where the original content is technically still available at the source but blocked for some reason. Probably got the wrong idea.

To clarify this:

> This status code indicates that the server is subject to legal restrictions which prevent it servicing the request.

The "server" here could be a proxy server or the original, hosting server.

> The use of the 451 status code implies neither the existence nor non-existence of the resource named in the request. That is to say, it is possible that if the legal restriction were removed, a request for the resource might still not succeed.

It's like IETF's own Glomar status code.

Some of the original discussions around the status code referred to proxies but adoption at the moment seems to be mostly but hosting platforms like github, wordpress.com [1] etc.

I think the original ideal scenario was that a 451 would be generated by the in-country blocks that get put in place by ISPs due to legal requests from organisations.

However, they aren't incentivised to do this as much as hosting platforms are and so I don't think we will see large adoption there.

Instead, hosting platforms are using the status code both for DMCAs and other legal requests where the content may only be blocked for certain countries as part of a pragmatic response that keeps the rest of the service up in those countries (In country blocks are usually overly heavy handed :)).

[1] https://transparency.automattic.com/2016/01/05/error-451-una...

Help Me Understand:

I am a government who is censoring content. I do not like the explicitly saying I am 'censoring' the internet I instruct my infrastructure not to use the status code 451. and I instruct my nation's infrastructure to reject or rewrite all responses with 451 status code to 404.

What stops me?

Nothing. This is not some kind of mandate. What the general public gets out of it is that, for sites nice enough to use it, the viewer will know why they cannot view something. The alternative is for them to not know why.

You are correct that censors are the least likely to use this code, but for sites that are being censored by the content controller, this is better than 404 "this content never existed". Instead it says 451 "this content did exist and probably exists somewhere else if you hurry and keep searching"

TLS.

A whistleblower.

Nothing. In fact, if I were a censoring country, I'd just look at who emits a 451 and take appropriate steps toward the site owner.

This is political bullshit. I really wish the standards body would stick to strictly technical issues and leave the politics to the individual members on their own time.

If I understand NSL correctly, its existence cannot be published without a government waiver? So in the case a repo needs to be taken down due to a NSL, what does GH do? 404? 401? 451? Returning 451 in response to a NSL would definitely violate NSL requirements?

Probably 404 would best fit the intention of an NSL, although I think 400 would also be defensible.

This means webcrawlers/bots can now compile stats.

I actually saw the status code 451 in the wild.

First time I saw it was in December and after that in January, both on the same site. The site that was blocked was archive.is.

This block was targeted at Finland and none of the different Internet connections I tried could get to the site, I tried my home connection, cellular and connecting from my school network. It's a shame that anyone even thought of censoring such an useful tool for history and other legitimate uses. I wrote a thing about it to a Finnish newspaper and a few weeks after that the block was gone. I suspect that the newspaper conatcted archive.is and it was removed so they don't get bad publicity.

It was kind of ironical that I had to subvert the archive.is censorship to read an archived version of a thread discussing web censorship in Sweden.

I think this error code is a bad idea as it legitimizes censorship.

Just to clarify: Archive.is decided to block all of Finland not because of some Finnish legal requirement, but because the guy running it had a bad experience in Finnish customs and wanted to have revenge of some sort. Not really censorship.

What a hilarious misuse of power. I hope my country doesn't piss off wikipedia or something.

And misuse of status code too. Rejecting a country because you had a bad experience with them should just be the general 403 Forbidden.

Would've been better to use 418 if he was just being petty.

The motivation for censorship doesn't really change the result. 451 is at least more honest than 403 would have been.

That doesn't sound like a good use of HTTP 451.

I don't think it legitimizes censorship. Rather, it makes censorship obvious and unmistakable. A generic status code (like 403) is indistinguishable from a technical error and obfuscates the existence of censorship.

A devious censor probably would not explicitly call out their censorship with a standard HTTP code.

It seems undeniable that this "feature" codifies censorship, making it more "acceptable".

"Oh, well they used the right code anyway"

back to approved activities

IIRC, the general rationale behind the 451 RFC was that this provides a means for a third party to communicate that they are censoring the content, presumably begrudgingly.

So, in the case at hand, GitHub is using 451 to communicate that a government censor has required that they take some content down, not that the company itself has decided to remove the data of its own volition.

If you send a DMCA takedown notice, you don't get to choose the status code that gets displayed after the content is removed.

I think the value of the status code is that it clearly calls out where censorship is happening. If a different status code was used, then you wouldn't know that it was due to censorship.

Interesting thing is, there wasn't any legal requests or censorship attempts by Finnish government. The maintainer of the website was angry at Finnish customs or something. (There was a slightly incoherent rantish post about that purportedly written by him floating around the internet but I'm not sure if it really was the person in question.)

edit. language + explanation.

Exactly.

Censorship (external legal request) exists and this is a fact. The mere engineering projection of it does not strengthen it. On the contrary, through the 451 status code we can know that you cannot have access due to an "external legal request".

Do you think you'd have found out and been able to do anything about it if you hadn't gotten that code? Legitimizing censorship is obviously not desirable, but having it masquerade as generic 404's doesn't solve the problem either, it just hides it.

> Do you think you'd have found out and been able to do anything about it if you hadn't gotten that code?

Yes because I really wanted to see that thing. If I get 404 on something I want to see I will try through Tor and I will try with at least a few circuits. And if Tor gets through I then usually test with other networks just as I did with this.

I know that many people probably don't do that and you have a valid point with the masking but a better solution would be stopping censorship, not a new errorcode

Well, I suppose the point is, we're not going to stop censorship with a vote from IETF.

But, we can enable Internet companies to better-expose when rights requests and censorship has been used to block information.

Before, all you'd get on GitHub (per the release) was a 403, which could have been any number of things and was, in any case, an obfuscation of why you couldn't access the resource. If a person doesn't see a reason to investigate, they'll just move on.

This gets us closer to conversations about censorship.

If I visit a website and get a 404 error, I'll assume that the website is down or my URL is incorrect.

If I get a 451 error, I'll know it's censorship and can use something like Tor to get around it.

Actually, I'm sure there will be some browser extensions soon to automatically use a proxy if a 451 error is returned.

But there's the rub: most of the time, most people aren't looking hard for something they know to exist. They are just casually passing through. The 451 is more about raising the alarm for that much larger group.

Stopping censorship would be good, but there are a lot of DMCA takedowns which are rightfully. Censorship is more complicated.

This status-code is a great solution to inform the public about it.

The problem with the DMCA takedown process is that it makes it easy to get information removed without having to prove anything, so it does get used for censorship (and harassment of competitors and to force anonymous speakers to choose between being silenced or revealing their identities in a counter-notice and ...)

It's pretty much the go-to thing for anybody nefarious who wants to have something censored without all the trouble and blatant impossibility of having their claims vindicated in court.

This code is meant for censorship, not DMCA. Replacing the content with "sorry, this thing broke copyright laws" is how you deal with DMCA, not stopping people from accessing whole sites.

It is not! Read the RFC - http://tools.ietf.org/html/rfc7725 - helpfully entitled "An HTTP Status Code to Report Legal Obstacles."

Those obstacles could be censorship, or copyright infringement, or illegal mathematics - anything.

That's what I was thinking as well. DMCA is just an excuse. Github already blocked content at the request of Russian government, and I'm sure they will block more stuff that they're not legally required to remove in the future.

Er, citation required? IME they do what's required by U.S. law and have previously been fairly principled about not complying with extrajudicial requests.

https://en.wikipedia.org/wiki/Censorship_of_GitHub#Russia

A 500 error doesn't legitimize bad server side code.

I don't think it legitimizes anything. Without it you'd just know the page was removed and not have any idea it was due to censorship.

I agree. Censorship hidden behind a 403 (or national security letter, for that matter) is worse than censorship for all to see. It's our response to it that legitimizes it or not.

451 legitimizes censorship about as much as 403 legitimizes hacking. Which is to say, not at all.

Reply on news.ycombinator.com