Open Rights Group has launched a UK Censorship Monitoring API and Botnet | Hacker News Reader

Open Rights Group has launched a UK Censorship Monitoring API and Botnet

blocked.org.uk

82 pointsnamos34 comments

Open articleView on HN

For the past 8 months we've been working on building a series of probes;

- Android https://play.google.com/store/apps/details?id=uk.org.blocked...

- Python https://github.com/openrightsgroup/OrgProbe

- Raspberry PI images https://github.com/openrightsgroup/lepidopter)

And an open API to power them: https://wiki.openrightsgroup.org/wiki/Censorship_Monitoring_...

This is helping to build a picture of what the UK Governments Internet Blocking/Filtering/Censorship actually looks like.

You can submit URLs to be tested by visiting https://www.Blocked.org.uk by installing the Android app or by tweeting a URL with the hashtag #IsBlocked

Once we receive a URL it is dispatched to all probes over a period of time where it is checked against a know list of blocking methods.

The results are returned back to the API for public evaluation.

For example: https://www.blocked.org.uk/results?url=https%3A%2F%2Fnews.yc...

Please help us build a picture of UK Internet Censorship by adding URLs to be tested, by helping develop new probes (browser extensions, an iPhone client etc etc) or just by spreading the word that https://Blocked.org.uk is available.

Please also see https://www.blocked.org.uk/faq which will explain the level of filtering enabled on some of the probes which will explain why some sites may appear blocked even if you, being on the same ISP, can reach said URL.

Whoaa why the hell is The Brights' Net being blocked by BT??!? Since when is atheism considered "censorable" content in the UK??

This is madness.

https://www.blocked.org.uk/results?url=http://www.the-bright...

Please consider freeing the data and make it downloadable as a simple CSV. I'd like a list of all of the sites blocked by particular ISPs...

Snapshots are made available: https://www.blocked.org.uk/assets/data/alexa_100k_isp_latest...

It would seem like some classification of the blocked URLs plus a suitable extension to use willing proxies would be quite useful as an exercise in making the blocklist completely pointless.

I'd imagine a lot of people that have issues with e.g. running a Tor node would be perfectly happy to offer up proxies offering up various subsets of the blocked sites whitelisted by category.

Have you considered setting a user agent other than Python Requests default? Probably not necessary, but good practice. Also, ironically, the app is not available on any of my devices.

Wow, I had not expected this. My personal blog (which is about technology) is being blocked by one of the ISPs. I don't host porn nor have I ever, so why is it blocked behind their porn filter? Interesting....

https://www.blocked.org.uk/results?url=http://lucb1e.com

Edit: TalkTalk live chat support reports that I am not being blocked. Citation:

> I can confirm that is not blocked by our Adult Filters as your website does not have any adult content.

So it seems blocked.org.uk has a false-positive...

TalkTalk are a completely awful ISP. Not sure what the US equivalent would be, but they're the kind who do promotions of "Broadband for £3/month"-level. You get what you pay for service-wise, but they are also by far the most enthusiastic ISP around when it comes to censorship; they have a spider that follows their users to all URLs they visit to categorise the sites[1] (giving them the nickname StalkStalk), and have been doing this long before the current set of idiots in Parliament forced censorship in (at least since 2011).

I'm in a similar situation, one site I run is blocked both by StalkStalk and BT. No porn, no other 'adult' content; it's deliberately kept PG-13 to use the US ratings.

[1]Check your logs for HuaweiSymantecSpider. It obeys robots.txt though, so a quick "Disallow: /" in robots.txt stops it, although they still grab content from unencrypted pages via DPI.

No HuaweiSymantecSpider in my logs in the past 8 months (grepped access.log), but the site is available over both http and https so they might have classified it using DPI (talk about evil...).

My geek blog and a couple of other similar ones are also blocked by TalkTalk. It seems to be a bulk problem. I have emailed them asking for an explanation.

One talk-talk subscriber has reported that he can see these blogs just fine. Though he does not have the "kid safe" setting turned on, of course. "Blocked" in this context probably means "blocked when using the most restrictive settings".

TalkTalk has more than one filter setting.

Ask them if your site is blocked at the "Kid Safe" level (the default) which is what the probes run on.

BT is blocking my site about my homebrewed beer. That's fun.

Ah I knew something was in the works as Andrews & Arnold were writing about obtaining connections from ISPs to test what was blocked. Great to know what is being censored (if you are not on A&A).

"(if you are not on A&A)."

Or if you're on any of the ISPs who do not have filters, including but not limited to A&A.

Who are the other ISPs who do not have filters?

There is no handy list that I know of.

The big six definitely do have the filters. Some smaller players do too, but a fair few don't. The only people I know for sure are A&A because they put their heads above the parapet and made a stand.

Some simply haven't implemented it because it would be work and they therefore won't until forced to (rather than because of any moral objection) - they'll not make any statement about not doing it as it will look bad if they eventually do.

There may of course be others in A&A's position who haven't done as good a job getting the message out.

I've been checking various websites I go on that are completely fine for kids. It's amazing what's getting blocked. One example according to that website is, TalkTalk blocks "http://www.writingexcuses.com/" It's a podcast that teaches story writing.

Blocking that is insane!

And the blocked website itself is blocked on two ISPs https://www.blocked.org.uk/results?url=http%3A%2F%2Fblocked....

This is with the new government mandated opt-out censorship enabled (except AAISP, who don't offer it) - if that is disabled, most blocked sites would be accessible, except that sites with 'adult content' are still blocked by default on mobile networks (EE, O2, Three), which is an account setting with age verification to enable.

Except every ISP that isn't a mobile operator or a major UK wireline ISP (e.g. BT/TalkTalk/Sky/Virgin). Including Andrews and Arnold, but also including many smaller ISPs like Zen or IDnet or Xilo and others.

A&A have played a massive blinder on the media front, by saying that they aren't doing something that no one is legally requiring them to do (it would be interesting to see what they do if they were actually forced to, though). It's also worth pointing out that the mobile networks have had filters for years, because of Labour government pressure and not because of the "wall of Cameron" or whatever people are calling it.

I also like how the site says "checked on the main UK ISPs" and then includes A&A, which no one would ever say is a "main UK ISP". They're the nichest of the niche. They're an extremely good ISP (although you can get similar quality from others at a fraction of the price), but they're not a "main ISP".

I would guess that A&A have paid towards the ORG or are contributing something and that's why they're getting prominence.

A&A are paying for the ADSL/cable connections (Sky, TalkTalk, BT & Virgin) and are providing an unfiltered ADSL connection for use as a base measure.

AAISP act as the (completely unfiltered) baseline control group in this research.

I think TalkTalk is either using a whitelist or an oversensitive keyword filter.

My personal website is blocked by Talk Talk apparently. I guess compiler writing must be harmful for children.

Here we go. If a lot of harmless or potentially even useful material gets blocked, people who want to use that material will simply opt out or route round the blocking, hence making it pointless.

I kind of hope they overreach drastically for exactly that reason. It'll be a lot easier to get the filters dropped entirely if we can attack them with "useful website X is blocked for no good reason" over and over and over again, than if we have to argue using examples that many people would actually have a problem with.

OK, so I'll be keeping an eye on personal Web sites put up by teachers (of which there are many, a couple of examples[1], [2], neither blocked to my knowledge) that can be accessed from UKERNA provided College connections but become blocked by domestic connections.

[1] http://www.hegartymaths.com/

[2] http://www.themathsteacher.com/

The TalkTalk filter is the default "KidSafe" level

If you test www.blocked.org.uk it doesn't show as blocked so either it changed between scans or there is an issue here. Maybe someone can test.

www.blocked.org.uk vs blocked.org.uk AFAIK

I just learned that one of my personal sites is blocked on BT and TalkTalk... there is zero adult or other inappropriate content on it. Time to complain, I guess, but I'm sure they will just go straight to /dev/null :(

Make sure you update the site to state that it is currently blocked by these ISPs, you don't know why, you have asked them to investigate, but you have not heard nothing back and until they respond it is completely out of your control.

That way anyone who can access it from work but not home (for example) or recommends the site to a friend who finds it doesn't work, will have a clue as to what the problem is (and it isn't your fault).

If your site is of significant general interest then this may also increase the chance of the ISPs acting quickly.

Have you considered legal action?

Nope. It's a medium sized site I don't get any money out of, not linked to any business. I suppose I could theoretically small claims court them, but I'm not sure if I'd win as they can probably piss about with their customers' connections in whatever way they please due to their T&Cs :(

Sad to see that The Best Page In The Universe (http://maddox.xmission.com) seems to be blocked by a non-zero number of larger ISPs.

Obviously all those children need to be protected from harsh reviews of their artwork.

Reply on news.ycombinator.com