You Can't Use the Live UK Train Data Without Accepting a Gagging Clause | Hacker News Reader

You Can't Use the Live UK Train Data Without Accepting a Gagging Clause

mocko.org.uk

128 pointsmocko35 comments

people forget that ATOC exists to protect and further the interests of their members: the privately owned Train Operating Companies -- they don't owe random developers anything.

Network Rail on the other had are (basically) state owned, and their extremely comprehensive feeds are now completely open: http://www.networkrail.co.uk/data-feeds/

these feeds even include the raw signalling data (you even get events when trains activate level crossings) -- there's nothing stopping you building your own NRE API.

I've been slowly building up a database of signal locations/berths to lat/longs by matching up my physical position (while riding on the train) to the NR feed events, with the idea of being able to produce a much better map of where the trains are in realtime.

> they don't owe random developers anything

IMHO ATOC, like everyone else, owe random developers one thing: to deal with them fairly; and not say one thing but do another. To not say that they're open when they're just giving it lip service.

There's no legal obligation to do that, so why should they? Because of morals? Are your morals demonstrably more correct than theirs? Why should your morals have anything to do with their morals? If you want a company to do something, you either need to go through your representative in our democratic process to suggest a change to the law, withdraw your custom, or shut up. Hoping that they follow your arbitrary morals is bizarre.

So if I understand your argument, it is basically "how can you possibly say that the arbitrary morals of 'fair dealing' is more correct than 'screwing everyone over'? Bizarre. Shut up."

Is that right? Or is there more to it than a simple derailing tactic?

> Or is there more to it than a simple derailing tactic?

I see what you did here.

Derailing the derailer? Maybe so.

That is depressing. Companies are nothing but collection of human beings. One would hope that most humans are not happy to go about their lives lying to other humans on a regular basis.

So yes, companies don't have a moral point of view. But human beings do, the humans who work for and run these companies. And while philosophers have opined extensively about the relativity of morals, I'd hope that there are some values that an overwhelming majority of human kind subscribes to.

> I'd hope that there are some values that an overwhelming majority of human kind subscribes to.

Sure, but "open" in the sense of what many developers perceive as "open" is not one of them.

Remember OpenGroup? That's the way most of the business world did and probably still does perceive the word "open." It needn't by synonymous with "no strings attached."

> There's no legal obligation to do that, so why should they? Because of morals? Are your morals demonstrably more correct than theirs?

The people get the government they deserve. If you demand fairness you might get it. This is why it is important people demand ethical behavior from companies and institutions. The quality of any individual’s own morals has little to do with whether one is entitled to do that or not. And even if it did why should they have to demonstrate it? Can you imagine a world like that: “we’re not going to help you until you demonstrate your morals”

"If you want a company to do something, you either need to go through your representative in our democratic process to suggest a change to the law, withdraw your custom, or shut up."

You forgot the frequently useful response of "make a public fuss until they change things". Which is, incidentally, what's going on here.

I guess I'm not revolutionary enough, but I can't see the problem with this. It makes sense for a private company to protect their reputation, and it makes sense not to provide data to people who run them down in public. They have an agreement to that effect that is provided in advance, and you can opt out be deciding not to participate.

I know I'm supposed to feel rage whenever there are conditions placed on the use of data, but I just can't muster it up in this case. Sorry.

A private company which is funded by companies which are contracted by the government to provide rail service. As a taxpayer subsidised body, they should have a greater degree of accountability.

By that logic, every contractor hired mainly by government would have to behave like a public organization. I don't think that leads to sensible conclusions, and it's redolent of after-the-fact justification.

> By that logic, every contractor hired mainly by government would have to behave like a public organization.

You say that like it's a bad thing. In fact, on major criticism of outsourcing and privatisation is that it becomes way to shelter dubious activities from Freedom-Of-Information requests that would apply if the work was done by a public organization.

Not so. NRE's raison d'être is to provide information to the public about train services funded by the public. There is no aspect of it's purpose that is not connected to taxpayer funding.

> It makes sense for a private company to....

So the railways in the UK used to be run by the state. They were privatised. sorta. On the grounds that we'd get the best of both worlds. Do you think that we did?

If this was a completely private enterprise situation, ATOC probably wouldn't be going through the motions of pretending to be open like they are.

Of course I wouldn't suggest that you get the best of both worlds — it's clear that you don't. I'm just saying that there's no base for outrage here. (And the status of the railroads themselves, as opposed to the ATOC, is only partly relevant here.)

> there's no base for outrage here

IMHO the basis for outrage (if you want to to call it that) is that the privatisation is shown to be somewhere between problematic and "not in the public interest". It's a relevant topic considering the current UK government's goals, and the extent to which they are connected to people who stand to benefit from privatisation.

I remember a while ago I wanted to get the location of things like bus-stops. (after the whole "showusabetterway" waste-of-time site the UK gov created. )

Found the information on a site some and found that I could ONLY use it if I was under contract from a UK local council.

Most of the info sources listed on that site were bound up in huge amounts of red tape / brick walls

Frankly I am not surprised that the trains are in the same situation.

Some things do get better, at least. The database that includes the locations of bus stops (NaPTAN) is now open data: http://data.gov.uk/dataset/naptan

Ahhh, NaPTAN, that was what it was called.

Thank you, I gave up a long time ago with the project that was going to use this data.

Glad that things are getting better at least.

Annoyingly this wasn't always the case. Since privatization this has been the case. Prior to that you could get a TOPS terminal/feed for a nominal fee and do what you want with it.

Now, it's covering arses and nothing more.

It's not just CYA. It's also profit, corporate reputation management and empire-building.

In other words, CYA 'cause if people knew exactly how bad your service was, somebody's head might roll.

Ironically, this is precisely the attitude that prevents companies from improving their product. Such a shame.

That's not ironic; the goal of a company is not to improve their product; its to SELL it. "Profit, corporate reputation management and empire-building" support selling; product improvement may or may not, depending on the market and the improvement.

Improvements are a cost-center.

As someone who had to shutter an app due to legal threats from ATOC, this doesn't surprise me one iota.

This happens with Bus data as well in the UK. Although possibly for different reasons.

I work with live bus time data and before getting access I had to sign an NDA because bus companies are fined by the council for poor performance. With access to that data I can see the percentage of buses that are late and could compromise the business of the companies.

NRE have a stranglehold on the only comprehensive source of real-time passenger train data in the UK. They developed it to power their own website but have recognised in it a way to exert power over anyone who wants to use real time train data and to stifle criticism.

What is referred to by "their website"? Why not just scrape the data?

Presumably National Rail Enquiries. How can you base a commercial application on scraping a website? That is not a sensible business model.

Google do

I'm torn.

While I certainly don't like a 'gagging clause', I can only look with envy at the data you UK guys can get your hand at.

So, maybe it's not perfect, maybe not good enough, but being able to get at that much data in the first place _is_ already quite remarkable imo. I'd love to see a German equivalent, gagging clause or not.

What's the transit data availability situation in Germany?

In the US, various public transit agencies do varyingly good jobs of this. Boston's MBTA has lots of data available, both geo, schedule and real-time, and hasn't made any complaints or tried to stop a site which is at least somewhat critical (http://howfuckedisthet.com), and have been stupendously helpful to me as a developer. New York's MTA built their own system using an open transit API for bus real-time data (http://bustime.mta.info/wiki/Developers/Index). Certain of my interactions with them have been less than efficient, let's say.

I just noticed this page on Transport for London's API docs [1], the summary of which is "Use our data, but don't pretend to be us". +1 for straightforward guidelines like this. For instance, "Please don't... Use our brands - the roundel, unless it's part of a map interface" , but "Please do... Use the words 'London Underground', 'Tube' and 'London bus'".

[1] http://www.tfl.gov.uk/businessandpartners/syndication/17102....

Reply on news.ycombinator.com