Transport for London plans to collect data from passengers' mobiles | Hacker News Reader

Transport for London plans to collect data from passengers' mobiles

news.sky.com

148 pointslnguyen113 comments

That is a pretty misleading headline - this isn't asking to see their text messages or something like that; It's just tracking when and where a MAC address is seen in order to work out traffic trends. In fact some of the data looks really interesting: https://imgur.com/Hx6mDSm.jpg (credit to bcraven for the link)

Most large public WiFi deploys come with this capability already included (albeit normally with an additional license required), whether or not the owners of the system are aware of the capability / are utilising it. Punishing TFL with sensationalist journalism for being open about this application will only make such use in future more hidden and isn't constructive.

The article does seem to be blowing up a relatively small issue.

However, if the intent is to work out traffic trends, it can be done much cleaner. For example, one can only track addresses within a single commute (it should not be difficult to guess from data). That is, use different hashes for the same MAC at different trips.

Also, IMO any general purpose data the government collects for the public benefit that it does not declare sensitive should be public. That is, quickly posted for public to see, use, check, etc. My 2c.

100% agree. And it's unfortunate that articles like this end up being the ones discussed, it actually hurts privacy discussions by moving the conversation away from your "This data could be better anonymized and published" to "the transportation bureau is tracking your every movement".

Don't Android and iOS randomize the MAC addr these days? Are they investing in something that's going to have a very short life?

At least on iOS it is only randomized while scanning, not while connecting to networks you have used before. Which is kind of obvious because otherwise your login won't persist.

You need to login to the TFL WiFi using your Virgin Media or supported phone network's login, so they can identify you by that instead.

Yes, but this is easy to opt-out from. Naturally, this is not what they are doing:

> At the end of 2016, TfL ran a pilot which tracked the Wi-Fi signals from 5.6 million phones as people moved around the London Underground, even if they weren't connected to a Wi-Fi network.

Is that just if you want signal under ground, or is having WiFi turned on a requirement for riding Tfl?

If you want to use their wifi :)

Another way to think about it is as taking advantage of a window of opportunity to gather the data before it disappears.

I can't imagine people getting as mad at it as the news article wants them to be. I'm picturing a world where the super aggregate data* is available via an API so I can have an app that says something like "For a faster journey today use line xyz via station abc"

* in the image's case, those percentages would do it

Doesn't google maps already do that if you request transit directions? It will route you the fastest predicted way. Worked for me on my trip to London.

Have no idea, I'll have to give it a go. Honestly being from a small town who commutes to London sometimes I'd never even considered maps for that sort of thing - Here it works as basically just a map

Having a state run public transport utility track everyone's mobile devices' unique IDs. Huh. You're seriously asking why this might be sensitive?

Wow, I guess you guys in the UK are seriously desensitized to data invasions like that.

All this stuff is tracked all the time anyway through cell towers, it's way past where the regularization point is for "state intrusion" in the UK.

That might seem weird to an outsider but that's just a culture difference. We're just as weirded out that anyone can own even a handgun in the US, let alone walk down the street with one.

I'm personally far more weirded out by the idea that a private company can track and use this data than the fact the state can.

My two major turnoffs for living in the London area (which otherwise, based on my multiple visits so far, seems lovely) are:

a) the pervasive surveillance (i don't think it's by chance that "black mirror" is written by someone - charlie brooker- living there)

b) the stupidly high housing costs

(in that order, actually)

While London does have a lot more surveillance than the rest of the UK, anywhere in the UK has a lot more surveillance than most other comparable countries.

It was 13 years ago that the UK's then information commissioner warned we were sleep-walking into a surveillance state: https://www.theguardian.com/uk/2004/aug/16/britishidentity.f...

The worries seem positively quaint now in comparison to the data that Facebook, Google and the state now collect.

Except they’re not storing any device IDs at all, they’re hashing the IDs with a salt that they rotate and dispose of daily. They can only infer movement of a device through the system, they can’t tie that back to any device after the fact, and can’t even tie the same device’s movements together over the course of more than one day.

The people who choose to spend their lives on cctv? Yep...

I understood them to offer wifi service and track its use, but maybe I was reading too much into it.

The revenue was discussed as coming from targeted advertising implying tracking of data usage.

I'm skeptical for a different reason: all they're going to see is a bunch of connections to facebook servers which will all look alike. /s

"Targeted advertising" in the sense of real-world, physical targeting. Like, where should adverts be placed in the station to maximise exposure.

That doesn't require and collection of data, and they have explicitly ruled out doing such collection.

They already have, e.g. adverts that run in parts down escalators. If you know plenty of people go from one corridor to another, you could extend such segmentation across corridors.

I imagine they can also do differential pricing of adverts across more of the stations with a better idea of footfall (though simple point footfall figures can be got in simpler ways), and indeed that is apparently part of what they're doing.

The 'sensationalism' is not at the technical detail, but TFL being opaque about their motives.

TFL's intention to "dynamically trade advertising space" is far more interesting though. Once that's in place it doesn't matter if TFL track you or not - Facebook and Google will just repurpose their existing technology, making things a step closer to the advertising in minority report.

This will likely result in a strong pressure to reduce the friction of everyone using the wifi - expect the 'paywall' to drop.

Here's the pdf from the pilot findings: http://content.tfl.gov.uk/review-tfl-wifi-pilot.pdf

I like this figure of the ways that people travel between KGX and Waterloo- this is the sort of data that you can't pick up using Oyster cards: https://imgur.com/Hx6mDSm.jpg

The PDF is a great read to understand the projects motivations, data collected and examples of how the data analysis could help anybody using the underground network make better informed routing decisions.

I'm actually happy about this.

The data that has been collected is great (http://content.tfl.gov.uk/review-tfl-wifi-pilot.pdf) and not realistically available in any other way. The only data stored is (hashed) MAC and timestamp. It's taking place on a system where most users are already tracked semi-anonymously using contactless cards or Oyster cards, and is trivially avoidable by disabling wifi.

This seems like a good trade-off.

MAC do not contain enough entropy to be usefully one-way hashed.

MAC address is 48 bits of which 22 bits is vendor.

The proposal, as it stands, is a salted hash, with the salt discarded once a day. If the salt isn't stored anywhere, then there's enough entropy.

Yup, I'm happy with it. Can't see how else you would get this amazing data which is obviously going to be extremely helpful for managing an increasingly overcrowded train system.

Even the article says they're using this to assess the value of advertising spots, rather than some of the more nefarious things that _could_ be done with this technology.

Even here in Germany of which most think that we have good privacy protection laws, mobile phone providers are collecting and selling anonymized movement profiles of their customers. Public transport companies use these to improve their service, as well as companies offering navigation systems.

I would be surprised if tracking via the phone network was not common practice in other countries. In case of London, WiFi would of course enable tracking of people on the Tube, but in general using WiFi tracking should yield a more fine-grained movement profile, which would not be possible with phone networks only in a dense city center.

For those who haven't seen it, here's a great illustration of what that data looked like for one person back in 2009:

http://www.zeit.de/datenschutz/malte-spitz-data-retention

I can imagine that it's only gotten more granular and comprehensive since then.

That site makes me happier about being able to seldom carry my phone. I don't leave it home for privacy reasons, but that's a nice side benefit.

Why do they need to do this? They already track people with Oyster and contactless which gives them journey information. This is just a convenient data grab and profiteering.

I'm going to sound old and curmudgeonly here, but I'm starting to miss the days when we bought paper tickets with cash.

> This is just a convenient data grab and profiteering.

I know people working on this project, and that is absolutely not their motivation. They want to better understand the movement of people within the system, so they can design better stations, trains, and services.

The traditional way of understanding such passenger movements is by parking students with clipboards and people-counters all around stations, doing passenger interviews and such. This traditional methodology is not only hugely expensive, but the data it generates is spectacularly bad, and has lead to some incredibly poor planning at times.

The point that this will be only used to improve services was denied in the article itself:

"What they told people at the time was we're going to use this data to improve services. But now thanks to [Sky News] investigative reporting, we find out that it's partly to improve the services, but also it's to exploit people's data for revenue, doing advertising."

Knowing more accurate footfall counts down individual corridors is essential for better planning of stations and services. And yes, it's also a boon to pricing advertising along those corridors. No way to improve one without improving the other.

But that's literally all they're getting from this: spatially granular and accurate passenger movements. They're not collecting personally identifiable data or even aggregated demographic metadata. Just footfall counts. (Read their policies: they just collect device MAC data, which is hashed with a salt that is destroyed on a daily basis. This allows them to trace MAC addresses through the network without ever being able to trace it to an individual.) But, yes, those footfall counts are relevant to advertisers. So? TFL isn't a profit-making organisation: everything they get from advertising is re-invested in operations and new infrastructure.

Anyhow, you're trusting them with your life when you ride they're vehicles. They do a pretty good job with that. Trusting them with fleeting awareness of your MAC address seems like a much smaller ask.

This just in, our reporter has learned that telling people "secrets" make them share articles with their friends. More to come in our next clickbait article.

Using the data to optimise advertising (and ad pricing) is pretty obvious. Nobody seems to be annoyed that there's advertising in the first place, this is a non-issue.

> I know people working on this project, and that is absolutely not their motivation.

It doesn't really matter the motivation if the system that gets built in the end has the same vulnerabilities and potential for abuse.

Attitudes change, and history shows us that governments have a bad track record for sticking to original motivations. Companies are even worse, once they realize the profit potential of something.

Could you give more details of the kind of information they receive from wifi tracking that they don't already through oyster barriers and how this helps with service designs?

Just guess but: Accurate real-time data of passenger movement through the stations.

Barriers only tell you when they enter/exit, they give you no information about the routes people use, how long it takes them, how many people transfer between lines etc

They want to detect overcrowding of stations which means before people have gone through any barriers

Passenger movements within stations, and between stations where there are often many possible routes.

In most cases Oyster will only tell them where you entered the tube and where you exited. At a few exchange points you might tap out and in again but at most you will not. On a bus it will tell them where you go on but IIRC you don't tap on exit.

Tracking you by WiFi MAC address or similar techniques (I've heard of Bluetooth being used in some places) allows them to see where you went between entry and exit and how long you spent there. It will also allow them to track you outside the network when you are in range of one of their APs. The CityCOnnect WiFi in York tracks people around the city in this way, I'm sure other cities have similar arrangements.

> This is just a convenient data grab and profiteering.

I'm not sure about where direct profits might come from, but IIRC when York's city-wide WiFi started they were quite honest (or at least refreshing undeceptive) about it being a bit fo a data grab. From the council's point of view it helps them to know how people, especially visitors, navigate around the place. This can be a great help when planning changes to pedestrian signage & bus routes, and to reduce disruption when planning maintenance works on paths & roads. Of course the commercial partners will use the same data to assess advertising potentials and other such.

Oyster allows them see entry and exit, but is doesn't show what route the customer took, which lines did they take if a change was required. It also allows TFL to see where people travelled within the station concourse, how long they took, and what print advertising they would have walked past from turnstile to train.

It's notable that there's quite a few places in busy stations where the most direct route between two lines (or from one line to an exit, etc.) isn't signposted, because the footfall between them exceeds the capacity of the (typically older) direct tunnel.

They want to be able to charge advertisers more based on potential viewers of their advertisements - they probably charge a fixed fee / week for ads now, then they can go "Yeah according to our analytics a million people viewed this ad, vs 100K for the other, please pay more".

>profiteering

TfL is a not-for-profit.

So they profit, take their money, then they are not-for-profit.

You do realise this is a government body that runs public transport, right?

Public transport which is notoriously bad at making profits, all around the world.

Sorry, but this comes across as a bit of a silly conspiracy theory, by people who can't be bothered doing basic research.

End result != motivation

Bad at making profit doesn't necessarly means you're not-for-profit.

Also from the article:

TfL IS not-for-profit. I.E. any profit it gets is reinvested in London's transport infrastructure.

Sky News hasn't really done any investigative reporting here, the information is publicly available and most interested people knew about this project years ago.

Is not-for-profit not superflous on a government body supplying 50% of it's income? Who pays for the 6 figure board salaries? Is that from the profit part, or me the tax payer?

Just seems a bit stretched. never-gonna-profit

https://tfl.gov.uk/corporate/about-tfl/how-we-work/corporate...

"Remuneration of board members

The basic fee is £16,000 per annum. Additional fees are paid for each appointment to a Committee or Panel, up to a maximum payment of £20,000 per annum"

https://tfl.gov.uk/corporate/about-tfl/how-we-work/how-we-ar...

25% of their funding seems to come from Government grants. £2.6 billion. That should be covered easy by the tax revenue London generates.

Especially so, since there's apparently a £26.5 billion tax surplus that goes out the capital to less well off regions: https://www.theguardian.com/business/2017/may/23/uk-budget-d...

So what are you complaining about exactly?

For-profit also doesn't fit well in an organisation running a bus network that spends £1.5b a year more than it takes in.

If you understand how people get from A-B (which lines, what paths within stations) you can better control overcrowding.

> Why do they need to do this?

This is explained in detail here:

http://content.tfl.gov.uk/review-tfl-wifi-pilot.pdf

If you would rather buy a paper ticket with cash you can still do that, and you won't be tracked if you put your phone into wireless mode. Personally I'm happy for them to collect limited amounts of data, with restraint, if that means they can improve the effectiveness of the network. But if you're not, that's understandable, and it's not that difficult to avoid it.

Because they don't know where people go on the system. All they know where you entered and exited.

I don't quite believe this because they could have additional readers set up that can read at a distance.

That’s nonsense and not how RFID readers should or realistically will work.

What do you mean should, are you an RF or electrical engineer of sorts?

There have been demonstrations of activists scanning in-clothes rfid identification from 100 meters across on the streets, which were used for store inventory assesment before.

How would tracking the Oyster card ID that's registered to my name, address, and credit card # in their system be an improvement compared to tracking the hashed MAC address of my phone?

Surely this would be an automatic sanction when GDPR comes into effect, leaving your wifi on isn't exactly "explicit consent".

Just for people as uninformed as myself:

the UK will apply GDPR post Brexit http://www.blplaw.com/expert-legal-insights/articles/gdpr-an...

Consent isn't always required for lawful data collection within the GDPR. Article 6 of the GDPR shows other lawful bases for collection

https://ico.org.uk/for-organisations/data-protection-reform/...

Is the Underground a private institution? Is it government? Does GDPR provide exemptions for government institutions? Do the police assist the Underground in performing searches and data collection? Can the police assume "consent" when they collect your Wi-fi MAC or your Bluetooth ID or any other data your mobile device voluntarily broadcasts?

Yes, no, no, no, N/A as GDPR provides escape hatches for actions to do with 'public safety'.

LU is pseudo-private. It falls under TfL, which falls under the London Mayor.

Only access point connection time, hashed MAC address and time are logged.

The only thing that can be found out about anyone, is where they were, at what time, possibly who they were (all of which can be obtained from CCTV footage), and (additionally) how many web requests were made.

No browsing data is stored.

This doesn't particularly change the state of existing LU monitoring.

Agree, this can be obtain from CCTV footage; so why don't them? Why log even more stuff?

I don't know, but using face detection to better analyse travellers seems even creepier that logging mac addresses from phones.

This has been suggested before, but the media exploded - people don't want to have visual recognition systems looking at them.

I guess it's easier to process MAC/Wifi data/numbers than faces.

I believe that some more modern trains allow TfL to measure the weight change of carriages under passenger load, which can be used to estimate numbers, but imagine the cost of trying to roll that out on all trains and platforms... The fares would have to double for a year at least.

I'm sure the Class 345s, soon to appear on Crossrail, do this to show loadings on platform screens (to trying to avoid crush loading in specific coaches while others are empty), but I can't find any source for that. The Class 700s on Thameslink show loadings on platform screens in the "Core" section, but this is based upon CO2 levels within the coaches.

This is why I switch my wifi radio off every time I leave home. Especially when going to travel via the tubes.

And this is why the "50-shades of wifi off" of ios 11 is so idiotic.

Don't have iOS 11 yet so don't actually know how many steps I need to take.

I’m not sure I see the issue with iOS11. 3D Touch Settings and select WiFi and toggle it off. It’s about as much work as opening control center.

If you don’t have a 3D Touch phone you have to open Settings and Navigate to WiFi which is right at the top of the list with Bluetooth and Airplane Mode.

The issue is that Wi-fi isn't really off when you change that setting. See https://news.ycombinator.com/item?id=15382836

The grandparent is describing the way to actually disable WiFi (i.e. not using Control Center)

You just gave me the first real use for 3D Touch since I got my 6S. Thanks!

On the subject, one swipe + one press using control center is really much faster than finding the settings app. For me it takes at least six actions to get to the switch: press home twice, open a folder, scroll, open settings, click wifi). Even if you keep settings in your home screen that's at least four clicks/touches away.

Note that in iOS 11 doing this through the Control Center it will say “Not Connected” which is exactly what it means - the WiFi hardware is still switched on, it just discconnects:

https://support.apple.com/en-gb/HT208086

We get it. We’re talking about turning WiFi off via settings.

I’ve got it on my home screen and it’s as simple as raise to wake while authenticating Touch ID then 3D Touch settings and select WiFi (you don’t have to lift your finger just slide) and tap the toggle.

You can also just ask Siri.

It’s completely contrived one has to have 3D Touch to access such a basic thing now. There is a long press context, with plenty of space in the expanded area for the same setting.

For a minor shortcut? It really only saves you an extra tap or two.

iOS randomises MAC addresses periodically, so it will be difficult to track you this way. This isn't a compelling reason why it's idiotic.

A more informative article was posted earlier last month (https://www.ianvisits.co.uk/blog/2017/09/08/tracking-smartph...), and there was more info on the data privacy elsewhere too (https://www.theregister.co.uk/2017/09/09/london_tube_trackin...) saying that TfL "depersonalized [MAC addresses] using a salt which [they] then discarded at the end of each day."

I used to work with a startup that did this data collection for British Rail at just these stations. It was used for fairly important things like "will this new bridge be sufficient for the passengers who currently walk all the way round to get to platform X. Will it be a crush risk / bottleneck".

A railway station should be through of as a maze where the walls really do move in and out every few minutes. hundreds of signals can appear and disappear just like that.

Its worth mentioning that I am a big supporter of GDPR but I will be one of the first to allow a variety of researchers to have blanket access to this kind of stuff for medical, town planning etc research. I need to look more deeply into that part but there surely must be ways for legitimate research such as this to be conducted?

The thing about slippery slope is that they are really awesome places to live, except for the constant risk of sliding where you don't want to be. The price of that awesomeness is vigilance, to remain precisely at the point on the slope you want to.

They've been collecting data for at least 3 years. It's only the detecting of attempted wifi connections but it's the best data source fro predicting overcrowding of stations. A lot of what they tried out to manage crowding during the Olympics is being refined and used to manage passenger flows so some passengers are sent on different routes.

For an example of a somewhat similar project, see Austin's bluetooth traffic sensor data:

https://github.com/cityofaustin/hack-the-traffic/tree/master...

They used to have the code used to anonymize the data available, but I can't find it. I believe they re-randomized the MAC addresses daily so you could track trips but not a device long term. With enough sensor locations that seems like it could still be vulnerable to de-anonymization to some degree.

Does anyone know if the iPhone's MAC randomization has any effect on this?

As far as I know, the iPhone only randomizes its MAC address after the device has been locked for two minutes and not connected to any known networks. Two things that are very unlikely to happen in a subway. One of the advantages of riding the subway is that you can use a phone and the other one is that there is rarely any cell connection (at least in my subway), so you need WiFi anyways.

WiFi randomization is great if you are going to the store by car because the devices on the way there cannot track you but once you arrive there, check your texts it is game over.

There’s not great WiFi coverage on the underground right now, but I imagine that it will continue to improve, especially if TfL want to collect more of this data...

Really? The vast majority of tube stations have comprehensive coverage. I can't imagine how it could get much better, beyond being available on trains themselves.

It's only unlikely to happen if you actively connect to the tube wifi, otherwise it works as expected, my phone will likely be locked while I get to the station, so I'll get a random MAC...

Your iPhone uses its real MAC address as soon as you unlock your device. Unfortunately, many networks still do security by MAC address, so a random MAC address would break WiFi on many networks.

The only source for this a could find was an old paper on iOS 8, while this more recent one https://arxiv.org/pdf/1703.02874.pdf seems to suggest the randomisation works as intended for unknown networks...

they talk about some great benefit for the passanger but they did not say what it actually is...

This is currently the only method available to get good estimates of # of people on a platform - unless you would prefer visual recognition software.

TfL needs this information to be able to stop overcrowding on platforms.

> unless you would prefer visual recognition software

Given they already have multiple cameras covering platforms from multiple angles, this seems like it would have been on the list.

When TfL have previously suggested visual recognition software, the media exploded. So they took a step back.

I suppose calling it "visual recognition software" invokes a kind of "facial recognition and tracking" vibe. I'm thinking more of the "is this platform crowded" visual recognition stuff - "How much of the reference image of an empty platform do we have today?" kind of thing.

Without recognising single persons, it would be much harder. You would have to consider that a person twice as close to the camera takes up 2x as much room as someone 2x further away. Generally, it can also be very hard to see people on a crowded platform and you can almost double the amount of people on the platform and see as little of the platform after as you did before. Wifi will probably generate better customers in that case. Add in the fact that you are still processing visual data and that you will get a lot of media attention and Londoners will lose trust in your services and you have a definite no-go for your project.

> you can almost double the amount of people on the platform and see as little of the platform after as you did before

I would suggest that "overcrowding" was achieved well before that point though.

> Wifi will probably generate better customers in that case.

Assumes that one person equals one Wifi though which isn't necessarily the case.

> you are still processing visual data and that you will get a lot of media attention

Perhaps. It would definitely require a deft hand on the PR tiller.

1. Not necessarily - 4 short people behind 4 tall people will do the job. But I would argue London platforms ARE overcrowded already, rather often. Mostly because you can only control where people go a certain amount. If everyone gets off many trains at the same stop, but don't leave, there isn't much TfL can do short of asking people to leave for their own safety. Maybe if TfL actually got any decent amount of funding (Like Southern and SouthEastern got due to being rubbish) from the government, they could work on creating further lines and spaces that would decrease Underground congestion. Instead, the government has cut funds and the Mayor has frozen fares. And yet people still want infrastructure to improve.

2. 1 Person != 1 Wifi, but if 1W = 0.2P, you can still get a good estimate.

3. TfL is a large bargaining chip for the mayor - no mayor would even consider risking a PR disaster like the one that could come from this.

how?

They go into a lot of potential benefits in their paper here: http://content.tfl.gov.uk/review-tfl-wifi-pilot.pdf

http://content.tfl.gov.uk/review-tfl-wifi-pilot.pdf

Anyone who hasn't seen this yet should take a look, it explains in detail:

- Why they are doing this - What the potential passenger benefits are - And bonus, it includes lots of cool infographics on passenger journeys

I only connect to their WiFi to email my workmates, and say "sorry I'm runnning late again, ----ing Circle line'.

They can intercept that traffic as often as they like.

They're not logging any wifi traffic.

They're storing a hash of your MAC address and timestamp - the idea is to log foot traffic over time.

I think this issue has been blown up somewhat by people who don't understand the technical issues.

They have already collected the data. The issue now is whether or not to make the data available to third parties, albeit anonymized.

I think a lot of transportation departments have been doing the same thing with car Bluetooth transmitters to monitor traffic patterns in real-time.

Reply on news.ycombinator.com