Google DeepMind's first deal with the NHS was illegal, UK data regulator rules
businessinsider.com
businessinsider.com
Where I work, we've spent nearly 2 years trying to get approval to collect samples (with patient consent) that are being thrown away. No one on the ethics board wants to sign off on it (maybe because they are afraid of being blamed if something bad happens?). We agreed to pay ridiculous fees to de-identify samples and are crossing our fingers that we will finally get the samples.
I am glad to see that Google got access to this data and is not being punished. There has to be balance so that medical science can move forward at a reasonable rate
No there doesn't. If I don't want my personal information being provided to Google by the NHS it shouldn't happen. In a country where private healthcare isn't an option for most, the only option (the NHS) shouldn't be giving personal data to private, foreign companies, in breach of the law. 'It'll make my job easier' is awful reasoning for violating privacy and the law.
Obviously as you said you're in the US and it sounds like your situation is very different (you haven't identified a law stopping your work just an ethics board that won't sign off for a reason you don't know).
Remember that what has been decided, and what is actually the case, are not always the same. That's why people advocate for changing the law. OP thinks the law is too restrictive; saying "well that's what the law is" is not much of a reply.
One directly causes harm to people. The other _potentially_ causes harm, only if that data gets into the wrong hands and/or is misused. I think saving more lives is worth considering the tradeoff and striking a balance somewhere in the middle (e.g. gathering consent, opting in/out, transparently seeing who what data is shared with, etc).
It is not that.
It’s "giving all information that exists about your medical history to a corporation that already tracks 70% of all payment processes and 96% of website accesses, and which aims to fully control your life". That’s a 1984 scenario if I’ve ever seen one.
I want my medical history to be analyzed, especially in aggregate of everyone else's: to find anomalies, develop new medications, advance science, better understand the genome, and a myriad other "positive" outcomes that come from a system like this. I don't care if the NHS builds their own data crunching system or they contract it out to someone else (although in this case I'd lean towards preferring someone else with a track record in ML already, e.g. a leader like Google) -- what's most important is the medical advances that are all but guaranteed to come from it, not the potential threats that could come from its potential misuse.
It can be argued that Google, specifically, is too biased to handle the data (in which case, it sounds like they just need to be more transparent with how they're using it). It could be argued that it could fall into malicious hands that would blackmail or do "evil" things like raise premiums (to which transparency and security are, obviously, very important). It could be argued that the chance of misuse outweighs the chance of it doing any good, and so on. Those are good arguments, and address vulnerabilities in a system that shares extremely private data, and each concrete criticism opens up the conversation to how to improve the approach. Just saying it'll be like 1984 isn't very helpful to argue. How will it be like 1984? How can we address that?
If technology can be used for good _and_ bad, we put checks in to make sure it's used for good and not used for bad; we (as a species) don't avoid creating it altogether. Transparency, security, accountability, verifiability, etc are all problems we've solved in similar situations throughout history to make sure a system benefits everyone, even when there is a risk of bad players. I assume you have a bank account, even though a bank manager could just bag up all the money and walk out with it any day (or not).
Not to be cliche, but: with great power comes great responsibility. Utopian goals like "solving disease" are always going to come with the risk of someone misusing some new panacea-enabling technology to spread an even worse disease. We should be making sure technology is properly checked and balanced so we can actually use it to do good, not fighting technology (or advancements in health or technology) from existing in the first place because of fears of "what if".
Independent research institutes, funded by endowments, taxes or donations should be able to.
But for-profit companies not, because they will find a way to abuse the data.
I’ve been running IRC bouncers for quite a while, and running stat services. Just the data you can correlate from that – when are users active, what’s their ping at what times, what’s their probable timezones, when do they go afk – allows you to correlate the location of users down to a town, without even having access to the IP. With access to that, it gets even better. Merely with IRC logs you can find out who knows whom, or interacts with whom, if you have detailed enough message logs. This is enough that most people get creeped out if you show them what you can correlate merely from that, and consider it a violation of privacy.
Google is orders of magnitudes worse. The data that you can correlate simply from what Google already has about you – I’ve decompiled quite a few Google applications for research purposes, and analyzed their analytics code – is enough to deanonymize any pseudonymous data. Better yet, it allows you to track where a person is at any time, who they know and talk with, what they read, see, what they listen to. What they pay money for, and who they love.
This dataset is already extremely extensive, and it’s in control of a private corporation that frequently violates antitrust laws to gain more and more power in more and more markets, and whose former CEO has said "privacy is dead". This power is far too much for any company to control, and it should be under democratic control, and not available to people who have something to gain from abusing it.
My point being that your twisting of my words to the point they are unrecognisable is silly. Some people may look at my policing suggestion as a positive you look at it as a negative. Same thing applies to privacy. You may trust doctors (and corporations) and mistrust the police, others feel the opposite way - or trust neither.
I also recognize that this might be a false binary, that streamlining the process might be orthogonal to maintaining privacy, but for the sake of argument let's assume they are not totally orthogonal.
I also don't think it's a debate we really need. More and more people are wearing health tracking devices. There have been research studies done through mobile apps. People are tracking health data themselves more and more and a lot of those people are going to be willing to share it. Let it happen naturally instead of opening the floodgates and leaving people without a choice.
You cannot only infer everything aspect of health from a single ultrasound scan, which is far more detailed. So what makes you think it would be possible with an App and a heart rate sensor?
Imagine being able to have 10x more patients for a drug effectiveness study, or being able to look at the entire patient population for adverse drug reactions/interactions. All the datasets we have for this sort of thing are currently pretty poor. Despite this, the FDA mines datasets like the Adverse Events Reports database [1] to identify drugs that are performing poorly. With better, more complete data, the FDA could far more quickly identify adverse drug interactions and drugs that are harming patients more than they are helping.
Medicine is inherently about rare events. Most of us are mostly healthy most of the time. More data by definition helps you find those rare events when someone is sick or are risk. As we start to ask more difficult questions about co-occurring events, cancer detection or genome associations, our statistical tools become powerless to find the weak signals in the data. The only real solution is vastly more data.
[1] https://www.fda.gov/drugs/guidancecomplianceregulatoryinform...
Personally I don't mind Google or similar seeing my not very interesting data.
Patient consent would make the data much less meaningful as it would remove datasets in a non-random way (better educated people might more likely opt-out). So that's no good solution either. I had recently a doctors visit and was asked exactly that. I opted out as I have no control over the data when released.
The reality is, the more data we have on people the more we can help people make better decisions across their lives. If aggregate medical data identifies a way to save lives or help curb waste/reduce C02 emissions etc... then we should be doing that.
DeepMind didn't know what they were doing was illegal. It took the ICO a year to make a decision and they set the punishment to zero, so apparently they didn't really know either. It still isn't obvious even in retrospect - the NHS itself agreed to the data transfer, and they were the owners of the data, so presumably it was the NHS that broke the law rather than Google.
With respect to patient confidentiality, don't blow it out of proportion. The NHS does not have strong controls on patient data. Breaches are common:
http://www.dailymail.co.uk/news/article-2833960/NHS-data-sno...
The FOI data discussed there is of course a minimum. Real breach levels are surely far higher given that records are mostly still stored on paper (no real way to know how many people see it).
The NHS could protect data much more strongly than it does, but it chooses to invest its time and money elsewhere, which is probably the right decision.
Finally, leaks of patient data don't kill people, but absence of life-saving drugs that could have been developed via better data do. It's easy to argue that the default should be to make large-scale datasets available to drug and tech firms. The cost/benefit ratio seems extremely positive.
How is that an excuse? From a company with hordes of lawyers? Dealing with privacy issues all the time? I'm not even sure Occam would point to a honest mistake either.
The agreement — quietly signed in September 2015 and revealed in full by New Scientist in April 2016 — gave the Google-owned artificial intelligence (AI) lab access to 1.6 million NHS patient records across three North London hospitals without patient's prior knowledge.
No it isn't. For one thing health is devolved in all four UK nations, so there is a separate NHS for Scotland, NI, Wales, and England.
And for England the Department for Health is the governmental department, and that's not NHS England. And NHS England isn't "The NHS" - the NHS in England is a bunch of clinical commissioning groups, and acute trusts, and mental health trusts, and GPs, and private providers selling service to the NHS (see eg lots of secure units, or lots of eating disorder inpatient provision).
In this story one NHS Hospital Trust (10,000 people employed by that trust, vs 1.7m by the rest of the NHS) made the decision, independently of NHS England or the Department of Health.
Perhaps you'd like to have chat with the people on that ethics board, you might learn a thing or two. The medical science can move forward without disregarding people's rights.
The U.S. is nowhere near that. So, no data for collective you -- except when you rip it from my fear-clenched hands, regardless.
Think I'm speaking in the abstract? In my 20's -- a few decades ago -- I had to purchase individual insurance. I had one -- one -- appointment with a psychologist in the prior two years of medical history then requested. A "touchbase" from some childhood counseling.
Fortunately, a friend of a friend was able to "unofficially" run my insurance application by the underwriting team. If it had been "officially" processed, the result would have ended up in a shared database of application results. (Yes, the insurance companies in the U.S. have long shared their underwriting decisions with each other -- surprise!)
The word back on my application: Denied! Why? (Again, I had an inside track on the process not normally available to the applicant.) Because of that one -- one -- counseling visit, in isolation and with no attendant diagnosis.
I learned my lesson. I will never share such data, that can be used against me, without ironclad assurances. And, as the past couple of decades' experience, both direct and learning by reading and observing the news and other cases, have taught me, such assurance is pretty much impossible to achieve -- even commitments made in law are taken back as soon as power shifts.
The one hope for this changing that I see? Universal health care coverage. Because then, at a societal level, you are covered regardless. Your individual history no longer plays a role in whether you are covered.
I hope that the logical, scientific side of medical science starts to see things this way: That the future, expansion, and greater facility and efficacy of their work, depends on it.
Hospitals routinely get hacked by the most bottom of the barrel mass malware. I doubt small practices are any better, though I guess they may still have paper files.
I had a work experience gig for a week when I was ~14 in a hospital, and in an effort to try to occupy me they gave me an access database on 50k people including things like HIV status to mess around with.
What’s so hard to understand about following laws? And why are you implying that the reason people are concerned is due to the safety of their data? It doesn’t matter if Google has their data, or a shitty hackable hospital has it, it shouldn’t be used in any clinical studies without approval.
Given DeepMind didn't do any trials, I'm not sure how that is relevant.
Labelling statistical analysis as a "trial" seems like it's own farce.
I'd argue the regulators are also to blame.
I'm sure you can imagine ways how they can abuse it from making shadow profiles, target people with med ads, not to mention they are having leverage on the competition not having access to the same data.
If only that. A close acquaintance of mine who I won't name worked in a big hospital in Poland for a while, and told me some horror stories related to data practices; I think it's enough to say that they could take out a lot of patient data, even by accident, and no one would notice.
Given the ease of it happening, I'm pretty sure the data is already circulating somewhere.
Maybe I'm crazy, but I think my data in the hands of Google would be more harmful to me. I mean, next think you know my health insurer in some not very developed country that I have to move to for work, comes to find out about my past medical history and raises my premium and deductible. I find it more plausible that they would get the information from Google than from some shady Bulgarian cyber mafia.
Not if the Bulgarian cyber mafia hacks Google to steal that data. Okay, Google has super-ultra security and it's never been hacked (a false statement), but even so, how many companies have Google's security? I think you can barely count them on one hand. I completely expect drug cartels, for instance, to hack into organizations holding user data any time they want. If Google is easily allowed access to this data, then a dozen other companies with much weaker security will also be allowed access to it.
The message that tech workers will take away from this sort of thing is "don't try to work with the NHS or do anything innovative in healthcare". DeepMind didn't appear to have any actual business plan in mind for Streams - all the stuff I read from them on this project implied it was driven simply by a desire to help the UK's national treasure/obsession. Being owned by Google does let important execs do pet projects like this so I don't see any need for conspiracy theories here.
Both the ICO and the NHS are part of the government and yet they reached different decisions about the same laws, I expect DeepMind had their own legal advice which also concluded it was OK. It is unlikely any other company or startup could get a hard-and-fast 100% guaranteed correct deal with the NHS to do things with patient data.
It's doubly unfortunate because the NHS has deeply dysfunctional IT and badly needs the assistance of data specialists. It faces huge challenges to become more efficient and shows little ability to do so. Healthcare records are still passed around on paper, the biggest IT failure in history was an attempt to digitise the NHS, medical data leaks all over the place and is guarded far more loosely than Google would ever consider, many genetic diseases are fairly rare and need a lot of data analysis to make progress on, and so on. Healthcare and the tech industry should be a match made in heaven. Instead the very, very few attempts to dance together inevitably seem to end badly.
Even with explicit consent from patients (and this article states this was not the case), the rules make it very hard to work effectively with the data. I can give a couple of examples.
[The 100'000 genomes project](https://www.genomicsengland.co.uk/the-100000-genomes-project...) is in the process of sequencing 100'000 genomes of patients with cancer and rare disease and will be the largest sequencing project in the world once completed. Because whole genome data can potentially identify a person and give information on conditions they have or might be prone to, their genomic data is seen as confidential. As a result, even with explicit patient permission, the data can only be accessed through a remote desktop onto a machine which does not have internet access, which makes working with some genomic tools (which rely on large databases) very problematic. I understand they're working on this, but it is just an example of the 'inconvenience' of abiding by security rules.
Another example, with my own research, is that even with patient consent, the thumbs up from the ethics committee, and being clinically involved with patients (with a corresponding contract with the hospital), I am unable to store patient information on encrypted devices on university computers. Instead this must be stored on NHS computers, even though the analysis occurs on university computers.
Having said all of that, there is a way to get really great data from many patients using NHS IT systems - [the million women study](http://www.millionwomenstudy.org/methods/) successfully did this and have not run into any ethical/regulatory problems. That's because they did it the right way - i.e. it is formally registered as research, permission was asked and received from the regional ethics committee, patients were given information, both by paper and verbally, to ask for consent for the study.
Of course this study was run by Oxford University, who, like other universities, know clearly the pitfalls and hoops of conducting research on patients. This is a whole world of bureaucracy (which has come about for good reasons), which tech companies will have to navigate and abide by if they wish to carry out human research.
They shouldn't, because ICO has been clear that this decision should not deter people from this type of project, and that the modifications needed are relatively minor.
https://iconewsblog.wordpress.com/2017/07/03/four-lessons-nh...
https://ico.org.uk/about-the-ico/news-and-events/news-and-bl...
> and the NHS
It wasn't "The NHS" (eg NHS England) but a single hospital trust. Many other NHS trusts and Caldicott Guardians expressed concern.
Of course in a hundred year or so China will itself be paralyzed by regulations and some other part of the world will take over.
So I'm not sure we want to go in that direction anyway...
http://www.ibtimes.com/china-use-big-data-rate-citizens-new-...
http://boingboing.net/2017/02/16/chinas-citizen-scores-us.ht...
No it shouldn't. It should be a personal decision for each individual.
What the hell? Why do we let Google get away with this.
> We aim to help organisations comply with PECR and promote good practice by offering advice and guidance. We will take enforcement action against organisations that persistently ignore their obligations.
I don't think that the first step they typically seem to take then is punishment, but steps to bring whatever it is the company is doing back in line with the regulations.
The trust is required to:
> establish a proper legal basis under the Data Protection Act for the Google DeepMind project and for any future trials;
> set out how it will comply with its duty of confidence to patients in any future trial involving personal data;
> complete a privacy impact assessment, including specific steps to ensure transparency; and
> commission an audit of the trial, the results of which will be shared with the Information Commissioner, and which the Commissioner will have the right to publish as she sees appropriate.
edit - ICO site https://ico.org.uk/about-the-ico/news-and-events/news-and-bl...
Edit - some more details about the case
> In terms of the technical security of the dataset, it is understood that the data is subject to encryption at rest and whilst in transit. It is also understood that the Royal Free has received confirmation from the appropriate body that approval had been obtained for the Logical Connection Architecture for the transfer of data, and that the hosting location has been confirmed as compliant with two relevant Information Security Standards2 . On this basis, the Commissioner accepts that there is no current evidence that the data has or will be at risk of processing by an unauthorised third party.
> - It is further understood that all access to raw patient-identifiable data by DeepMind staff as part of the system administration is carefully logged in an audit trail, and is only carried out under the instruction of the Royal Free as part of the data processing. The ICO would like to make it clear that for as long as the data remains in DeepMind’s or indeed any third parties possession, appropriate audit trails, logs and restrictive access provisions should be in place;
https://ico.org.uk/media/action-weve-taken/undertakings/2014...
I think the level of the breaches, security of the data and how far things have been taken beyond what is legal make a big difference in what they've decided.
So Google signed a deal with a government agency that was approved by said agency and they should be fined for that? Wow.
There are several layers between the Royal Free and government.
IIRC you have to have a license to work with certain types of data, that person/those people need to lose grades to the point they have no controlling decision with respect to personal data. It shouldn't be up to their colleagues whether they get held responsible.
I work in a craft studio and face a large fine - so I'm told - if I reveal personal data, or make that possible ... because perish the thought someone could learn you painted a piece of pottery, but reveal personal health information against patient wishes and nothing happens?
>> I work in a craft studio and face a large fine - so I'm told - if I reveal personal data, or make that possible
I'm guessing you're subject to the exact same process the NHS has went through here. The only reason you would be more likely to receive the fine is because it makes sense as you're not funded by the taxpayer. Fining the NHS is in effect fining the taxpayer for breaching the rights of the taxpayer. It's the same reason the police and other public bodies don't often get fined.
We especially do not want that in NHS organisations[1] - we want people to freely and openly admit to mistakes made.
[1] I gently dislike it when people say "The NHS" - that's not a thing, and it hasn't existed for many years now. There are 4 NHSs in the UK, and each of those is made up of a bunch of smaller organisations that have little connection to the rest.
In this situation "The NHS" did nothing. One hospital trust made this decision. Many other NHS trusts (and the Caldicott Guardians) were alarmed by the decision.
That is highly questionable.
This has an advantage when action eventually is taken: the offender finds it harder to claim to not know what the law is, or that their behaviour is compliant with law.
Edit: Also, the fact that people are actually OK with a corporate entity free access to our medical records is appalling. Especially on a website such as this. I can't believe people are actually defending this.
HEALTH: https://ico.org.uk/action-weve-taken/enforcement/?facet_type...
LOCAL GOVERNMENT: https://ico.org.uk/action-weve-taken/enforcement/?facet_type...
Weaker than who's?
Oh wow. How can any Google executive still be ignorant about the privacy implications of its data mining and tracking technologies? That seems unbelievable to me.
https://www.theguardian.com/technology/2017/jul/03/google-de...
With such mentality, people should really question just how seriously Google takes the privacy of their email, Drive docs, and so on, if they don't even consider the privacy implications of using your data when building their data mining tools.
The list of legislation is here: https://ico.org.uk/about-the-ico/what-we-do/
- allow for cryptographically safe usage of the bundles for research (with some limitations?)
- make exact mapping of the health data bundle to a particular person impractical?
Considering the miracles of public-key cryptography and blockchains, it would seem to me that there could be a scheme that would allow for something like this.