Babylon Health admits GP app suffered a data breach
bbc.co.uk
bbc.co.uk
(a) promoting a "Virtual assistant doctor" -- glorified series of 20 questions -- that did not spot medical 'red flags' like chest pain, breathlessness at rest, etc; providing dangerous and inappropriate advice
(b) fleecing the NHS on expensive contracts and aggressively marketing "phone first" or "digital first" consultations
(c) "partnering" with NHS Digital and NHS X to build their commercial products upon medical records (this is related to the 'Care.data' programme);
(d) generally speaking, muscling into "core" NHS territory (if you use their app, you used to be automatically de-registered from your real-life GP -- meaning that they got some money for providing GP services instead)
(e) having several high-up executives who have made significant donations to the Tory party, or who are Tory members.
I'm somehow not surprised they've had a data leak.
Everyone I’ve spoken to who uses the Babylon powered NHS GP at Hand service in the UK seems happy enough with it.
They claim a 96% satisfaction score.
Do they know their video chats were recorded and retained?
If you are a contractor living in shared houses and moving every few months, you cannot get this proof of address and cannot access NHS care.
Normal gp surgeries do not seem to want to register patients. They push back really hard at the slightest opportunity.
Babylon seem to want to register patients. They registered me based on my parents address. I could get video appointments, in person appointments near me in different cities all over the UK, and referral to a hospital near where I was working.
After about 10 years of no healthcare in was able to get some chronic issues sorted.
NHS is not perfect, and we shouldn't shit on all innovation.
No. It is helpful for the GP surgery if they can prove your identity and know where you live, but they must not refuse to register you if you do not provide ID. Chapter 8, page 141: https://www.england.nhs.uk/publication/primary-medical-care-...
I know a bunch of GP surgeries tell you that you must provide ID, but they're wrong. I can help you put complaints into the relevant CCG if this is still a problem for you.
They have to offer out-of-hours and home visit services, but they only have to provide these to people who live inside their catchment area.
> Babylon seem to want to register patients.
Initially Babylon were doing the same thing -- refusing to register patients who couldn't provide ID. They were stopped from doing it by user complaints.
I am aware that the regulations should allow GP surgery to register a person with limited ID to register, but my experience was that, even after exchanging emails from practice managers they were not willing to do so.
I was not aware that the CCGs were the right place to push back - will know for next time.
Like I said I'm currently using Babylon, and my experience is that bluntly they refer you for whatever you ask for with little assessment or care on their part. Which is fine by me but would be terrible for my gran.
GP services get paid a flat fee per patient regardless of how much attention that patient needs. Babylon is trying to attract healthy tech savvy patients from across London and is leaving the more challenging patients to local GPs who will now be cash strapped.
They are trying to attract patients to their service of course, that's their business. The NHS hasn't properly recalibrated costs per patients and so it happens that the type of patients Babylon gets (young, tech savvy) are higher margin patients
It's not true that gps get a flat fee, the fees are scaled depending on patient age and other factors, but still aren't in line with costs
The NHS needing to improve efficiencies and fix systemic failings is one thing, allowing a private org to privatize profits (skimming high margin patients) while socializing the losses (dumping patients requiring more care on the NHS) is quite another.
Disclaimer: American, familiar with broken, dysfunctional healthcare. Protect what you have. The alternative is not so good.
They had to drop those because that's not how general practice works in England.
But it's reasonable for people to remember this and to hold Babylon to account for it.
Source: I used to work for them.
(a) is not (entirely) correct. There are literally hundreds of epidemiologists, doctors, NLP engineers, etc that have worked on the models. They are not 20 questions. (I'm definitely not saying its perfect!)
(b) is false. Babylon get paid as any other GP practice does with a fee per patient (on the NHS side). Digital & phone first is GREAT. I've had appointments 7 minutes after opening the app before.
(d) is also not correct, it depends which of their services you use. GP @ Hand is pretty explicit about that being the case - it's the entire value proposition.
If you use the private service (as I do) then you keep your GP (I spoke to mine this morning).
(e) I'm not really sure how this makes it more likely they would have a data leak?
You almost couldn't have got any more of what you said wrong - why the hate?
I am not sure whether recording the sessions and storing them for longer than necessary was a good trade-off compared to the risk of having such data exposed down the line. This breach would've been less damaging if video recordings didn't exist and only symptoms or notes from the consultation were stored.
This is also why I'm so opposed to data collection such as analytics or telemetry; even if all the players involved are acting in good faith there's the risk of a technical error exposing data that wouldn't have been there in the first place if it wasn't for the analytics.
As you say, data can be lost or stolen. But companies also change hands, and it is notoriously hard to prevent it being used for other reasons after acquisition (particularly in the US). Perhaps you can even look at if you actually need names and other details when handling discounts? Could you validate eligibility or do whatever is required, then assign a verified token to it? If it's more complex, a blinded signature might let you attest to a given identity being eligible for a discount, without you being able to look back and check which signature it was. I'm all for finding ways to not store data that isn't strictly necessary.
Everyone calls data the new oil, but I'm over that, and now see it as the new asbestos. It's expensive to have it, expensive to keep it, and expensive to get rid of it (if you do it right)
It sounds more to me like an access control/authentication failure - firstly on listing recordings that belong to another user, secondly on then giving access to those recordings when requested. And arguably also a failure in storing the recordings in an effective "plaintext" meaning they could be retrieved by a user that shouldn't have access - given this is about as sensitive health-related PII as you can get, I'd argue these recordings ought to be encrypted with a per-recording key available from a separate key management system?
I guess this gives rise to a bigger question around whether online GP consultations should be recorded and preserved like this, given this kind of situation was/should be clearly in the threat model, and will be from now on?
Edit to add: From Babylon's website, "All data is encrypted and stored securely to guarantee your privacy" - once again, "non end-to-end encryption" fails to save the day due to a failure of another component of the system, and becomes snake-oil marketing copy.
Yep, I agree. Perusing their careers page, I can see they use ruby and java. It’s so easy to misconfigure resource authorization in a rails app, which is why it’s so important to have solid testing around those components of an app.
> I guess this gives rise to a bigger question around whether online GP consultations should be recorded and preserved like this
I also completely agree here. EMR systems are well established at this point and doctors have somehow gotten this far without needing full recordings of most consultations. It wouldn’t surprise me if they were developing an abridgment system that automatically creates notes for doctors or something, but like you said, storing that training data is such a huge risk.
Sounds like somebody accidentally shipped a bug that didn't lock down permissions properly. Not something any of us wants to happen, that's why we have QA procedures, code reviews, and hopefully somebody independently auditing any new code that touches personally-sensitive data to ensure security standards remain watertight. Obviously that part of the process failed this time round and the bug slipped through the net.
Bugs happen. Nobody releases 100% bug-free code 100% of the time. Not even NASA. Some firms, due to the nature of their data and the risks involved, have a greater responsibility to run processes that minimize the likelihood of bugs but also to deal with them quickly when they are spotted in the field.
This was fixed two hours after it was reported. A handful of users had temporary access to other users' data that they shouldn't have had. And then they didn't. There is no evidence of any enormous data dump on the dark web containing yottabytes of personally identifiable patient secrets, though that is no doubt what a journalist wants you to imagine/fear when you read the words "data breach".
Regardless of your political/economic view of Babylon's business model and its potential negative effect on the public healthcare system (which may well be valid criticisms), it sounds like from a purely engineering perspective they should get some credit here for addressing the issue so soon after it became apparent.
This case won’t make it into the database since it doesn’t impact US-based consumers, but in general these kinds of breaches are (tragically) very common, and remarkably well-documented. Ultimately, this page is very convincing evidence for the often-touted idea that your SSN is effectively in the public domain.
The most common errors include sending report-containing emails to the wrong addresses, and leaving networks open with default access credentials.
From their own FAQ (on their NHS GP at Home service) at https://support.gpathand.nhs.uk/hc/en-us/articles/1150037092... :
"All recordings are encrypted and accessible only to you and restricted members of the Babylon GP at hand senior clinical management team managing your care, for the purpose of ensuring exceptional quality of service.ted and accessible only to you and restricted members of the GP at hand senior clinical management team managing your care, for the purpose of ensuring exceptional quality of service."
Asides from the fact it's got a copy-paste typo on it (perhaps not an important topic), no real info on how long recordings are retained for.
Edit: From https://www.gpathand.nhs.uk/legal/privacy-policy, it says recordings are retained until 10 years after death (with a few caveats), so "forever"... But only accessible via the app for 14 days, then you would need to ask for access.
They also say at https://support.gpathand.nhs.uk/hc/en-us/articles/1150036923... that
"All medical records are centrally held and meet or exceed NHS Standards. We work to processes and procedures that meet ISO27001 standards and have been independently audited and confirmed by BSI. The application has also been independently security tested."
They could probably argue these recordings aren't "medical records", but a recording of what was discussed is tantamount to a medical record. Just goes to prove to any doubters left that "tickbox" security doesn't work, regardless of ISO standards... And that a pentest is only as good as the pentesters, and the version of the system they tested.
It might be encrypted, but it appears that encryption was ineffective (i.e. the API or application has access to the keys), since a user was able to access others' recordings. To me that's the big problem with everyone claiming to be "encrypted" - Dropbox was also encrypted back when they forgot to check user passwords for a few hours and anyone could log in "as you".
As far as I know (I read the article), the recordings were made available to random users due to a fault in the backend side. Whatever encryption and access control they are using is totally bogus.
They're also hiring full-stack engineers and senior software engineers under their "Clinical & Business Platform" group, as well as a (Sr) Full Stack Engineer under their "AI Cognitive" part.
If these listings are reflective of their structure, you are probably right! Is there any good data or research on why heading up tech/engineering with a non-tech business person is bad? It's a pet bug-bear of mine!
In this ideal dataset I'd also be really keen to look at how the C-suite changes over time with respect to funding, and the outcome of more "investor friendly" non-tech folks end up in what would normally be tech roles. CISOs without security engineering background, CTOs that never built a side project etc.
[0] https://www.glassdoor.co.uk/Reviews/Babylon-Health-Reviews-E...
The leadership and management issues being so common makes me wonder if the resulting founding teams lack the experience of running technical teams and cultivating a good environment and morale. I suspect it's inherently NP-hard to manage a tech team whose work you can't yourself do. I imagine little of the founding C-suite of "health' startups has experienced the "length of a piece of string" engineering deadline when trying to solve complex issues etc, and perhaps that feeds into culture?
Nonetheless, I believe that it is mainly the fact that health itself is hard to iterate on, and requires knowledge from so many domains. It’s possible that the employees who were frustrated in the companies were expecting quicker iterations.
I agree with you. I think successful healthcare tech companies go through a growth stage first where the key is a solid basis in the healthcare field. Engineering experience isn’t as important in the beginning... right up to the point where everything comes to a screeching halt as tech debt reaches the point where it rapidly expands to consume pretty much all of the engineering team’s time.
My intuition is that healthcare is a field that is fundamentally at odds with the tech community mantra of “move fast and break things”. Rapid iteration is absolutely necessary to get a company off the ground, and that is not _incompatible_ with healthcare. I’ve found that there is an inflection point in the growth of healthcare companies where they have to rapidly change from “startup” to “mature business” and that seems to be very difficult to manage from a business standpoint.
Very broadly speaking - if a company is founded by “tech people”, this point is when their product reaches product/market fit and begins to gain users exponentially. The tech stack is typically pretty solid at this point (common and expected issues with tech debt, etc., excluded), but there often seems to have been insufficient attention paid to the regulatory issues surrounding PHI/PII. This can lead to data breaches like in the OP, but more often I think it leads to a period of slower iteration while engineering retools the product to properly deal with the regulatory concerns.
If a company is founded by “healthcare people”, security is _usually_ handled appropriately almost from the very beginning, but this point is reached when the feature set of the product gets too big for the developers to keep stable while implementing new features. This seems to lead to instability and a slow death march for the development team as the performance issues start to pop up more and more often, and new features take longer and longer to implement. Successfully navigating this means bringing on more experienced developers/engineers and a focus on “best practices” - automated testing, a proper CI/CD pipeline, etc.