People die all the time from cancer or car accidents. People very rarely die from data leaks.
Some countries like Sweden make people's private financial data public information - and yet their people seem happier than ever. Perhaps privacy isn't as important as we think for a good society.
There exists another, often unmentioned option. And that option is for state/business to open up, to increase their "information surface" towards us, their citizens/consumers. That will also achieve information (and one hopes power) rebalance. Every time it's actually measured, how much value we put on our privacy, when we have to weight privacy against convenience and other gains from more data sharing, the revealed preference is close to zero. The revealed preference is that we put the value of our privacy close to zero, despite us forever saying otherwise. (that we value privacy very very much; seems - "it ain't so")
So the option of state/business revealing more data to us citizens/consumers, is actually more realistic. Yes there is extra work on part of state/business to open their data to us. But it's worth it. The more advanced the society, the more coordination it needs to achieve the right cooperation-competition balance in the interactions between ever greater numbers of people.
There is an old book "Data For the People" by an early AI pioneer and Amazon CTO Andreas Weigend. Afaics it well describes the world we live in, and also are likely to live even more in the future.
If not, why?
Within the UK NHS and UK private hospital care, these are my personal experiences.
1) Can't email my GP to pass information back-and-forth. GP withholds their email contact, I can't email them e.g. pictures of scans, or lab work reports. In theory they should have those already on their side. In practice they rarely do. The exchange of information goes sms->web link->web form->submit - for one single turn. There will be multiple turns. Most people just give up.
2) MRI scan private hospital made me jump 10 hops before sending me link, so I can download my MRI scans videos and pictures. Most people would have given up. There were several forks in the process where in retrospect could have delayed data DL even more.
3) Blood tests scheduling can't tell me back that scheduled blood test for a date failed. Apparently it's between too much to impossible for them to have my email address on record, and email me back that the test was scheduled, or the scheduling failed. And that I should re-run the process.
4) I would like to volunteer my data to benefit R&D in the NHS. I'm a user of medicinal services. I'm cognisant that all those are helping, but the process of establishing them relied on people unknown to me sharing very sensitive personal information. If it wasn't for those unknown to me people, I would be way worse off. I'd like to do the same, and be able to tell UK NHS "here are, my lab works reports, 100 GB of my DNA paid for by myself, my medical histories - take them all in, use them as you please."
In all cases vague mutterings of "data protection... GDPR..." have been relayed back as "reasons". I take it's mostly B/S. Yes there are obstacles, but the staff could work around if they wanted to. However there is a kernel of truth - it's easier for them to not try to share, it's less work and less risk, so the laws are used as a cover leaf. (in the worst case - an alibi for laziness.)
In general, I find the ongoing public scare about sharing data, to be anti-thesis to the original spirit of the Net, that was all about sharing data. Originally, we were delighted to connect to perfect strangers on the other side of the world. That we would never have gotten to communicate with otherwise. I accept there might have been an element of self-selection there, that aided that view: people one'd communicate with, although maybe from a different culture, would be from similar niche sub-culture of people messing with computers and looking forward to communication, having a favourable view of that.
I think this might be a bit of a social bubble thing - I think it isn't a forefront concern for the vast majority of people.
About myself personally - my Name Surname is googleable, I'm on the open electoral register, so my address is not a secret, my company information is also open in the companies register, I have a a personal website I have put up willingly and share information about myself there. Training models on my data doesn't seem riskier than that.
Yeah, I know I'd be safer if I was completely dark, opaque to the world. I like the openness though. I also think my life has been enriched in infinitely many ways by people sharing parts of their lives via their data with me. So it would be mildly sociopathic of me, if I didn't do similar back to the world, to some extent.
Boundaries - yes sure they exist. I don't have my photo albums open to the world. I don't share info about family and friends - I know people by default don't want to share information about them, and I try to respect that. Don't share anything on Facebook, where plenty share, for example.
At the same time, I find obstacles to data sharing codified in the UK law frustrating. With the UK NHS. 1) Can't email my GP to pass information back-and-forth - GP withholds their email contact; 2) MRI scan private hospital makes me jump 10 hops before sharing my data with me; 3) Blood tests scheduling can't tell me back that schedule for a date failed, apparently it's too much for them to have my email address on record; 4) Can't volunteer my data to benefit R&D in NHS. ("here are - my lab works reports, 100 GB of my DNA paid for by myself, my medical histories - take them all in, use them as you please...") In all cases vague mutterings of "data protection... GDPR..." have been relayed back as "reasons". I take it's mostly B/S. They could work around if they wanted to. But there is a kernel of truth - it's easier for them to not try share, so it's used as a cover leaf. (in the worst case - an alibi for laziness.)
I'm for having power to share, or not share, what I want. With Google - I do want them to know about myself and use that for my (and theirs) benefit. With the UK gov (trying to break encryption) - I don't want them to be able to read my WhatsApp-s. I detest UK gov for effectively forcing me (by forcing the online pharmacy) to take a photos of myself (face, figure) in order to buy online Wegovy earlier today.
For UK - I'm reasonably sure some people will have died because of the difficulties sharing their data, that would not have died otherwise. "Otherwise" being - they could communicate with the NHS, share their data, similarly via email, WhatsApp etc, to how they communicate and share data in their private and professional lives.
People at personal level have a fairly reasonable stance, in how they behave, when it comes to sharing their data. They are surprisingly subtle in their cost-benefit analysis. It's only when they answer surveys, or talk in public, that they are less-than-entirely-truthful. We know this, b/c their revealed preferences are at odds with what they say they value, and how much they value.