further discussion on the original NYT story here: https://news.ycombinator.com/item?id=32538805
Once you reach the scale and importance of these services, it shouldn’t be legal to lock people out like this.
Playing devil’s advocate, possession of CP is criminal with absolute liability [1]. Deleting someone’s account, far from it. (Civil. Maybe.)
Weighing someone who might have CP, even to an absurdly low probability, against them being suing and like two people deleting their Gmail accounts, I can see how someone chose the former. If they got the call wrong, Google would not only be liable for damages, the individuals involved could be criminally charged.
I’m assuming everyone in this chain would appreciate laws clarifying an email bill of rights. On the other hand, Google made their bed with its surveillance-first culture.
So how come this has never happened?
Google and Facebook have existed for decades, and so has CP, they didn't have automatic detection untill decently and yet none of their employees have ever been charged.
There is no good excuse for this kind of spying apparatus, period. If anything, having a mandated formal process for reinstating banned accounts will make the scanning more acceptable, which it shouldn't be. "You see, yes, we're regularly checking out your dick pics but it's all good, we have a process in place and if we tag an image as CP, when it's not, you'll eventually be exonerated. No harm done"
> If anything, having a mandated formal process for reinstating banned accounts will make the scanning more acceptable, which it shouldn't be.
We should fight first for a proper legal recourse, and once this happens, then we can also fight for no scanning.
It does not matter, because we don't have any fight left in us at all..
I'm more than prepared to just stop working for a few days or even a week or two.
I already actively boycott Chinese products where possible (I buy non Chinese if at all possible, even at significant higher prices), support the Georgian Legion with my own cash and have probably spent hundreds of hours on unpaid online activism this half year on top of what I do for my church (although in all fairness it still does more for me).
If enough people band together and just plain stop working, sooner or later politicians will have to listen.
This can't keep going on like this. To somebody like Google, an account might just be a profit source, but to the affected people it can be their lives or livelihood.
The issue of right of redress really needs to be forced, legislatively.
This might go against some people's idea of a free market. I think it should only apply where there isn't a free market - when a customer doesn't have many other options, such as in the case of Google or Apple, or Visa/Mastercard, a major supermarket, or the choice of cellphone provider or Internet provider in markets where there are only a few options. It wouldn't apply to small independent traders by definition.
Capture enough of the market where there isn't enough competition and you get obligations to treat them fairly.
Google can use the defense that somewhere in the TOS it probably says you agree to share everything with them when it goes to cloud storage.
HIPAA doesn't prohibit people from sharing info about themselves (even accidentally). Though it does probibits your doctor from putting photos like this near Google anything.
- We would allow people to upload files
- these files would go into a 'holding tank'
- the files would then get greenlit if some rudimentary algorithm determined they were landscapes or other innocent content
- the remainder would be flagged for review
- bulk images would be displayed in a grid of 8x8 thumbnails
- the vast majority of those would not be reason for further action and would be greenlit, the remainder would go into yet another holding tank
- now the pictures would be viewed full size, any kind of image that would be against the TOS would be rejected and a hash would be kept to avoid seeing it again
- any kind of image that was deemed illegal or borderline would be passed on to LEO for another decision, possibly resulting in criminal charges against the uploader.
The complications:
- uploaders would try to mask their identity
- uploaders would be all over the world
- even after reviewing LEO would sometimes indicate that an image was inconclusive (in those cases we erred on the side of caution)
--
In short: companies can and do work together with law enforcement to stop service abuse and illegal activities. Whether it is an AI that flags it or a human is from a legal perspective utterly immaterial, and if LEO clears it that definitely means that you are allowed to either view it or pass it on.
Surely your reviewers never deliberately looked at the material a second time after marking it as abuse material for law enforcement, as that would be legally risky I would think.
The issue here is even if local LEO clear it, the federal prosecutor's opinion can differ and he can use the fact you knew AI flagged it and that you released it anyway to damn you. There are cases out there where local/state government cleared someone of wrong-doing but they still got hemmed up federally.
https://news.ycombinator.com/item?id=32178108#32178137
The biggest worry about these kind of situations is not that mistakes are made, but that it's almost impossible to make contact with a real human being to put things right [even, as in this case, when press and police are involved]. It's only a matter of time before someone loses their livelihood or even their life over this kind of cock-up.
How about "oh, everyone says we are wrong click to unban"?
Not even "everyone".
How about "law enforcement that we reported it to says we are wrong"?
cmiiw
The "blue check fix" minimizes negative publicity while doing nothing for 99% of people. If a bad policy is consistently applied, at least it can be understood and criticised.
Can one invoke EU rights, or is it tied to being physically resident like a DVD region?
What if the EU citizen flies to Europe? Can they then invoke their right from the physical territory of Europe?
I feel this is somewhat uncharted water.
Therefore, this account and all the data contained is already gone. The most they could do is allow him to create a new account with the same name, but to my knowledge that isn't supported technically in their systems (too many bugs to be found by re-using unique identifiers).
That's why they stand by their decision - they don't have the capability to implement any other option.
In this case, they actually satisfy the two requirements:
1. "ongoing legitimate interest". They are still in a dispute with the customer. Deactivating an account does not require deleting data, so long as the case it not closed. There is no "30 day" limit in this case, and it seems like an arbitrary time period imposed by someone who hasn't thought of the consequences.
2. "users permission". In this case they do have the user permission to keep the content, because the user wants to keep access to the account.
> they don't have the capability to implement any other option.
As mentioned above, you can deactivate an account without also deleting it. It may of course be easier for Google to just delete it instead of introducing a new state, but I would argue that a company such as Google does have the capability to implement this option. I would go so far as to say this is a failure of Google to not have such a process in place.
That works both ways - as well as not being able to get the customers permission to keep the data, they also would not be able to get the customers request to delete the data if they were to keep it 'just in case'.
Until they have built some functionality to communicate with the customer behind a suspended account, they have to delete the data within 30 days.
For example, if the login requires a password and SMS 2fa, but the SMS is to a Google Fi phone number, then you're going to have to also enable the Google Fi account for that user to be able to login. Or what if they need to use email password resets to another suspended account? (remember you will probably suspend all accounts of a user at once if you suspect something illegal).
Making a special 'chat' tool inside the login flow that only requires a single factor (the account password) which allows presenting evidence to a google rep seems like the obvious solution (or requesting the account suspension be delayed, or that the data be deleted).
Building and staffing that sounds like quite a lot of work though, which would be hard to justify for the tiny number of impacted accounts and zero revenue potential.
It isn't as far as I can see.
IMO it is just Google trampling over peoples rights while trying to use a weird reading of GDPR as excuse.
I'm so ready for EU to start punishing more of the blatant abuse of GDPR that we see, especially by the big players in the market.