These days I think everyone's breaking some rule in the EU
With respect to much older law, e.g. copyright, I think we still haven't fully interpreted what constitutes "copying" or "distributing" in a digital context.
With respect to data privacy, though I was part of a team that was responsible for ensuring my company met GDPR obligations, it's still not clear to me what really constitutes deletion or erasure for these purposes. What if my DB doesn't delete stuff on disk immediately but marks some records with an in-memory tombstone, so normal DB queries will no longer return the record but files containing the record do still exist? Am I obliged to delete all DB backups when any individual exercises their deletion rights? If my datalake uses columnar files that record events (e.g. clickstream data) from many users, every time any user exercises their deletion rights, do I have to re-write all the files that included any event from them? To find all files containing a user efficiently, I'd probably need to start indexing by user, which if anything puts my team on the path to using user-specific data more intensively going forward. Or is it sufficient to mark their ID in a "forgotten" file and ensure that datalake results do not include information from their records, though the records are in principle still readable? If you didn't have a good systems/data engineer participating in the drafting of the policy, it's easy for a regulator to just write "delete" without thinking through what the actual definition should be, and what the implications are.
No, just don't use that individual's data in any processing going forward, unless you have a lawful basis for doing so. If you restore from those backups, don't lose the fact this individual exercised their right to be forgotton.
> What if my DB doesn't delete stuff on disk immediately but marks some records with an in-memory tombstone, so normal DB queries will no longer return the record but files containing the record do still exist?
If you don't process those records any more, that's fine. Ideally purge your database tables from time to time, because you should also have retention limits, set to the minimum possible. "Forever" is not an acceptable limit for PII.
> do I have to re-write all the files that included any event from them?
No. Write a list of users whose events you mustn't process any more, and preferably stop collecting events for those users... assuming that specific data you're collecting is data which you don't have a legal basis to process without their continuing consent. If they withdraw consent, you have to stop. It's that simple. Design with that in mind.
> the records are in principle still readable?
It doesn't matter about "in principle", it matters if you continue to process the data of individuals who have exercised their right to be forgotton and withdrawn their consent, and you have no other lawful basis with which to continue processing their data.
> which if anything puts my team on the path to using user-specific data more intensively going forward
"The 'no beatings' law just makes we want to beat people even harder". You sound petty.
Treat PII as if it were hazardous waste; minimise your collection of it and processing of it, dispose of it as soon as you can.
> it's easy for a regulator to just write "delete" without thinking through what the actual definition should be, and what the implications are.
They did think it through. Section 67 of the GDPR: "Methods by which to restrict the processing of personal data could include, inter alia, temporarily moving the selected data to another processing system, making the selected personal data unavailable to users, or temporarily removing published data from a website. In automated filing systems, the restriction of processing should in principle be ensured by technical means in such a manner that the personal data are not subject to further processing operations and cannot be changed. The fact that the processing of personal data is restricted should be clearly indicated in the system."
Read the rest of the statute, it's not that difficult to understand: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CEL...
You just don't like having to do it, and would prefer if once a user gives consent (or even if they don't), you can keep that data and do anything you like with it for all time and there's nothing the user can do about it.
I'll note that two paragraphs above the point you quote from, under 65, the description of the right to be forgotten includes that "a data subject should have the right to have his or her personal data erased and no longer processed". I.e. erasure and cessation of processing are not the same but are both required. Your point about 67 only clarifies the second of those requirements. 66 describes something of the extent of erasure, but doesn't define it erasure -- one controller should inform other controllers who are processing data "to erase any links to, or copies or replications of those personal data", which doesn't seem to clearly support your interpretation.
> "The 'no beatings' law just makes we want to beat people even harder". You sound petty.
No, the point is if you have just a pile of event spew, it's legitimately more work to first e.g. group by user. If you have a user-indexed pile of events, without any particular hostile animus, still your data team has a warm start on user-centric analyses. But you sound snide.
> Treat PII as if it were hazardous waste; minimise your collection of it and processing of it, dispose of it as soon as you can.
In the ML/AI era, it quickly becomes very hard to justify ignoring your data that happens to have user-linked data in it. Users want and use recommendations and experiences that are informed by their personal data. Even if their data doesn't include PII fields like names, addresses or phone numbers etc, their right to object to processing, or ask for all their data to be disclosed to them, or to ask that it be deleted extends to all the usage and behavioral data that products are built on.
> You just don't like having to do it, and would prefer if once a user gives consent (or even if they don't), you can keep that data and do anything you like with it for all time and there's nothing the user can do about it.
Maybe don't try to tell other people what they like? I'm actually in favor of strong data protection laws, and I wish my own country would adopt some. I think the broad provisions of the GDPR are generally positive. However, I do think very little clear material was available for companies who were trying to come into compliance by the time it went into effect. In particular, there was no certification system, no official or regulator who you could get a live response from, no qualified inspector you could hire who could confidently evaluate whether a system did or did not meet the requirements of the new law. GDPR went into effect in 2018 and only this year was the Europrivacy certification fully approved under article 43. In the period from 2016 to 2018 many companies, including those not in the EU, were in the bad position of needing to draw up plans for large projects to make our data architecture meet the requirements as interpreted by our counsel, and in some cases evaluating whether it even made sense to do business in the EU (if you happened to only have a small amount of business there).
It might be very hard for you, the provider. Me as the user I don't care at all about your technical challenges (to put it mildly). I expect you to fix it so I voted my representatives to force you to fix it. Because if the AI can do everything as we are told again and again, I'm sure it can also find a way to purge itself of my data. Sarcasm aside, it's not 2018 anymore and there are more and more practical guidelines how to apply GDPR and everything else, so worst case let the AI swallow those as well and find that way.
> the controller, taking account of available technology and the cost of implementation, shall take reasonable steps, including technical measures, to inform controllers which are processing the personal data that the data subject has requested the erasure by such controllers of any links to, or copy or replication of, those personal data.
There is a lot of leeway in that. Depending on your circumstances, you may have an obligation to erase data, and communicate to other controllers you gave the data to, who also have to do the same. But note you can take into account your technology, cost, etc. - you must be reasonable about it (and instruct others to do the same, so the subject doesn't have to ring round all the 100s of spammers trying to datamine them); work with the process and do the best you can. Don't come up with excuses as to why you're going to ignore the data subject's requests.
The person who will be invoking this directive against you will be a national of one of the EU member states, and they'll be bringing their information commissioner with them. I seriously doubt they will proceed against you if, technically, you still have some of their data in old backups or keep tombstoned records for a specific amount of time before permanent deletion, but you've made sure that individual's data will never resurface in your system, even if you restore from backups, and you've minimised retention to what you absolutely need.
Of course, if you can delete a specific person's data, and they ask you to, and you don't have other legitimate reasons to keep it, you must. Indeed you should delete it even if they don't ask, as soon as it stops being relevant - and you shouldn't go with the plan of "oh but it'll always be relevant, so I'll keep it indefinitely".
Event streams? Have the user's client not send the event, or go round purging events, but given you have to check if the user has opted out of processing anyway, you might as well delete the data when you can. Data warehouses? Go through the user's records and overwrite them with randomised junk, and flag them as unusable.
> In the ML/AI era
Yeah get outta here with that thinking. Also see articles 21 and 22, the right to object to automated decisions.
> I do think very little clear material was available for companies who were trying to come into compliance by the time it went into effect.
I sympathise somewhat, but it's a regulation for individual states to actually legislate, and you had 2 years! The main thing you had to do was get the general message and not look for loopholes. From what I see, information commissioners have only been going after egregious holdouts, rather than well-meaning companies, especially if they don't have their own certification scheme running.