Comment by nemothekid | Hacker News Reader

Comment by nemothekid | Hacker News Reader

Parent Full thread

>There is quite a bit of discussion online on whether FB simply marks the data in their database as deleted or whether it is actually removed.

This shouldn't be a discussion. I can't think of a single non-toy database that actually deletes data when you press DELETE. All the LSM databases (pretty much anything using Facebook's RocksDB) doesn't actually remove data until a compaction event - and MySql/Postgres don't actually delete data until you a VACUUM occurs. (then there's whole minefield of did the OS actually remove the data, or did the filesystem just mark the affected area as deleted, but didn't scrub the bits). Then theres the backups - FB may have a ton of backups in cold storage that may have your data that wasn't removed.

The conversation about "FB simply marks the data in their database as deleted or whether it is actually removed." is unfruitful. The answer is "no" for almost every company on the planet, not just FB.

I don't think that's a meaningful comparison. There's quite the difference between "The data can be readily returned from the database if you omit the NOT deleted clause." and "You can use forensic tools to get data back. Maybe."

There is obviously a certain leeway in time allowed for a deletion request to actually be fulfilled.

But other than that: if technology gets in the way what people and the law demand, it's technology that needs to change, not people.

Exactly - and for all intents and purposes, that should be good enough. If the data is marked "deleted" and then written over within some reasonable amount of time (whatever their VACCUM cycle is to rewrite the immutable partition) then that should be good enough.

Now as for backups, this is much harder. I don't know how reasonable it is to ask them to discard backups in cold storage - seems like a compliance nightmare that would absolutely punish smaller players that can't build infrastructure to do that.

Regarding backups, I think the answer might be: "keep the backups only as long as you need".

Is it really reasonable for an active site to keep backups older than a couple of months? A year?

> Postgres don't actually delete data until you a VACUUM occurs.

FWIW, a large portion of cleanup happens on access. If the entire page will not be accessed until the next VACUUM that doesn't change anything, but in OLTP workloads that space will often be reclaimed much earlier.

Doesn't change anything about your larger point though.

Reply on news.ycombinator.com