Hacker News Reader

Comment by Karunamon | Hacker News Reader

Of all of Facebook's behavior online, the advertising, the tracking, the privacy snafus.. this is probably the one that bugs me the greatest.

How this was not made a priority zero, drop-everything-and-fix-this-crap-yesterday internally is beyond my comprehension. I hate to think of the condition of their internal systems if they can't even get files deleted from their CDN in a timely manner.

3 years is not a timely manner. A month and a half (their target, FTA) is not a timely manner.

A few seconds to a minute would be great, thanks.

View on HN

You generally don't delete files from a CDN. They just get evicted when they are the least recently used, which may never happen if the URL is still in circulation.

Facebook may have set near-infinite Expires/Cache-control headers on their static content, since it never gets overwritten. In that case, the CDN never goes back to the origin to check if the file is still there. The CDN they are using probably does have the ability to explicitly remove files (e.g., to respond to DMCA take-down notices), but probably doesn't have the ability to do this at Facebook picture deletion scale.

In other words, Facebook probably can't do much about it unless they get the CDN provider to make a big investment, build their own CDN, or switch to another provider that does support it (usually a costly decision).

I have this same issue with my site, but in case anyone is looking for a solution---I have a tentative one: NetDNA (or MaxCDN if you're not looking at their enterprise options) has a XML-RPC API that lets you purge individual files. So you can delete something on your server, then push the purge to them.

So while things have been bad in the past--they're getting better. Having everyone try to use s3 as a CDN has really forced the industry to evolve.

Doesn't Facebook run their own CDN, though? (At least that's what the URL's would suggest).

And in the case of major CDN providers like Level 3, removing assets is dead simple. They provide an API for you to pass assets to expire immediately. Or barring that, having a person spend 5 seconds on the keyboard to log into the control panel and paste in the URL to kill.

It is possible - the author of the Ars article had his pictures erased after writing the article.

All of my Facebook photos are transparently hosted by Akamai. For example: https://fbcdn-sphotos-a.akamaihd.net/hphotos-ak-ash4/393462_...

The hostnames point to a CDN somewhere -- a few tests indicate at least Akamai.

"""having a person spend 5 seconds on the keyboard to log into the control panel and paste in the URL to kill"""

Few people, even technologists, can grasp what it means to have 800 million users. Anything involving individuals just logging in is a non-starter.

Not on a one off "Hey FB support, this image needs to be really really gone, make it happen." basis.

Even Google can provide that level of support.

In other words, they are now valued at $100 billion, but providing the most basic privacy functionality imaginable, after being given years to do so, is just too expensive?

value != revenue (Not justifying the status quo -- just pointing out that a valuation does not imply liquid capital)

While that's definitely true, Facebook is of course not lacking in free capital to throw at this problem.

They had $4 billion in cash before the IPO. They'll have $9 billion after it.

No CDN caches content for that long, even if the origin tells them to. (Services like Akamai's NetStorage excluded, which Facebook most likely does not use.) It's quite obvious that Facebook is actually keeping the images on their origin servers long after they have been deleted.

>How this was not made a priority zero, drop-everything-and-fix-this-crap-yesterday internally is beyond my comprehension.

Its simple. Minority concerns like this don't upset enough of their users to impact their bottom line in any meaningful way so they don't do anything about it.

Class action lawsuit anyone? That'll bump it up the priority list.

The more this becomes public the more likely it is that stories will surface about how "my life was destroyed because FB didn't actually delete the photo and now I need to be compensated" and there will be a whole bunch of lawyers waiting in the wings to take up their case. They may wait until the IPO has been completed...!

#1) If it was easy they would have done it. Pushing photo deletions up to the CDN is hard without flushing the entire CDN cache. And Facebook has 90 billion photos, so flushing the cache is not an option I guess. (Wrap your head around 90 billion photos. The scale of the problem is huge.)

#2) Anyone with access to the link to the CDN had access to the original photo. So this really doesn't protect someone from viewing your photos you wanted to delete - since if they wanted it before they could have had it.

So ultimately this should not bug you the greatest of all the bad things they have done. The ultimate blame belongs to the person who uploaded the photo to Facebook and then later wanted it deleted.

Yes, but the CDN is not going to cache 90 billion photos indefinitely. Clearly photos being on their for years after deletion, presumably being accessed very rarely, are going to be prime candidates for flushing. In all likelihood these photos are still on the origin server.

Reply on news.ycombinator.com