Hacker News Reader

Comment by statictype | Hacker News Reader

As an engineer we are taught to look ahead when designing software and to do it with the "good of society" in mind.

Keep in mind, that when the photo features of Facebook were added, they probably didn't do it while thinking about half a billion users or whatever gazillion terabytes of photos they have to operate with. Heck, they may have even started out with simply storing photos on disk in their servers where deletion was easy as opposed to on a CDN.

View on HN

I've designed a couple of small scale websites before and I've always thought about how they would react if the user base would expand significantly. It's always something in the back of my mind when programming.

Even with a small user base; if a user flags a photo for deletion -- it should get deleted. Period.

I run 9cloud.us, a photosharing site.

When a user presses the delete button, we...

1. Delete it from the public facing servers that we control. 2. Put a request in the queue so the backups, and thumbnails will be deleted. 3. Pray that the CDN will purge the picture eventually.

#3 is a big deal... we don't control the CDNs cache options, and can't push a 'delete' upstream so the picture might be gone 2 seconds after the deletion is completed on our end, or it might take 2 weeks.

I'd thought Facebook would have been able to engineer or negotiate better options than we do, but apparently not.

There's large scale, there's really large scale and then there's Facebook scale. They operate at a scale that almost no one does, and in fact, if you're thinking of how to operate at that volume when you start your new project, then I'd say you're doing it wrong (I know you didn't actually say that - just pointing out that no one should be thinking of how to operate at that scale when designing their basic architecture).

I don't really know how their photo system works - maybe it is a trivial thing to purge files from CDNs on demand - but I'm willing to give the benefit of the doubt and say never attribute to malice, what could be explained by earth-population-level scaling problems.

Also unrelated: Facebook has shown many times that they can't really be trusted on privacy issues - I never upload anything there under the illusion that I can freely delete it and it's really gone. But I don't personally believe this thing is a privacy issue they're intentionally balking on

Maybe I mis-represented my argument. I didn't mean that I code small scale projects with large scale back ends just in case a user base boom happens. I meant that I try to code as using a "module" approach as much as possible. When (and if) the opportunity presents itself, the small scale project will be able to integrate a large scale solution much more easily.

You're making the same mistake that many awful managers make; namely "If the problem seems like it should be easy, it must be easy".

You aren't familiar with Facebook's infrastructure and comparing working on small websites to working on something that scales to roughly 1/12 of the entire population of the planet is laughably naive.

Saying you use a "modul[ar]" approach is nothing but handwaving. You think that facebook doesn't use modules? Or doesn't have a service oriented architecture to some degree? They friggin' developed their own RPC transport!

Thanks to those of you who give Facebook the benefit of doubt. I don't work on this particular system, so I'm not qualified to discuss any specifics, nor do I speak for the company. I can say this though: People @ Facebook do care about these types of problems and work to solve them. I'm not some new grad; I've worked with large systems for a while. The scale is hard for me to comprehend. The systems I do work with illustrate very well that nothing is as simple as you think it might be and most of the obvious solutions get tossed out the window because they won't scale.

Reply on news.ycombinator.com