Facebook’s Blu-Ray Powered Cold Storage Data Center

I used to have a job as one of those robotic arms.

The datacenter were I worked had lots of shelves full of tapes, and the computer would beep and display a tape number and a drive number. You'd have to jump up, find the tape and put it in the drive.

During the day you were running about constantly trying to keep the thing happy. On the night shift you could sit a watch movies and only get up once or twice an hour.

I wonder how Facebook is able to implement the EU privacy directives with this technology as their long term storage solution. If a customer requests deletion of their data you're supposed to comply with that, even on back-ups.

I am glad companies like Facebook is using Blue-Ray. Hopefully this will trigger progress in increasing density of optical discs and/or decreasing cost of high capacity discs and writers for consumer market.

Eager to archive my yearly ever increasing personal media (thanks to having a kid) onto a high capacity disc and just store it in the safety deposit box.

Just to help add some clarity, this isn't Blu-Ray in the common consumer sense. I've seen some comments on here about the stability of the dyes and capacity. This technology, as mentioned at the end of the article, is Optical Archive storage [1]. The media are in the form of cartridges with 12 discs in each cartridge, and Sony claims that the media is "50 year rated". Sony currently manufactures cartridges with up to 1.5TB of storage[2]. They also make a 7U, 10-cartridge library [3]. So, in a standard rack, you could fit a maximum of 90TB of Optical Archive storage. That's admittedly not a lot given the storage density of hard drives, so I'm guessing that Facebook's decision to invest in this is partly driven by energy savings or other considerations.

[1] https://en.wikipedia.org/wiki/Archival_Disc

[2] http://pro.sony.com/bbsc/ssr/cat-recmedia/cat-oda/

[3] http://pro.sony.com/bbsc/ssr/cat-datastorage/cat-opticaldisc...

A few months ago, Frank Frankovsky left Facebook to start a startup based on this tech! He was previously "VP of Hardware Design & Supply Chain Optimization" and has also been super involved in the Open Compute Project.

I'm pretty excited to see what they produce, and hope they can ship before someone like Amazon tries to acquire them. This level of cold storage is going to be critical as the world generates exponentially more data every year. He also mentioned this is an open source venture, which is doubly awesome. :)

https://www.facebook.com/frankovsky/posts/10203257723264639?...

So how does this work with the way Facebook delivers content? I understand the advantages of disc storage for a service like Amazon Glacier as the customer expectations are set for significant time and bandwidth constraints but Facebook is supposed to deliver personal data as fast as it can be served up. They've created a "customer" expectation of instant access that can't be provided by an offline disc.

It makes for a good solution as a 2nd or 3rd level backup but you'd need some form of RAID to prevent users from experience a lengthy delay every single time a disk fails. That's a lot of money to invest in a novel solution that creates a lengthy interruption in service should it ever be needed. Maybe the power savings are worth it when weighed against the rarity of such use and expected level of revenue lost to user frustration but that's an equation made fragile by so many variables.

I could also see it used in conjunction with some amazing use-prediction algorithm to store data that will most likely never be requested. If Facebook truly keeps everything then they have millions of photos, videos, hidden posts, and other digital detritus tied up in dead, inactive, or orphaned repositories. I know half the groups I joined back in high school are now impossible to find but the data they contained is still important for the purpose of linking people and interests together. I also have several friends who have passed away, making their account a kind of digital mausoleum. Their private data will never be accessed again but it must remain in FB's datacenter until someone is given access to the account to delete it. The number of Facebook accounts tied to the deceased is only increasing so these racks of Blu Rays might be considered the first digital cemetery. I don't know how many resources are allocated to this kind of data but this is where cold storage makes the most sense.

I really like how open Facebook is about their infrastructure, hardware designs, etc.

I would have thought that they would be trying something with higher capacity, but off the shelf == lower costs.

Interesting that FB needs to handle 900 million photo uploads a day. With higher resolution cameras on smartphones, their storage requirements will keep increasing. My Note 4 takes 4K video and very high resolution photos - FB, Google, etc. can store lower res versions only, but the pressure will be there for higher resolution support, especially with 4K TVs, and other high resolution devices.

I wonder how projects like Archive.org could benefit from something like this. I know it has it's up and downs, but if you have enough backups it shouldn't be too bad. I'm sure there's tons of untouched content that could easily be archived (an archive inside the archive), but other sites might as well, such as wikipedia. Of course they would need a bit of funding for this somewhat new endeavour. Maybe facebook can 'donate' the implementation to such sites.

Can I have one of those Blu-Ray storage units searching all my discs would be way more convenient if i didn't have to do it manually :)

Isn't the life span of a burned disc quite bad?

I wonder how long it takes to put data from that storage online. What happens e.g. if I want to see my 5 old images fullscreen?

All our chat history is there