Hacker News Reader

Comment by madebysquares | Hacker News Reader

if you have 17TB of data. Where is the best place to back up that much data? Other then redundant back up drives... Is there a suitable cloud solution for this? If not is there enough demand to build one? I dont own that much data. I cant imagine having to keep track of that much data, I hate having more than 1 TB of data to keep track of.

View on HN

Yev from Backblaze here -> Cloud Storage services are better suited for that much data. I wrote a post recently where I went in to the differences between Sync, Backup, and Storage as it relates to the cloud -> https://www.backblaze.com/blog/sync-vs-backup-vs-storage/.

I'm not sure what happened in this user's case - but my guess is that something happened on their machine where our backup client decided that data on the machine didn't match with what is in the DC and restarted backing up the data so that we'd have a clean copy, resulting in 13TB of data getting re-scheduled for backup.

For a backup service, you seem strangely unconcerned about what went on here.

I wouldn't say that, it's just that I don't have a lot of data about this particular case to go on. In some cases if you overwrite our index files, you can get something that's known as a Safety Freeze. Again - I have no idea what happened with this customer, but I would venture to guess that since this occurred on their C: drive, something similar to what causes a Safety Freeze (https://help.backblaze.com/hc/en-us/articles/217666178-Safet... happened here, and they needed to re-upload the data (in most cases, previously uploaded data is still available for restore - but with 13TB of data to re-upload and not a lot of bandwidth, it would have taken them way longer than our 30-day history to do that).

Plenty of backup solutions — just nothing very cheap. 17 TB is a lot. Then again, this was posted on a forum for self-identified Data Hoarders.

The best thing to do is probably to buy an NAS, seed it with your 17TB of files and then colocation it at a colo facility or a friend's house and then use some sort of one way sync utility (rsync, rclone, carbon copy cloner, bittorrent sync, etc) to update the remote NAS as you add/delete files to the source.

This is the way I would go took, although with two NAS boxes. One local and a similarly sized one off site. Then you backup to the local one and you sync the state to the remote one.

Lose data locally, restore from local NAS

Lose data on local NAS, restore from remote NAS

Lose data on remote NAS, restore from local NAS

>> if you have 17TB of data. Where is the best place to back up that much data?

Not the cloud if you have a typical home internet connection

Amazon Snowball?

17TB is a lot to send over the wire. If you've got a residential ISP connection it'll probably take months.

Backblaze and Crashplan both have (or had) a thing where they'd mail you a large hard drive to load your data on for a deposit of a few hundred dollars. Although at 13 TB I guess you'd still need to prioritize what you really wanted backed up straight away.

It's trivial: use a couple of USB external hard drives.

You don't want to deal with cloud on a home connection, it's gonna take forever.

If you are a company, you pay for storage on S3, Google Storage, OVH or IBM Softlayer. It's great quality yet simple and moderate costs.

Companies keeping data in house pay for NetApp, NetBackup and tape hardware.

Even with USB it's going to take many days to back up 17TB.

You should have started the backup process well before you hit 17gb of data.

These kind of stuff support incremental and differential backups.

Reply on news.ycombinator.com