Hacker News Reader

Comment by smonff | Hacker News Reader

Self storage is fine but does it guaranty long term documents safety?

I am very sceptical about my ability of maintaining it on the long term.

This is usually a difficult problem. Then I divided everything I have data-wise into two buckets: stuff I can't afford to lose, stuff I can. It's now simple. Also I mandate that 'everything is a file'.

Turns out the former is only around 22Gb of mainly code, documents, videos and photos. The latter is 250Gb of music, films etc.

The latter goes on a NAS with a mirror on the laptop.

The important stuff lives on the laptop protected by bitlocker, an encrypted USB stick protected by bitlocker that is on me 24/7, the NAS and another offline unencrypted USB stick that I drop off at my parents' and cycle every six months. All disks are replaced every three years. All USB sticks once a year.

All file transfers are done with Beyond Compare by hand and verified after.

> It's now simple.

~> A handful of manual steps you need to remember to do.

The only offsite you have is at your parents, and is only "uploaded" every 6 months.

Isn't it just better and easier (at least long term) to just use duplicity (or equivalent) and sync your NAS to any backend (Drive/Onedrive/Dropbox/Hubic/etc)? And then just use your NAS as your "true" storage? Then it'll just run until your configuration changes (NAS switch, service discontinued, etc) at which point you reconfigure it once.

Yes that 6 months window I decided is ok for me. It might not be for you. If my house burns down, my car burns down and I end up in hospital, losing 6 months of data potentially isn't a major issue.

If you make it a conscious scheduled process you know it's working.

I'm a relative expert with duplicity (I run a whole pile of stuff in AWS that uses the S3 back end for backups) and it's not a solution I really entirely trust. First you incur the wrath of configuration i.e. the initial ramp up to get it working, then there's the monitoring overhead which isn't trivial as it isn't the quietest bit of software known to man, and finally we have the complexity perspective which introduces a lot more code, network transport etc.

Plus it is horribly buggy, particularly with incrementals, metadata and corruption. The same is true with Google Drive, OneDrive, Dropbox which I have used.

Edit: comedic timing perhaps, but since I wrote this post I've just got a duplicity failure to deal with...

This is why I went with CrashPlan. Backing stuff up to a friend's computer? Free and only limited by their storage.

or.. put a box at your friends house with as much storage as you want. Back when I had FIOS (and it was fast & reliable) this is what I did. Put a NUC with a friend, had an external drive, we were each others offsite backup location. Quite handy.

Every six months is a very painful potential loss window. There are great sync tools that can work point-to-point over the Internet. The cost of having a NAS at home and a NAS at your parents, and then syncing between them is totally worth it.

crashplan has a client that handles this, it's free for this sort of usage, you only pay if you use their cloud service.

Are you seriously less trusting of your own ability to drop a bunch of documents onto a USB drive every week or few than you are of a company which explicitly tells you they can shut off your access with no warning and no recourse for any reason?

...yes?

Because the likelyhood of Google banning your access is way lower than * Data corruption * House fires * Human error (Cleanup, house moves, random acts of brain farts, and suddenly you can't find your USB stick)

Google's data is presumably replicated across multiple AZs, and they have more automation invested in data integrity.

This is, I believe, the correct analysis.

I am personally/professionally qualified to keep my data extremely safe and distributed. It's one of the things I do at work, at scale. But it's not easy. It is, after all, work. I don't have to worry about that using Google and other online services.

And, in order to mitigate against the extremely unlikely event of Google going haywire and perma-deactiviating my account, I run takeouts twice a month. These large data dumps don't need to be heavily redundant, since it's exceedingly unlikely I'll need them at all.

Granted, going from a takeout backup back to 80% functionality would be a ton of work, but I'd lose very little, and I could indeed become functional again in relatively short order.

I wonder if people who decide to ditch most things Google/cloud for no other reason than concern about the risk of said cloud providers terminating their access have truly considered the risk/reward. Bad news, like specific cases of 'unjust/surprising' account termination percolates up to wide visibility quickly and easily, and stays with the people who read such things a long time. The hundreds of millions of accounts that have no such problems escape our attention.

I don't know the odds of my Google account getting terminated, but I'm pretty certain that it's in the same magnitude as the number of people winning the lottery per year.

Having said that, there are a lot of other good reasons to think about ditching cloud providers.

I have a home server, which is then mirrored to a rented server. Anything critical and entirely irreplaceable goes onto blu-ray disks, because I can't accidentally delete those; though those are kept at home as well, which is not ideal.

So: 2 backups, from which one is off-site and archives, just in case - this should keep you going for a while with self storage.

>Anything critical and entirely irreplaceable goes onto blu-ray disks, because I can't accidentally delete those

There's bit-rot though: https://en.wikipedia.org/wiki/Data_degradation#Decay_of_stor...

I'm aware of that too well; blu-ray, however, is slightly better than the predecessors: even the current, non-archival grade single layer Panasonics should - according to some accelerated tests - last 5-10 years. Now, looking at the current ideas from Sony[1], within 8-10 years every data I ever had will need 1 disk, so I'm not worried.

[1]: http://www.computerworld.com/article/3057889/data-storage/so...

Blu-ray? Really? Sure, they may last, but, will you have a blu-ray reader?

I've worked in IT for over ten years, and have not once, once seen a blu ray in person, or ever handled a machine capable of reading one. It'd be incredible afraid to have my backups in something that requires some expensive exotic device to be written/read.

You'd be much better off with USB sticks.

Check out M-Disc, most new drives, especially from LG, support burning them: http://www.mdisc.com/

They claim the discs should last 1,000 years. Obviously no-one has tested this.

I've settled on having a NAS at home that backs up my most important stuff onto Amazon S3. I also have a lot of stuff with a paid dropbox account, that of course has local copies on my desktop and laptop. I'm looking into using Fastmail vs having my custom domain with an old grandfatherhed in google apps for your domain account.

Reply on news.ycombinator.com