Hacker News Reader

Comment by WalterBright | Hacker News Reader

> “It is very scary and devastating that someone can do factory restore on my drive without any permission granted from the end user,” one user wrote.

I'll say again: backup drives must have a physical write enable switch on them. To all the people who argue against this - just you wait till it happens to you!

Not having multiple backups is also a very bad idea. One day my 8T drive slipped out of my hand and smacked on the floor. That was the end of it. But I wasn't out my data, just out the cost of another drive.

Cloud backup is a stupid idea. At any moment it could go dark, for any reason, and you have no recourse. Might as well bare your throat and hand some random stranger a razor.

View on HN

> Cloud backup is a stupid idea. At any moment it could go dark, for any reason, and you have no recourse. Might as well bare your throat and hand some random stranger a razor.

I really disagree - physical storage in your home is the common alternative for a lot of consumer users - and that physical storage will involve maybe unplugging your external hard drive between backups - but otherwise never checking for the consistency and accuracy of the data nor the hardware. If you're working in a data center then it's your job to do these things and it doesn't take very much time... for normal folks cloud backups are likely going to be more reliable.

> cloud backups are likely going to be more reliable

I've seen enough HackerNews articles:

1. My account got deleted and I can't find a human to talk to and all my pictures going back 20 years are gone!

2. A hacker guessed my password and deleted my online presence! It's all gone!

3. My storage provider went bust and all my stuff is gone!

I think you've fallen into the trap of being scared by the news. Normal hard drive failures don't make it to HN because they're so boring and common. You even had one yourself. That's how unreliable maintaining your own backup device is.

This very story is just that - people had their own hard drive and it wasn't maintained for security so it got erased. At least this article needs to be added to your mental list of failures of local storage to balance your news-driven bias.

"At least this article needs to be added to your mental list of failures of local storage to balance your news-driven bias." What a toxic comment. You are saying far more about yourself than you are of someone you don't know at all. Projection and a lot of anger/hate inside.

> You are saying far more about yourself [...] a lot of anger/hate inside.

Two wrongs don't make a right.

Then why be wrong/unhelpful/provocative?

If my cloud backup backend provider cancels my account, my backup software will quickly complain. There will be a window of exposure, but I can act quickly.

If my backup harddrive is in the same house as my main storage and my house burns down, I'm fucked.

If I'm fancy, I may have multiple backend storage providers, too. Lot easier to do that then to have multiple houses. I wanted to use a safe deposit box in the past, but last time I called banks near me, they didn't even have safe deposit boxes available to rent! I can ask some friends, but then will be limited in frequency of updates, still.

"Redundant Array of Inexpensive Cloud Storage Providers is not a backup!!!" <grin>

(Can we make RAICSP a thing? Or do multi-cloud solution vendors already has a snappy marketing term for storing everything on all of S3, Azure Blob, and Google Storage at once?)

It's not? Why?

Yeah, I’m falling on this too. A client with write-only privs to a buckets on multiple cloud storage providers is literally the most durable backup in human existence right now. Like nothing else will get you worldwide DR and 30+ 9’s of durability.

3 (designed for) 10 9’s systems whose failure modes are wholly independent might be 30 9s.

I strongly suspect (as in “for the right price would stake my life on it”) that there are some common failure modes across the major cloud vendors’ offerings.

Your advice is directionally sound (and is what I do for family docs and photos); I just think it has fewer meaningless 9s than you’re imagining.

It's probably not a bad idea, but if they're all linked to a single e-mail account/provider, things might turn sour quickly.

Sorry, just riffing on the “RAID is not a backup” that gets yelled at everybody who ever did an rm -rf on their raid by accident…

I haven't yet implemented it but perhaps a reciprocal syncthing arrangement with an IT knowledgeable friend?

I didn't think syncthing supported encrypted replicas - that's why I passed on it years ago for a dropbox-like use case. I wanted to have a replica in the cloud as a relay, without having to fully trust the machine. A quick google suggests they may have something beta along these lines, so this may change.

I believe Resilio can do it, but it is closed source.

If you're looking for backups and not dropbox-like functionality, then something like restic might be more appropriate. It does actual content addressed backups, so you've got history for those times when you realized you messed something up six backups ago.

Syncthing now supports untrusted encrypted nodes, and it's fairly reliable.

Companies like Iron Mountain provide this service to businesses. Even if a bank no longer offers safe deposit boxes (and you were relying on their "safe", which for small banks was an arbitrary thing), there are usually facilities in larger cities that offer the same sort of service.

Just went through this with my family (in Australia) when the local bank branch said they were shutting down the service. It was basically a locked room in their branch, there was no "safe" involved.

Backblaze's latest drive report shows an annual failure rate for hard drives of 0.85%. I very strongly suspect the total of all failures of the type you're listing there is many orders of magnitude lower than that across all cloud storage users.

If you care about your backups, a single cloud vendor/account vanishing isn't going to be a problem, in exactly the same way that if you care about your backups,m a single drive failure will not be a problem. They are both predictable and mitigable risks.

> in exactly the same way that if you care about your backups, a single drive failure will not be a problem

0.85% is extremely high. I've rolled enough 1s on d100s to know that I wouldn't take those odds!

Yeah, but it certainly feels in the right ballpark from my personal experience. I’ve normally got 20 or so drives in use at any time (perhaps fewer spinning rust drives not that ssds are more common), and I have a drive go bad occasionally - not every year or two, but certainly every 5 years. So I reckon they’re at least order of magnitude right.

I have found that filesystem corruption or accidental deletion is far more common than physical drive failure.

Well I guess you should have both. That’s the point point right? Redundancy?

There’s “cloud backup” and there’s “storing everything primarily in one cloud provider”.

> I can't find a human to talk to and

Obviously customer support is a finite resource.

That's why companies need to start paid/premium customer support: when you lost access to your account, you should be able to pay 10, 100, 1000 of money to talk to a real person who understand how the system works and where there a mistake was made.

Like a hacker stole your account and you cannot take it back? A special person will to do a background check (e.g. call your employer) to verify that you is actually you.

Robot erroneously blocked your account because of misclassified spam-like activity? Your account is restored, and you are fully refunded.

And:

4. My crypto exchange disappeared/went bankrupt/was a scam and now my crypto is all gone.

I swear, some people never learn.

Don't be silly! That could never happen to my crypto exchange

Of course not! And they're _totally_ 1:1 fully backed with USD!

not your keys, not your coins. no exceptions.

do you know of any tools for consistency/accuracy? I've been meaning to do some basic md5 checks for all my files but haven't gotten around to it. Every now and then I find a corrupt jpg image that has a rectangular band running through it or something of the like.

Generally I'd recommend using a filesystem designed for that. Something like btrfs or (I think) ZFS, which have checksums built into the filesystem (and if you set them up in a RAID configuration, these checksums can be used to correct data as well)

I believe that's some form of bitrot. AFAIK only ZFS can deal with this (assuming you use ECC RAM).

EDIT: According to Wikipedia, ZFS, Btrfs and ReFS offer strategies to deal with various forms of data degradation, although it appears as if ZFS is still king.

https://en.wikipedia.org/wiki/Data_degradation

For all my important photos I use par files. Using command-line tools like par2create and par2repair is simply part of the routine of storing photos from the camera on the NAS.

I also use it for music files, in the past too many mp3s got broken.

Yup, I also do this. Photos get moved into monthly folders, and every now and then, I run "par2create par2file *".

Before I did this, in the past, I restored from backup and found I had damaged jpeg files. With par2, there's at least a recourse.

parchive and md5deep

http://md5deep.sourceforge.net/

> Cloud backup is a stupid idea. At any moment it could go dark, for any reason, and you have no recourse. Might as well bare your throat and hand some random stranger a razor.

It's not stupid but redundancy is important, don't rely on cloud only.

My own anecdata: I lost a couple TBs with Streamload[0] ~2007 when they switched over to MediaMax. They wouldn't admit it was their fault and I never got any kind of compensation - even though I had just renewed my yearly subscription a couple months prior. Thankfully I had offline backups on CD/DVD but that was definitely no fun. More recently CrashPlan decided to no longer offer unlimited backup for non-enterprise users i.e. I would have to pay a LOT more money to back up multiple TBs. At least they offered me a partial refund for the remaining months in my yearly subscription. I really really hope Backblaze keeps their unlimited plan.

As for physical backups, my 8-drive RAID-5 GPT partition got somehow nuked when I was trying out a new Windows 8 installation (one of the first official builds). I never found out what actually caused it, but I immediately went back to Windows 7 and waited for 10 to be out for a couple years before finally switching over. I'm still paranoid and keep my backup server separate and off the network except to make diff backups.

Bit-rot is also becoming a serious issue. I hope to finally build a home ZFS server (with ECC RAM of course) to deal with this, as I've noticed non-streaming encoded videos sometimes just stop working after 15+ years. I'm not sure if it's actually bit-rot but I can't think of anything else that would cause this to happen.

In any case, keep multiple backups: offline and online!

[0] https://en.wikipedia.org/wiki/Nirvanix

> bit rot

Hard disk drives go on sale regularly. I replace mine usually once a year or so. They're not expensive relative to the loss of the data. I just bought a 500G SSD drive on sale for $60.

Do people buy SSDs for backup purposes? Sure it's fast, but isn't it better to get 6x the storage for the same price?

I've read that SSDs are also generally more susceptible to data degradation earlier than spinning rust, although I'm not sure how true that is. Wikipedia[0] says:

- Solid-state media...store data using electrical charges, which can slowly leak away due to imperfect insulation.

- Magnetic media...may experience data decay as bits lose their magnetic orientation. Periodic refreshing by rewriting the data can alleviate this problem. In warm/humid conditions these media, especially those poorly protected against ambient air, are prone to the physical decomposition of the storage medium.

In either case, only ZFS/Btrfs/ReFS seem to implement strategies to deal with data corruption. Kinda sad that Apple recently released a new FS and didn't bother to address this.

[0] https://en.wikipedia.org/wiki/Data_degradation

These are small and very convenient, and don't need a power supply. Great for trip use. I wouldn't use them for long term storage.

You must have multiple copies of data. Assume you always have one less copy than you actually have. Drives fail for any number of reasons.

Cloud backup is perfectly fine as long as it's one of those copies -- I cloud backup my desktop and it saved me from a lot of loss.

Three copies or it doesn't exist. At least one should be elsewhere from the other two.

This is often stated as the 3-2-1 rule. At least: 3 copies, 2 different kinds of media, 1 copy offsite.

also different methods/software is important as you never know what bug may be lurking leading to bad backups

Cloud storage is convenient, as one can then access it from any device. It's great to use as a temporary backup when traveling.

But not as a backup strategy.

> But not as a backup strategy.

Nonsense. Any robust backup strategy has multiple copies, and multiple locations. Cloud is a sensible component to this.

Why not as a backup strategy? I had my desktop harddrive fail and while I had some local backups but the cloud backup restored me to almost the minute of the failure. The only downside to cloud backup is the slow restore process.

> The only downside to cloud backup is the slow restore process.

If you use BackBlaze, they offer multiple options to restore data quicker if you're willing to shell out more money, including AWS or sending you a physical drive with your data on it, albeit the latter is kinda expensive, but if you have a ton of data and need it ASAP, is probably worth the cost.

I recently lost a 16TB drive for the second time in 6 months and my local backups were out of date by a couple months, meaning I needed to restore ~200GB of data. Aside from the web interface file explorer window being cramped (and support telling me to basically go fuck myself after suggesting that), I was able to make ZIP files from recently changed data and use their downloader tool which IMHO is a piece of crap and probably designed by some intern, but at least the download speed was fairly fast, even in Europe, in contrast to upload speeds capped at around 250 KB/s (though thankfully support multiple threads).

> The only downside to cloud backup

Data theft is another one. Encryption might help, but you have no reasonable way of knowing if the encryption is secure.

Encrypt locally before cloud?

Any single backup is a stupid idea. Combine them, and you've got a great backup solution. Why would I keep important data in one physical location?

My house could burn down tomorrow, but I'd still have my cloud backups. My cloud backups could roast in a fire tomorrow, but I'd still have my local backups. And probably several other cloud backups.

If you put all your eggs in one backup, sooner or later, you're going to get screwed.

He only reliable backup mechanism is starting a successful cult whose main tenet is that your data is valuable communication from God, so that they continually duplicate it. Anything else will go bust long, long before the heat death of the universe.

Amazon just announced AWS Cult-As-A-Service. For the small fee of just $0.05 per Gigabyte per month they will store your data in three separate cults located in geographically distinct regions.

(Data transfer charges not included)

"Anything else will go bust long, long before the heat death of the universe."

As opposed to what? I think you've got too much faith in your method.

Well, you could always alter the laws of physics so that a zero entropy state corresponds to the data you wish to store.

_writes this down as a National Novel Writing Month idea_

They already wrote Canticle for Leibowitz.

Seems accurate; I may or may not have resurrected and rehosted the works of a cult I joined as a youth. Thanks, Internet Archive.

You made me chuckle and question reality I am in. Thank you.

>Cloud backup is a stupid idea

WHAT!? How the heck did you get that conclusion from this story? These are literally your people, who backed up to a local device and essentially chose their own local NAS rather then the cloud. Which unfortunately for them came from a company with bad support that dropped it 6 years ago and likely did a shitty job even before that. But not everyone can be a tech person! Even tech people can't be that knowledgeable about more than a fraction of information technology, the field is crazy vast [0].

The failure modes here are exactly why "cloud" makes so much sense for so many. Keeping stuff up to date, or migrating if the company drops it (even noticing that support has stopped). Securing access. Verifying integrity. Amortizing the cost of different media cold storage fallbacks with things like tape robot facilities. And on and on. You wanna opine a bit on the relative probability that Amazon, Apple, Backblaze, Dropbox, Google, Microsoft, rsync.net, or a host of others just "at any moment going dark, for any reason" vs Western fucking Digital My Dropped-in-2015-Book dying? Think carefully.

I mean shit, I actually have a custom Epyc-based TrueNAS Core (not FreeNAS anymore! gotta keep up!) system with tens of terabytes as my main backing store, and replication to a remote site, with more usage of network segmentation, wireguard, ZFS user privileges so compromise of the replication credentials can't delete old stuff, then I'd expect nearly anyone to have any idea about. I still have it going to Backblaze B2 as well, and I pay to have them hang onto it for a year rather then 30 days. And the only reason I can swing it all is that I can amortize a lot of it professionally, get other usage from it, and draw upon a bunch of both knowledge and metaknowledge that my particular life path happens to have granted me.

For 99% of the people in my life "turn on iCloud/Backblaze/whatever" is the right advice. Backups are very risky without heavy automation and regular attention on top.

----

0: One thing working at various levels has given me a deep appreciation for is what broad shoulders I stand upon, many layers deep, and how valuable and deep the knowledge of so many other tradespeople I interact with in life is. The mechanics, electricians, plumbers, general contractors, structural engineers, medical folks, researchers and so on I work with or have worked with all possess skills and knowledge that I never will. That's why we're social animals. I've traded on my skills helping someone improve their network for a hand with skilled carpentry or electrical work or the like. I don't see any shame in that.

They hooked the backup to the internet. Not a good idea. Nothing connected to the internet is secure, especially since nothing comes with physical write-enable switches.

Incredibly shallow and dangerously bad hot take. I think you have a very poor understanding of what "security" is, nor the concept of backups in general which necessarily (like security) have a very strong human factors requirement. A backup which is too much of a PITA and requires nearly any level of manual effort simply isn't going to get used much or maintained well at all by the vast majority of the population. Tools and systems exist to serve humanity, not the other way around, and when a system fails badly for many it's not humanity that must all change, it's a shitty system.

Fact: if these people had been running to a decent cloud service, they'd probably still have their data. Really "don't connect backup to the internet" is like, fractally stupid. The more one digs the more stupid angles turn up recursively similar to higher level stupid. "Don't connect backup to the internet" means what to you? What about the computer itself making the backup? If that is connected it could get infected as well, now what? Everyone is just supposed to give up on all networks entirely because "hurr durr nothing on a network is secure"? Which is wrong anyway since security an economic equation, not some ethereal absolute. I hope you never give this advice to anyone IRL.

Read about the recent ransomware attacks.

> if these people had been running to a decent cloud service

They thought they were.

> If that is connected it could get infected as well, now what?

That's what the physical write-enable switch is for. A hardware read-only device cannot be written to.

So all your data is stored offline and (presumably) on-prem? What if there's a fire?

> (presumably)

Consider that I worked on the 757 stabilizer trim gearbox. The idea is no matter what fails, the airplane lands safely. I've been trained to think that way.

Consider also that I've worked with computers for 45 years. I've seen about every failure you can think of, including fires and floods, explosions and earthquakes, viruses and phishing.

The most unreliable, by far, part of computing is the internet.

"Cloud backup is a stupid idea. At any moment it could go dark, for any reason, and you have no recourse."

If you are aware of the issue, and get kicked off AWS, or whatever, you move the backup. It is only a problem if your data catxhes fire and you get removed from cloud at the same time

Cloud backups are just one layer of a robust backup strategy. A copy on your desk, one in your closet, one at your parents, one in Backblaze, one in Glacier, etc.

“It’s not a backup until you restore it.”

Personally, I'm relying on the NSA having backed up all my data.

Good luck getting them to a) admit they have it and b) send you a copy ;)

As a taxpayer, aren't I entitled to a restore? :-)

I know you're joking but to stay on topic, you still have to even get them to even admit they have your data, since it's supposedly illegal. Thanks to Snowden we know, but even in light of those leaks, they have and will probably continue to lie during Senate/Congressional hearings. Once we get past that hurdle, then we can talk about restoring your illegally captured data ;)

File an FOIA request.

It’d definitely take longer to process than Backblaze FedEx-ing you a HDD ;)

My backup drive has a physical switch. I used to be meticulous in setting it to read-only except while making a backup. But now I just leave it in read/write mode because I lost my enthusiasm. It's not plugged in when not in use though, so it's a bit safer from this sort of thing.

How would you do automated backups with a physical switch? Are you proposing nobody does automated daily or more frequent backups?

I suggest rolling backups. Tapes are also a possibility, as they can be written in append-only mode.

Just remember that ransomware will encrypt any drive you hook up to it, including your backup drives. A physical write-enable will prevent that, and it should be disabled anytime you are restoring from backup.

I want a write-enable switch even for simple tasks like copying one drive to another, just so I don't mistakenly copy the wrong direction.

> and it should be disabled anytime you are restoring from backup.

What I was alluding to but didn't make clear was that physical switches are easily defeated by human laziness or mistakes. Why have a physical switch on drives you're copying if you might accidentally switch the wrong one or eventually stop bothering? This already happens with warning popups in Windows when you try to run an untrusted program, for instance. People get trained to bypass the security because it's just a tedious obstacle.

You personally might be careful enough to forever set the switch correctly, but people who didn't even know their hard drive was years-un-patched and internet connected yet left it there to get hacked also wouldn't reliably set the switches every time either.

Regarding accidentally copying in the wrong direction, I think a safer way than a physical switch is to show some details of what will get deleted. Maybe previews of images, tiny snippets of text from the files, a graphical view of the space they occupy, etc. Make it more visceral, like throwing away an actual book where you can see how many pages it has and what the picture on the cover is.

> backup drives must have a physical write enable switch on them. To all the people who argue against this - just you wait till it happens to you!

Yep, I once reformatted the wrong drive. I plugged in a firewire drive into a friend's Mac and didn't realize he had an internal drive that apparently also showed up with a firewire icon. Whoops!

>Cloud backup is a stupid idea. At any moment it could go dark, for any reason, and you have no recourse.

I wrote something about it [1] just a few days ago. And I have been banging on about it for years if not a whole decade.

>What we need is something like iOS TimeCapsule. Let the Phone do the Photo Management on Devices, and have an on site ( Home ) DataBackup as well as offsite ( iCloud ) Data Backup. Apple could even make iCloud Backup subscription a requirement for TimeCapsule to function.

We need both. In a simple, safe, easy to use fashion.

[1] https://news.ycombinator.com/item?id=27605365

> Cloud backup is a stupid idea.

I don't know what you mean by "cloud", but an off-site backup is essential. Your home can be burglarized, or burnt down, or flooded. Basically anything you insurance covers for your home you should cover regarding your data.

> At any moment it could go dark, for any reason, and you have no recourse.

There is no difference between your backup going dark or your main data source going dark. The whole point is that the data is duplicated and the odds of both of them going dark at the same time is low. This only means that you should check your backup regularly.

> I'll say again: backup drives must have a physical write enable switch on them. To all the people who argue against this - just you wait till it happens to you!

Reminds me of the little sliders on memory cards. Though I think those were software driven. It didn’t actually lock it out from writing.

Are there any portable USB style disks that have a hardware enforced read only switch?

you do need some sort of off-site backup for disaster recovery. House can burn down, get burglarized, flooded, and you absolutely must account for that if you really want to mitigate the risk

Absolutely right. If you're a company, rent a storage vault.

get _burgled_

> Cloud backup is a stupid idea. At any moment it could go dark, and you have no recourse

Home backup is a stupid idea. Any any moment your place could catch fire, and you no recourse.

> At any moment it could go dark, for any reason, and you have no recourse.

I suspect your drive is more likely to die in its own that AWS to fail.

Your faith in the internet doesn't match mine. (Far too many single points of failure.) But suppose you're right. I can buy 10 drives for cheap, but not 10 different cloud accounts.

I also don't care to have the internet snooping on my data. Yes, I know about encryption. Enough to know it cannot be trusted.

Reply on news.ycombinator.com