Why I Don’t Rely on Time Machine | Hacker News Reader

Why I Don’t Rely on Time Machine

joeontech.net

49 pointslelf42 comments

This is a good time to remind people about proper backups: the 3-2-1 rule.

3 backups. 2 backups (or more) should be on different mediums. 1 (at least) should be offsite (ie. home copy at work or in the cloud)

You need three backups. If we assume that every copy has a chance small but real chance of failure, the chance of all failing at once becomes exponentially less with each copy.

You need at least two different systems (media) for backup to reduce the possibly that one of them appears to be backing things up, but is in fact corrupted. So, two of some system, like Time Machine, Carbonite, Backblaze, Arq, rsync, etc. Ideally, you need to be able to verify your backups. Backups that stores things as files means you can periodically try pulling down some files to make sure they're good. Cloud backups should be in the mix, but consider them last resort for restoration as they are usually pretty difficult to verify everything is good.

You need one offsite to prevent your house burns down and takes your machines and local backups. You can do this with physical media rotation (ie. have two machine backups swapping places with home and work) or have a backup to the cloud.

There is this joke that you need three, in case your second one is damaged, because the first one will be damaged for sure.

I heard it in form about lighters in survival.

I like to say that if you have 1 copy, you have none. If 2, 1, if 3, you have 2 copies, the absolute minimum. And try to decrease single points of failure, e.g. I use rsync for rsync.net and BackupPC to local disk, and tar to LTO tape after giving up on the opaque and baroque Bacula.

You need one offsite to prevent your house burns down and takes your machines and local backups.

I'm a long term fan of tape, from the first time I used UNIX(TM) in 1978 and a DECTape saved me when I was learning the power of the -rf flags to rm.

LTO tape systems for the home are not insanely expensive (then again, this is my minimum system today: https://news.ycombinator.com/item?id=10483361 ECC is a requirement for me). Street price for the HP LTO-4 internal SAS EH919SB I'm currently using is ~1,300 USD (and it wasn't difficult to get it at around that price in 2011).

Add a LSI 2004 chip based SAS-2 controller such as the "Avago" LSI 9211-4i at $167 from Newegg right now to drive it, and a semi-disposable flash drive to feed it quickly so the tape doesn't "shoe shine" (the tape moves very quickly, if you don't feed it quickly enough it has to stop the tape and backup enough so that it's at speed when it starts forward again and hits the location where it ran out).

I.e. at whatever speed write a 10GiB slice of your backup to the flash drive, then write that all at once to the tape. Considering media costs, if you start from scratch today you'd probably be better off going LTO-5 now that LTO-6 is shipping, the media costs the same as for LTO-4 despite a near doubling of capacity. I've got enough LTO-4 tapes that I'll keep using them until my LTO-4 drive dies, then buy an LTO-5 drive, which per the standard can read two earlier generations, and write 1 earlier, and new LTO-5 media for it.

Current LTO-4 and LTO-5 tapes from HP or Fuji (I wouldn't trust anyone else) go for $22-23, uncompressed capacities are 800 and 1.5 GB respectively.

And then store copies off-site, and copy your most important stuff at least once a day to a site like rsync.net, which saved me when a tornado trashed my apartment in 2011.

> Current LTO-4 and LTO-5 tapes [...] go for $22-23, uncompressed capacities are 800 and 1.5 GB respectively.

I think you meant 1.5TB.

However, why not just use hard disk drives at just over twice that price? You'd get quite a lot of them for the cost of the tape drive.

Because tapes are much less fragile than hard disks, and I can stash a bunch of tapes in my safe deposit box and other places. As well as have many copies of my data as I keep monthly full backups for at least a year. And, hmmm, my tapes as I use them are more reliable than my Seagate enterprise disks, and that's with the latter not moving around (i.e. in all my post-mid-2011 tornado hardware, I've had two disk failures and no tape failures).

For the level of data security I want, and the tiny AT&T DSL straw I'm currently forced to use, even with things like Amazon Snowball and ever lower prices on "nearline" cloud storage, if a tape drive lasts long enough, the numbers work better for tape. And compared to the usual enterprise, I don't beat up on tape drives, they're over-engineered for my duty cycle. I also have a desire to be able to make due without monthly charges, which a cloud solution doesn't allow.

And, yeah, TB not GB. The DAT tapes I used previously were in the GB range as I vaguely recall.

>Because tapes are much less fragile than hard disks,

In 'single serve server' configurations, such as small businesses, tapes work well because administrative rather than technical staff can replace the tapes daily reducing costs. If the tape is ejected then the user knows it safe to put the next tape in. If there is a failure the software itself can notify a systems administrator. With hard drives a non technical user has no idea if the disk is still mounted and live, leading to corruption issues if it is removed.

And even more fragile if still spinning ... yeah, it would never occur to me to have anyone but trusted technical staff physically swapping hard disks around. Whereas I think people, at least us older types who used VHS tapes especially, have an idea for how tough/fragile computer backup tapes are (maybe a little tougher than a VHS tape, but not enough to make a difference in how you handle it).

(We'll ignore for the moment that I never trusted anyone but myself to run the backup system and mount and unmount tapes, with one exception. :-)

Yep. For Macs, that means something like TM + Superduper (or CCC). I do a weekly backup with SD which I copy to another machine on my network and backup offsite.

>Huh. The Time Machine preference pane suggested that I try again later, and I did, with the same result. Since there was an error creating a folder, I guessed that the disk might have some errors, so I ran Disk Utility. Sure enough, there were errors, but Disk Utility couldn’t fix them. Neither could DiskWarrior. Or TechTool Pro. So I was left with no choice but to reformat the disk and start over from scratch. I still don’t know what caused the problem, but neither Time Machine nor any disk utility I tried could fix it—and the hardware itself was, as far as I could tell, just fine.

The last sentence makes absolutely no sense. He said Disk Utility found errors, but couldn't fix it, and neither could DiskWarrior and TechTool Pro. So clearly the hardware itself was NOT "just fine".

>When my backups become suddenly inaccessible for unknown reasons and I’m offered no solution but starting over, that sort of diminishes my faith in the software.

In light of that, what should Time Machine have done? Magically fixed a disk 3 specialized disks couldn't fix and continue?

What would ANY other backup software do?

For unrecoverable corruption and no hardware failures you don't need bugs in 3 pieces of reading software. You just need one bug in the writing software. Or a momentary hardware issue like a power surge or lost power during a write in such a way that the journaling can't fix

That's the thing though: it could be hardware failure (or software failure unrelated to TM, like the power surge/writing glitch that journaling can't fix etc).

He just says there were errors on the disk and that 3 disk utils couldn't fix them. Nothing of that translates to that the error was TM's fault, so it makes no sense blaming it.

If the disk had issues, they would have also affected any other backup software.

Well, I agree with the author but not with the way he presents it;

I think the message should be something like "for important data don't rely on a single backup mechanism".

This has nothing to do with Time Machine specifically, any (backup)software can fail at some point...

I've never used Time Machine specifically because I heard early reports of it not saving someone who suffered a total disk failure. I haven't bothered trying it since then. (First impressions are powerful things.)

I've used Carbon Copy Cloner (a glorified rsync + a great UI) for 5+ years and I've survived no less than two total disk failures. The best part is that CCC makes my full-disk backups bootable, so I can literally plug in my USB drive (or my 256GB SD Card--yes, I do full-disk backups to SD cards) and boot my entire laptop directly from the external drive, bypassing my (possibly defunct) internal disk. That allows me to continue working while I wait to get my laptop replaced.

When the new laptop arrives, I install CCC and do the imaging in reverse, copying the external disk to my new Mac. Then I reboot and my laptop is back to the way it was the last time I used it. (This depends on replacing my old laptop with a new one using the same hardware, but that's not difficult if I stay within a few years of the latest hardware.)

You replace an entire laptop when you have a tiny drive failure? A 256GB SSD is ~$200 depending on form factor.

I'm a Mac user and buy their extended warranty, which covers HW failures for 3 years. There are really no replaceable parts in their new laptops (soldered SSD and RAM!), which is a shame but an OK (not great) compromise considering their weight and other advantages.

Surely it's OK for a backup drive to fail occasionally. Your working drive might fail occasionally too but it's extremely unlikely that both of them fail within a day or so of each other. That's the situation that would lead to loss of data.

That's for a pure backup. For a history of previous versions, of course you'd want more reliability if you only have one copy of that history.

I think the concern here is that the backup drive could have silently failed at some point in the past, corrupting the backups, and then the primary would fail too, leaving you without any backups whatsoever.

When you only have a master and backup drive, in the case of failure, you put yourself in a critically dangerous position.

Restoring from backup isn't just stressful for the human, it's also stressful for the hardware.

When you have a single failure, it's nice to at least have TWO independent backups in case during the restore there is a second failure (either due to hardware or operator).

I had a similar problem. I encrypt the backups. The container partition that holds the encrypted partition had a corruption. TM took forever to backup, and then failed. Deleting the encrypted partition didn't solve the problem. Although this is not related to TM, it had the same effect. I didn't try to restore from this backup, looking back that was a mistake, just to see if that still worked. I had to erase the disk, create a new container partition and a new encrypted partition, and now it works.

I use Crashplan as well, two external disks, one offsite, one of 4TB with three partitions, one for TM, two to sync a backup using Superduper - the paid version that just backs up the difference. I hope this is good enough.

I use Time Machine for general system backups, but I also push a few select things like ~/Documents and ~/Pictures offsite with Tarsnap. I want to find a way to integrate Syncthing into my workflow too.

I agree with the author. I use Time Machine but don't rely on it.

I also had a disk with errors that OSX could not fix. I ended up having to reformat the drive using a Linux laptop. I have had two USB external drives totally die in ten years so I rotate between three drives.

I have cheap cloud storage via OneDrive so I sometimes create dated encrypted ZIP files and push them to OneDrive - additional backup safety. I do the same with Google Drive.

I've had issues on (10.10 and 10.11) where FileVault 2 (FDE) irrevocably corrupts some metadata and it cannot be fixed by Disk Utility. (It "works", but performance is terrible and free space is wrong.) Reinstalling by rsyncing latest backup to recreating volumes fixes it.

Also, the usual popular, secure rsync-like service: tarsnap ( https://www.tarsnap.com/ )

    brew install tarsnap

https://github.com/pieter/Tarsnap-OS-X-GUI

https://github.com/BTBurke/tarsnap-backup-osx

Generally: Probably want to validate exclusion of big files for remote backups like VMs, caches, tmp dirs, swap/hibernate files to cut costs, unless they're important / service (&& internet) is cheap (BackBlaze).

I'm convinced that time machine over a network is a bad idea. Seems convenient, but if there's network flakiness, TM will decide that the backup is bad and force you to create a new one. And the first step in that process is to delete the existing backup.

I had that happen once, when the drive I was backing up was in the process of failing. It got about 5gb into the new backup before it was totally dead. Even a somewhat flakey full backup is better than 5Gb of a 2Tb drive.

Deleting the old backup is no longer the first step AFAIK. Yes, I have seen that behaviour in the past. Now, it gets renamed and then deleted once the next backup is completed successfully.

Still, it sucks that the solution involves losing your entire backup history.

Yes, it's completely rubbish, I agree. I'm pretty sure the people who are working on Time Machine are aware of this problem and I'm pretty sure it stems from the unreliability of mounting disk images over a network. Quite possibly due to hardware issues (e.g. flakey AirPort/Time Capsules) or problems with HFS+ filesystem. At a guess, I'd say they had to do their best to work around these issues. There is no excuse though, especially when it comes to backup.

It still is on Mavericks, which is where my laptop is for now. (I'll check Yosemite, as my wife's laptop got the same message.)

I had flakey network stuff happen last night (which only seems to happen with one of my APs), and it flipped into needing to creating a new backup. The old backup is gone, the new one has gotten to about 7%, got interrupted, and now claims that it's in use.

Has the state of ZFS on OS X improved at all? It seems like the ideal filesystem for backups, but last I checked there were major issues with using ZFS-formatted external drives — enough so that the developers working on the project recommended against it.

If it hasn’t I might put together a cheap FreeBSD box for the express purpose of networked file storage and backup.

It hasn't as far as I'm aware but here's the project page: https://openzfsonosx.org

I just did the cheap FreeBSD box thing with FreeNAS and a lenovo server I picked up on Amazon, works great

I use Time Machine for full system backups, mainly for convenience if my SSD fails. I also rsync onto two separate drives the two main folders that I'd be devestated to lose: photos and music.

I also always liked the quote of there only being two HDD's in the world: ones that have failed and ones that haven't failed yet.

I had exactly that story, disk crashed, time machine not working. Fortunately most of the data on the disk was ok and I had cloud backup as a second backup. For me Time Machine + Arq for online backup makes me feel safe...

I've had the same experience, TM backups were suddenly broken several times.

Did the backup drive perhaps become unplugged or fail to unmount cleanly?

I've managed to do that to an external hard drive under Linux and had the drive labelled as read-only subsequently.

Just wondering aloud...

There is a mounting problem with OS sometimes, too. I don't remember all the occasions, but I am very careful with ejecting the disk (at the moment OS X forces me to shutdown to "eject"). Still, I'd understand if the last backup would be broken that way, but why the whole disk? A backup shouldn't be that brittle.

Sounds very similar to what happens if you unplug a mounted USB hard drive from a running Linux without 'ejecting' it first.

My understanding is that the kernel labels the device (recognised through its UUID) as 'read only' thereafter. And it is the whole device (i.e. /dev/sdb or whatever) that is labelled.

Your symptoms just struck me as very similar. I can't offer any way forward alas.

I bought a WD MyCloud that claimed to support TimeMachine, and it has never worked properly. Lacie and Apple's own Timemachine products have never given me any grief.

WD - not even once.

my solution is an external WD drive that has RAID1. It's not gonna save me from all scenarios of failure, but it does increase my confidence that if my computer dies, i'll be able to get data back. I also rotate out drives and keep them for historical purposes to have something to go back to that might be several months old, but at least have my "stuff".

I use time machine w/ 2 drives. One will always fail, but probably not 2 at the same time.

Never used TimeMachine.

I use SuperDuper to simply image my entire box to a disk every so often.

Sounds like you have a faulty drive.

I experience failures like this with Time Machine every couple of months. The only thing to do is wipe the previous backup and start anew. This happens on all my Macs so it's not device specific nor is it related to the underlying backup medium which is working perfectly. If you absolutely cannot take even the smallest chance of losing your data, TM is not a good choice. If you need something that runs for years on end reliably, TM is a terrible choice. It won't do that. If you can deal with the regular FUBAR'd backup and redoing it from scratch (so you have about a 24 hour period with either no backups or two separate backups) it's not a bad system. Sure is better than whatever Windows 7 offers which will never remove old data and will inevitably lead to full disks.

Why are the operating system supplied backups so terrible when there are really good examples to build from?

That said, in Windows 8+ the file history (not a full system backup, but a user data backup) seems to work well enough at keeping history and not filling up the backup drive. In general taking one system image from the original install gets you back up and running well enough.

Reply on news.ycombinator.com