Did you actually read the article? The mirroring in question is NOT block level like you'd see with DRDB or RAID.
It's the --mirror option to much of git's plumbing, and it's not the same thing.
It's the --mirror option to much of git's plumbing, and it's not the same thing.
> Mostly it's because they have decided that RAID 1 or RAID 5 should be a decent "backup" strategy, but then there are those who believe mirroring systems is how to do backups.
I think he is implying that this is a case of the second. That is to say, I think he is saying that --mirror is not a backup strategy.
You're assuming it works like a traditional file system or block level mirror, but it doesn't. Corruption would in most cases have been caught. The weak (and accidental) link was relying on the server to give us a proper accounting of the current valid repositories.
We have established that he knows they used git --mirror, and I am pretty certain that you could not possibly know that he did not read the article.
You're better off asserting that your objects haven't changed (Which they weren't, and I agree that they should have been) and were valid in the first place (See above).
With snapshots, you'd invevitably want to dedupe them, which would be basically the same thing since it's append only, but with the dedupe infrastructure as another failure point.
The problem I think needs more attention here is ext4 silently corrupting data. ZFS has it exactly right with the built-in checksumming on write and read - it can't stop a disk going bad, but it can tell you exactly what's affected and _when_ - corruption would've been caught the moment the mirroring operation tried to read back bad data (and would've faulted the process, rather then happily return bad data).
Well, the KDE incident proves otherwise.
The next two paragraphs identified two things that they weren't doing that they should have been. Otherwise they'd just have lots of snapshots of bad data.
It would have been just as easy to push those corrupted repos to all of the backup tapes in the rotating snapshot set. A snapshotting filesystem could be a good backup (and seems to be what one of the sysadmins is pushing for).
But even more important is to fail fast and identify git repo corruption as soon as it can be detected so that further damage can be avoided.
Let me repeat what I said, italicizing the important parts:
Mostly it's because they have decided that RAID 1 or RAID 5 should be a decent "backup" strategy,* but then there are those who believe mirroring systems is how to do backups.