On overwriting disks (on OpenBSD 7.0-current) | Hacker News Reader

On overwriting disks (on OpenBSD 7.0-current)

amissing.link

29 pointshucste14 comments

For the better part of a decade there was a challenge put out on the internet and put to every recovery company on the planet for a very standard 80GB drive that contained a standard FAT32 filesystem and a few Megabytes worth of files to be recovered after it had been written with zeros from /dev/zero on linux using DD. The challenge offered $10k for recovery of the files. Once the process had been explained not a single commercial company in the entire world believed they could recover that file and no one took up the challenge and it finished up about a decade ago.

People have been doing some really extreme things to clear out drives including 5+ passes and using true random sources but the fact is it is well beyond commercial means to recover data on a drive that is simply set to all zero just once. Maybe a state actor somewhere could read the edges of the data or the theoretic differences of zeroing and its state before that 0 was written but so far no one in the past 2 decades has shown it happen. If you are destroying a drive that is highly secret that another nation might want then sure be paranoid, but otherwise there is no reason for anyone to do anything but zero out a drive and that includes almost every single company.

Most drives these days come with a secure wipe and that is the fastest way to clear them. Use that or just zeros, anything else is not making the (ex) data more secure its just taking longer.

[1]https://tinyapps.org/docs/recovering_data_from_zero_filled_h... is a similar more recent experiment but not the one referenced since that one disappeared years ago.

The article is about randomizing a drive before using it for full disk encryption. That does in fact require a random pattern. The idea is to hide the structure of the data.

>...but otherwise there is no reason for anyone to do anything but zero out a drive and that includes almost every single company.

Overwriting doesn't work with SSDs. They leave copies of data laying around on the disk if you do that. That is even if you zero the whole disk as they have extra space in the form of overprovisioning.

>Most drives these days come with a secure wipe and that is the fastest way to clear them.

Tests have shown that a significant proportion of SSDs do this badly.

A lot of this is driven by regulatory, compliance or other outside factors.

Working in finance, healthcare, and other secure industries you must demonstrate control of the environment, which includes processes to clear up data.

Once a standard is common it’s easier to follow it and demonstrate that adherence to a regulator than it is to invent your own process and convince them it’s safe.

Even in current world with SSDs that don’t follow the same mechanics as spinning disks, regulators expect a 9-pass DoD wipe because that’s what they’re used to.

It’s easiest to just pass drives to a certified data destruction company who will provide a cert of evidence they did a DoD, NIST, etc certified wipe.

Reality is seldom part of the process.

Similar is the process for password resets. Regulars ask for 90 day resets and crazy complexity schemes when NIST actually just recommends relatively sane complexity, slightly longer passwords and no rotation.

Why is Secure Erase never mentioned in these articles? Not every drive supports it but it is a more secure way to erase SSDs and a faster way to erase self-encrypting disks. Also, for disks that require a full overwrite, it puts the disk into a mode that will continue running to completion even after a power failure.

On spinning disks, it might be good to cover your deletions as you go with random data, but you can’t use dd to do that. You’d need a filesystem-aware tool. On SSDs, the TRIM function may help you approach that.

https://ata.wiki.kernel.org/index.php/ATA_Secure_Erase

Personally, I would never trust the drive to implement secure erase in a secure manner.

Secure Erase is a very good method of erasing drives. However, this article wasn't looking at how to erase the content of the drive, it was looking at how to fill a drive/partition with randomness in preparation for using an encrypted file system on it (the randomness of the data the partition is filled with is important to help hinder cryptanalysis attempts).

OpenBSD’s dd(1) natively supports printing status with SIGINFO. If you run “stty status ^T”, then during dd you can press Ctrl-T to get an informational message including progress and rate of transfer.

    $ dd if=/dev/zero of=/dev/null
    load: 0.27  cmd: dd 2358 [running] 0.37u 0.65s 1% 99k
    528102+0 records in
    528101+0 records out
    270387712 bytes transferred in 1.051 secs (257262549 bytes/sec)
    load: 0.27  cmd: dd 2358 [running] 1.00u 1.93s 10% 109k
    1467936+0 records in
    1467935+0 records out
    751582720 bytes transferred in 2.930 secs (256426100 bytes/sec)
    load: 0.32  cmd: dd 2358 [running] 1.68u 3.12s 19% 109k
    2424899+0 records in
    2424899+0 records out
    1241548288 bytes transferred in 4.834 secs (256784572 bytes/sec)

This is one of the niceties that macOS happens to inherit from the BSDs.

GNU dd has a similar feature, it prints the status on SIGUSR1. There is no keyboard shortcut though, so you'd have to open another terminal and run `pkill -USR1 -x dd`.

Doesn't pv give you an ETA if the output device size is known?

I recently did

pv </dev/zero >/dev/sdb

to zero a drive and I'm pretty sure it gave me an ETA...

You have to supply the input device (or file) as an argument, instead of using stdin. Then it will read the size of the device/file.

As far as I know, this does not work for the output device, and the size must be specified.

So:

pv /path/file >/dev/device

Will give you an ETA.

On Linux, pv can actually determine the size of the output device (using lseek on stdout), so `pv </dev/zero >/dev/sdb` does give a correct ETA (tested with pv 1.6.6 on Debian).

That seems to be exactly why it's endorsed.

> For larger disks, an ETA proves invaluable. I like pv for this task.

Or you can just use dc3dd that features a progress bar by default.

Reply on news.ycombinator.com