Hacker News Reader

Comment by leecarraher | Hacker News Reader

rm just unlinks the files at the inode level, seems like a disk forensics utility like the imagerec suite could have restored alot of the 'lost' data. In fact i've done it on source code after learning that the default behavior of untar was to overwrite all of your current directory structure. since it was text i didnt need anything fancy like imagerec, instead i just piped the raw disk to grep, and looked for parts i knew were in files i needed, then output them and the surrounding data to an external hard drive.

View on HN

Back then yes. These days with SSDs, the OS will issue trim commands to the disk, zeroing the blocks from the OS point of view, and on SSDs with "secure delete", from a forensics point of view as well.

That is very interesting, I didn't know there were SSDs with "secure delete."

I remember that Apple removed the "Secure Empty Trash" feature in OS X 10.11 because they didn't feel like they could guarantee secure deletion with their new fast SSDs present in most of their computers.

"secure delete" (actually "secure erase") is a term from the ATA standard, it's supported by practically all ATA and later SATA devices for a very long time. The idea was you'd tell the disk to erase everything, it would do the actual deletion in the background but would not allow reading the old data. This was also the way to reset passworded harddrives, you could send a secure erase command without authentication, the drive would wipe itself and once done, the password is gone.

Once SSDs arrived on the scene, they were limited by the interface as ATA didn't specify a way to mark sectors as unused. People found that their performance would degrade with use as all the sectors became utilized. But a secure erase command would mark all sectors as erased, so the drive would work like new. Later on, ATA got the TRIM (and later queued versions of TRIM) so the OS can mark specific sectors as erased. But the result is that a lot of people confuse flash sector erasing with secure erase.

Perhaps I'm misunderstanding you, but trim doesn't secure delete anything. It merely indicates to the SSD that the sector is unused so that it can reap the underlying NAND location the next time it decides to garbage collect.

In other words, the data is still there in the flash, but only the SSD firmware (and physical access) has access to it.

I think the reality is that some blocks will be lazily erased. I believe modern flash firmware periodically GCs in the background to prevent write amplification. If this is triggered you may overwrite data.

NAND requires an erase before write so I wouldn't be surprised if some controllers are lazily erasing blocks to get better long term write speeds and prevent GC hiccups.

The reason for marking it as erased is so the firmware can physically erase the flash sector. It could happen immediately, a minute later or next week. But a well-written SSD firmware will try to erase blocks any time it's not busy doing something else, as erasing blocks takes way longer than writing them.

You might be able to recover something from the physical flash, but there's definitely no guarantees.

> erasing blocks takes way longer than writing them

Honest question, could you elaborate on that? Intuitively I would've thought writing and erasing are _the same_ from a physical standpoint, insofar as "erasing" means writing zeros.

There are two operations on flash memory: "erase" (set all bits to 0) and "write" (set some bits to 1). The former is expensive, the latter cheap. Note that "write" can't set bits to 0. Hence the benefit of keeping zeroed blocks around.

That makes perfect sense, thank you.

For flash, erase resets an erase unit to its default state, which can be all 0 or all 1 depending on the technology. Writes changes bits from the default to the opposite only. Depending on the flash chips interface, you may be able to do this at the bit, byte, or block level, but changing in the opposite direction is expensive and time consuming.

In theory we could make flash with tiny erase units (down to the bit level), but in practice we don't because the extra circuitry would drive the price through the roof.

That's interesting, if nanotech (the part that's about assembling 'things' at a molecular level, hyper-grained 3D printing so to speak) really enters the economic breakthrough we've been expecting since the early 80's, I see one huge improvement for flash right here.

At least Intel (and probably others) now provide full-disk encryption by default. So the flash is unreadable by any other controller: http://www.intel.com/content/dam/www/public/us/en/documents/...

I found this article about recovering data from SSDs very interesting.

https://news.ycombinator.com/item?id=11608568

The same question was asked on Quora: https://www.quora.com/Did-Pixar-accidentally-delete-Toy-Stor...

thanks! oh those destitute days of swap as an actual resource instead of just a, something must be broken, i just filled up 32 gigs of main memory.

Yup, disk arrays can be quite a problem when it comes to forensics recovery. This has been a bit of a nemesis of mine over the years. Friends or family will decide to buy a single RAID solution for backup and configure it to write files across the disks for performance because they don't know any better. Four years later they'll come to me because something happened to the array like a failed or corrupted controller NVRAM and they want to recover the files. For backup I recommend mirrored single individual spinning disks, preferably in multiple locations.

That's really only possible on a single user system. VFX studios usually have large NFS servers (filers) typically running proprietary filesystems, and with hundreds or thousands of clients writing files simultaneously they don't get all laid down on the attached arrays in nice neat chunks representing entire files. Typically they would even try and distribute writes to multiple raid groups/arrays for a single file. Also consider the size, you were able to copy it onto an external drive to recover. Studios don't have a spare, empty filer sitting around to dump the random contents of a few racks worth of disks onto.

Reply on news.ycombinator.com