Hacker News Reader

Comment by boomboomsubban | Hacker News Reader

>fdupes will let you make hard links on any filesystem, but hard links do the wrong thing when you edit them or need different permissions

Hard links mostly work as you tell them to. You may need to set things up properly, but it's essentially the same thing being done.

>fdupes will not let you reconcile duplicates in snapshots on ZFS

Why not? The files are kept in a hidden folder in the base directory.

>ZFS deduplication is block-based

It is also live. You don't gain Mich with offline block deduplication over file, though it is better. Live deduplication is a feature even if unnecessary for most.

View on HN

> Hard links mostly work as you tell them to. You may need to set things up properly, but it's essentially the same thing being done.

For files that are used by anything, they might be written to. I can't have a write to one file appear in a completely different file just because they had the same contents at one point. And what if my duplicate files have different owners or permissions?

> Why not? The files are kept in a hidden folder in the base directory.

Those are read-only views, not real folders.

> You don't gain [much] with offline block deduplication over file, though it is better.

In general, perhaps not. But for ZFS specifically you gain a lot by not using its deduplication system. It needs tons of memory and explodes files into fragments.

>I can't have a write to one file appear in a completely different file just because they had the same contents at one point.

The act of saving should destroy the link, though I know that isn't always the case. Still, this is a benefit of a CoW filesystem, not offline dedupe. Permissions may be a problem, I'm not great with them.

>Those are read-only views, not real folders.

Then mount them writable,

>But for ZFS specifically you gain a lot by not using its deduplication system. It needs tons of memory and explodes files into fragments

Live deduplication is valuable in some instances, otherwise there are still offline alternatives available.

> this is a benefit of a CoW filesystem, not offline dedupe

It's one method of gaining CoW. And ZFS does not support CoW between files. But whatever feature you lump it under, it's an advantage Btrfs has.

> Then mount them writable,

You can't.[1] And even if you could, you can't make a hard link between subvolumes.

[1] You can 'clone' a snapshot to make a writable subvolume, but the original read-only snapshot can never be deleted while the clone exists, so this doesn't help you fix duplicates.

> Hard links mostly work as you tell them to. You may need to set things up properly, but it's essentially the same thing being done.

That's bullshit. hard links are the same exact inode, with all that entails. reflinking is at the data block level. Totally different.

How is linking based on file and linking based on block "totally different?"

Hard links are essentially the filename that points to an inode. Creating a hard link creates just another filename associated with the same inode. Once a hard link, always a hard link, until destroyed.

Reflink has its own fs metadata including inode, with (initially) shared extents. Those shared extents can have their blocks individually and independently modified, per file. The point at which there are no more shared blocks, they're not reflinks.

IIUC: If you hard link two files, changing one file changes the other. If you dedupe two blocks, it will copy-on-write, meaning that changing one block does not change the other.

It's not an intrinsic property of file-dedupe vs block-dedupe. It's just how it's conventionally done.

hardlinks are paths pointing to the same inode - the same metadata, the same contents - they are the same file - just happen to have multiple addresses.

Inodes that happen to share blocks are not the same file. ie totally different.

Reply on news.ycombinator.com