Hacker News Reader

Comment by ilurk | Hacker News Reader

> PLANNED FEATURES: erasure coding

Now that is something I'd really like to see in Linux filesystems. AFAIK the GNU/Linux implementations of ZFS do not support ECC. Only the Oracle version.

View on HN

The problem with the erasure coding implementations: patents. An example:

https://www.techdirt.com/articles/20141115/07113529155/paten...

There are a few different profiles offered in the Ceph Erasure Coded Pool support, afaik they are based on research papers. Assume due diligence done to ensure they can be used openly (ianal etc).

http://ceph.com/docs/master/rados/operations/erasure-code-pr...

edit: Indeed, checked jerasure site and that's gone.

> Now that is something I'd really like to see in Linux filesystems. AFAIK the GNU/Linux implementations of ZFS do not support ECC. Only the Oracle version.

When people talk about "ECC" in relation to ZFS, they are generally referring to ECC memory, i.e. RAM that detects and corrects single-bit corruption. This does not need any support from software, it is a hardware thing. So if you have ECC memory, it will "work with ZFS" no matter your OS.

Erasure coding here refers to redundancy in the block storage, so you can lose a disk and still maintain access to the data. And I can confirm that the linux ZFS implementation, along with all others (as they're all based on the same codebase), support that.

Unless he is talking about scrubbing and self-healing? In which case, the same thing still applies -- all releases do this as far as I know.

See: https://pthree.org/2012/12/11/zfs-administration-part-vi-scr...

Well, scrubbing and self-healing are really just the user-facing operations enabled by the erasure coding, so it's really the same thing.

AFAIK zfs doesn't use erasure coding. It has a checksum for each block. Together with redundant storage of blocks, this allows it to determine which block is corrupt when two copies of the same block have different contents.

Redundant storage of the meta data - not the actual blocks. But at least you know they're corrupted if the become corrupted ;)

Nope, all blocks, unless wikipedia is wrong or I am completely misunderstanding the following:

"When a block is accessed, regardless of whether it is data or meta-data, its checksum is calculated and compared with the stored checksum value of what it "should" be. If the checksums match, the data are passed up the programming stack to the process that asked for it; if the values do not match, then ZFS can heal the data if the storage pool provides data redundancy (such as with internal mirroring), assuming that the copy of data is undamaged and with matching checksums."

https://en.wikipedia.org/wiki/ZFS

"ZFS can heal the data IF the storage pool provides data redundancy"

You can configure ZFS either with or without redundancy.

Yes, which is why I said "Together with redundant storage of blocks" in my original comment.

I'd love to see it to, but I've yet to see an implementation of EC that wasn't far, far too slow to be used for any kind of performant workload.

There are RozoFS it's a french company that has made a high performance erasure coding fs.

https://en.wikipedia.org/wiki/RozoFS

What ZFS implementations are you talking about? I'm only aware of zfsonlinux which AFAIK is CDDL (and out-of-tree -- but works fine) -- is there a port licensed under GPL?

Reply on news.ycombinator.com