Venti: a new approach to archival storage (2002) | Hacker News Reader

Venti: a new approach to archival storage (2002)

plan9.bell-labs.com

27 pointsrbc17 comments

The idea of a Merkle tree[0] is used in Git and Bup[1] actually splits files into ~8k blobs, using Git as its data store.

Edit: what would be the state-of-the-art Git store format ?

[0]: http://en.wikipedia.org/wiki/Merkle_tree [1]: https://github.com/bup/bup

I recently discovered Ugarit and although I haven't used it at all, it traces its lineage back to Venti (and also Git).

Not sure how mature it is, but might interest some:

https://www.kitten-technologies.co.uk/project/ugarit/doc/tru...

Somewhat related, I posted this here some time ago: https://news.ycombinator.com/item?id=7877328

I do use venti outside of Plan 9 too. The CAS model is very simple, and the data is very simple to manage. Unfortunately, it screaks a lot compared to Plan 9, but it works, and I have daily versioning for all my files.

/edit: quoting on this site sucks, I gave up.

GlusterFS offers WORM volumes. Volumes made available with NFS/SMB.

Date: somewhere around late 2001 to early 2002.

The latest-dated reference is from 2001.

True, but how many modern file system solutions provide snapshots for all users. Fossil and Venti will do that for you.

WAFL (netapp) and ZFS (sun) to name two?

I think the poster was trying to say that this article is interesting despite its age, because versioned file systems are still a rare bunch.

I would love to use ZFS everywhere, but sadly I use it almost nowhere. I think that was also parent's point.

Other than that, even considering snapshotted filesystems, I still think venti model is worth discussing because the model is just so different. No deletion, and it doesn't expose a filesystem-like layer. It exposes a much simpler and deeper abstraction.

Interesting data point, since nearly all of my computer use is constrained to be in and around home, I use ZFS and WAFL pretty much everywhere, between the filer that I've built and the one I bought. All 'changable' data lives on them so that a system failure can be quickly recovered from. It makes a windows laptop with a small Flash drive a lot more tolerable in my experience.

If you could make it possible to use ZFS in one place that you cannot use it today, what would that place be?

The number one reason I don't use ZFS is that I have to use Linux and OS X. My data at home indeed does reside on ZFS, and I also have a few Solaris machines in the cloud, but a huge part of what I have to do, I have to do on Linux. This is not my decision, and usually I can't influence it. ZFS does run on Linux, but it's not by default, so from the perspective of the people I have to work with, it could as well not exist.

But you asked me only about a single place; my OS X laptop. After I use Solaris for a while (where I have automatic snapshots), when I come back to OS X I feel so weird that I have to remember that I can't just delete a file and recover it from a snapshot. Also, I've experienced silent data corruption on my laptops; this would be detected by ZFS (unfortunately it wouldn't be solved, as laptops with more than one hard drive are very rare, but with mSATA and a SSD, now there's at least a way on some laptops).

I also have to run old versions of Go every day; on Solaris I have versions of binaries from every commit and I can just run them to test something. On OS X is more complicated, I use venti, but since it's not integrated with the filesystem (because FUSE sucks), as on Plan 9, it's more annoying.

ZFS, Btrfs do this with COW. NTFS (volume snapshot service) and ReFS makes it COW. LVM2 offers a new thinp COW based snapshot that could make this work for any filesystem.

Edit: And Apple's CoreStorage also has some kind of COW capability at least when used for "fusion" drives. And then Time Machine and OSTree are effectively COW as well. The trick here is how to make what's written, read-only and thus write once. Btrfs at least has the concept of read-only seed devices, which can be put together in sequence, but might need some work to make it fit better with this use case.

If delaying the "write once" aspect of a volume (or tree) is OK, several of the above options offer read-only snapshots. Btrfs ro snapshots aren't writable even by root. They can be deleted or a rw snapshot can be made from them.

IIUC both ZFS and btrfs require that you be root to work with file system snapshots.

On Btrfs an unprivileged user can create subvolumes/snapshots. There's a mount option that lets them remove subvolumes they own.

I didn't know that. Very cool!

Solaris has a fine-grained privilege model where you can delegate any aspect of any dataset to any user. However, FreeBSD and Linux don't have this, so on those systems you have to be root to do snapshots.

Ah, I see. I've only used ZFS on FreeBSD so I guess I'm missing out. Thanks.

Reply on news.ycombinator.com