Looks like this isn't meant to be used on a LAN to squeeze out every bit of performance, but rather to work across hosts on different local networks over the whole internet (even passing through NATs) in a somewhat distributed fashion. Cool!
We use it in ATLAS to deploy both tagged releases and nightly builds. These can be about 100GB in size each.
The releases are used for production jobs across the worldwide grid, while the nightlies are used as development targets for individual developers.
[1] Though for publishing files you have have many concurrent transactions open from different machines so long as they lock unique paths.
It appears to simply be a read-only POSIX-fs interface to Merkle-encapsulated bits distributed via http.
Ceph and Gluster are completely different animals.