UtahFS: Encrypted File Storage | Hacker News Reader

UtahFS: Encrypted File Storage

blog.cloudflare.com

132 pointsjgrahamc42 comments

Love the idea of this. I've tried to use something similar (at least in function), called Cryptomator. However, Cryptomator would always choke on larger files or directories with thousands of items (e.g. my Google Photos gallery). Will be interested to see if this or another solution is what I'm looking for.

Boxcryptor is a paid solution, similar to Cryptomator. I think I gave each one a try out on my phone, but ultimately just went back to using plain Google Drive and Google Photos. It fell into the category of "what is the real tangible benefit of this to me actually?" The real cost was added complexity, decreased reliability, a worse photos experience, and possibly another subscription cost.

How do you have it setup? I had similar issues when I was trying to access my WebDAV vault directly from Cryptomator on Ubuntu. I setup Nextcloud to sync directly with my local Ubuntu install and pointed Cryptomator at the vault in the synced directory and it works great! It opens all the files in the vault almost instantly and they’re navigable as if they're local files.

Mobile is still a problem though. A gallery system would be a big help but you’re always stuck with a bandwidth limit unless you cache the vault locally which for my vault isn’t feasible and also undesirable as I don’t want the files on my device.

I just started using Cryptomator for sensitive files on a self-hosted NextCloud (since i access it via internet). The photos thing was an issue so I just sync those unencrypted and use Cryptomator for non-media files - i suppose if I had a photo that I wouldn't want exposed i'd just move it individually over.

True, the messy bit with Cryptomator is that it's not really very easily to share files with others. I've found it more of a tool really for an individual to use. I find myself sometimes training people on it a recommending it to people and then switching them to something like Tresorit when they have problems.

Thats also rather large. I have a cryptomator “vault” of approx 40 GB, performance on linux, windows and iOS seems fine.

For anyone thinking about playing with this, please make sure you note this line:

> Keep in mind that this system is not used in production at Cloudflare: it’s a proof-of-concept that I built

That's right. The Research Team does all sorts of stuff. Some of it ends up as open source projects, some of it as production software.

I find it difficult to set up a good system for encrypted storage.

1. Cryptomator: it's immature and buggy, especially the 1.5 version. See comments in forum.cryptomator. The files and folders disappear, vaults crash, vaults fail to mount, etc.

Boxcryptor is the paid version and not buggy. But it's not open source.

2. EncFS. Has security issues that haven't been resolved.

3. CryFS. Too slow and immature.

4. Encrypted backup, like rclone or duplicity. These are not sync tools.

5. eCryptfs: Used for Ubuntu home encryption (even then somewhat outdated), not for cloud.

6. AWS KMS: server side encryption; amazon has the keys.

7. Gocryptfs: It's OK. Reasonably fast. Cons: command line only, and for Linux. Uses OpenSSL library which isn't all that secure.

It seems to me gocrytfs is the best among these.

I would also add: 8. Securefs [1]

Gocryptfs has a comparison of these projets, here [2].

Focused on Windows only, to my experience, securefs is the one that is working the best as it is not using dokany but winfsp (FUSE for Windows). With all other solutions using dokany, the copy or sync of large number of files is damn slow or hanging.

[1] https://github.com/netheril96/securefs

[2] https://nuetzlich.net/gocryptfs/comparison/

What’s wrong with LUKS / tomb?

It's one big container. A small change means the whole file has to be uploaded again. If you are lucky for Dropbox blocks are synced only.

Also it lacks authentication. The snapshots of the XTS mode are prone to certain attacks.

Regarding your first paragraph, that’s always going to be a tradeoff that has to be assessed based on threat model and resource budget, since doing it differently will inherently leak sidechannel information, right?

The second one, TIL and good points!

They should use Peergos [1], which satisfies all their requirements. It's built on top of ipfs, encrypts metadata as well as data, and hides file sizes and directory structure too. A directory is indistinguishable from a small file. We can also use S3 compatible object storage, including allowing direct reads and writes from the browser. Disclaimer: I'm the founder.

1. https://github.com/peergos/peergos

Peergos doesn't hide access patterns as far as I can tell.

ORAM is pretty costly for a feature that many people don’t actually need. Yes it is log(n) for reads and writes, but the bigger downside is that ORAM by design thrashes any cache you happen to be using. I imagine this would be particularly noticeable for filename resolution with deep directories.

Plus, ORAM provides no protection out of the box for timing based attacks.

We use a similar structure to their oram, a merkle champ, with random labels. Some operations do move blocks to new labels, but reads currently don't because of the overhead as well as wanting to support multiple concurrent readers.

Awesome job! I prototyped a very similar FS for IBM Research a few years back, but I like your design better! They used it for Hadoop.

Love the crypto fs work you're doing at Cloudflare.

Interesting name for a file system. Reminds me of this: https://en.wikipedia.org/wiki/Utah_Data_Center

Also https://en.wikipedia.org/wiki/Tahoe-LAFS

While reading this I kept thinking "Isn't Utah the name of Zooko's FS?" Took me a minute to remember it was Tahoe.

There a couple of solutions that are fuse based and dealing with folder based encryption and gocryptfs has a page that compares them here[1].

It will be interesting to see how this project does against its competition.

SiriKali[2] is a front end to a majority of these projects and it works on Linux, MACOS and windows.

[1] https://nuetzlich.net/gocryptfs/comparison/

[2] https://mhogomchungu.github.io/sirikali/

One way to achieve encrypted remote file storage is to combine Linux's EncFS² with some other type of remote storage protocol such as CIFS. The basic idea is to mount a CIFS volume from a server you don't trust with your unencrypted files, then you mount that directory again with EncFS to get to the unencrypted files.

The main drawbacks are that EncFS will not hide your directory structure and file sizes and also that not all of the EncFS vulnerabilities that were discovered in 2014 have been addressed with an EncFS 2.0 release.

² or one of its similar alternatives of course

EncFS 2.0 does not seem to have been released. I don't recommend it due to security problems.

mtgx - you are dead.

I haven't looked at native OpenZFS encryption... The method I used for a while was having a Veracrypt volume locally of all my files, and zfs-sending it to a remote location. It is a large file, several gigabytes. However, any updates made locally would get synced remotely at a block level vs. a file level, so the whole encrypted file didn't need synced. Much better than rsync, for example.

I did not read the entire F'in article yet, I just stepped in while a CI/CD pipe was running at work :) BUT! this looks like a pretty accessible article about filesystems.

I also wonder if it is named UtahFS because of the NSA facility near Salt Lake City.

I don't know about the crypto design of OpenZFS, but I have been running Ubuntu 20.04 with root pool using OpenZFS crypto since it's release. You have to make a small hack to the installer to do it, but otherwise it seems to be working great.

I documented the installer hack at: https://linsomniac.gitlab.io/post/2020-04-09-ubuntu-2004-enc...

Before that I was running my storage server using LUKS on the raw devices and ZFS on top of that, since 2008, and that worked well.

> I also wonder if it is named UtahFS because of the NSA facility near Salt Lake City.

Yeah I wondered this too. They don't say why they chose that wording. Maybe it's a subtle nod to that datacenter

I'm not sure what the karma threshold is, but you might be able to open the dead comment and click vouch to make it replyable. I went ahead and did that since it seems to be a good comment. I don't know why so many of their recent comments are dead. Some I can understand, but others...

IPFS already has a storage and data layer as well as integrity and other properties. I wonder if it would be a good backend for this filesystem.

Until your pk is compromised

You'd have to find someone to permanently pin the data (nobody will be interested in random encrypted databoxes without being paid for it). And at that point you're just loosing performance compared to S3 or WebDAV.

Yes you can pay to have someone pin it--there are pinning services--but you can also just replicate it among your own nodes. It would be comparable to Dropbox plus encryption.

Another thought less well explored would be to swap space with friends such that you hold each others pins but you can't read your friends' encrypted blocks. This is kind of the premise behind filecoin, which lets you buy space (ie if you want someone to hold your blocks) and sell space (ie, mining is equal to holding someone's blocks).

I would point out that IPFS provides no encryption out of the box, you're just layering this below something that encrypts that could equally well work on Dropbox. Filecoin atleast has a integrated payment structure but modern storage providers are pretty damn cheap (I pay ~10$ for 3TB of data including ingress/egress) When I last checked by filecoin they could only barely compete with AWS storage options (which quickly get very expensive)

For clarification: IPFS has transport encryption by default. IPFS does not encrypt content itself for you.

Transport encryption isn't something I'd mention when talking about "Encrypted File Storage", I'd assume that to either don't matter or be the default.

Will keep an eye on this!

Dropbox and Google could easily encrypt the files locally with your own key prior to uploading them to their servers. But of course they won't do that, because then how else will they mine your data?

Also, you probably shouldn't trust them even if they did offer this anyway.

Sure, if you don't want search or sharing or 90% of the other features of Google Drive then encrypting on the client side makes great sense. For people who demand that their computers are useful, it doesn't make any sense.

Sharing can be accomplished like encrypted file-hosting sites do: encoding a decryption key in the anchor of the URL. Search of course is harder, but it's a bit unreasonable that it's not an option at all.

Also (and more charitably), I'm pretty sure at least Dropbox deduplicates files between customers.

If the key for the file is the hash of the file, which they encrypt with the user’s passphrase, they can prevent anyone who didn’t originally have the file from being able to read it, including themselves.

Which allows them to charge many people for “storage space” that they aren’t actually using.

Reply on news.ycombinator.com