Nullfs: A virtual black hole file system that behaves like /dev/null | Hacker News Reader

null

github.com

94 pointssigna1147 comments

At first I thought "oh yet another college homework with FUSE API". But this is actually a non-trivial implementation since it's a kernel module, and it doesn't just discard the syscalls (and pretend it worked). It preserves file attrs, sizes and directory entries.

    > It can handle regular file operations like mkdir/rmdir/ln but writing to files does not store any data. The file size is however saved, so reading from the files behaves like re ading from /dev/zero with a fixed size.

    > Writing and reading is basically an NOOP, so it can be used for performance testing with applications that require directory structures. Implemented as kernel module, instead of using FUSE, there is absolutely no overhead for copying application data from user to kernel space while performing write or read operations.

What's a practical use case for this?

testing, especially performance testing.

Nice to see an open source version of WD My Book Live

Amazing, software truly is eating the world.

I got a helluva chuckle out of this. Thanks.

Still needs improvement to be a real time bomb.

Nice to see open source versions of core Azure services.

At my startup we have been relying heavily on the cloud version of this open-source service for our customer data. [0]

I'll try to convince my manager to see if we could deploy this on our K8s cluster.

[0] https://devnull-as-a-service.com/

Hehe. I’m reminded of the S4 storage service:

S4 - Super Simple Storage Service http://www.supersimplestorageservice.com/ « S4's state-of-the-art write-only interface removes the headaches commonly associated with reading data. »

Will my mongodb instance be even faster piping to this instead of /dev/null?

Does this also support sharding?

Yes, maximum horizontal database scaling can be achieved using shards backed by nullfs.

Also shard keys are automatically managed.

With strong consistency as well.

it will be w e b s c a l e

For those who didn't get the reference: https://www.youtube.com/watch?v=b2F-DItXtZs

This looks like just a reimplementation of the write-only memory, that was first developed in 1975 by Professor Homberg T. Farnsfarfle

FWIW a null block device: https://libguestfs.org/nbdkit-null-plugin.1.html + https://libguestfs.org/nbdkit-loop.1.html It's very useful for testing, in our test suites it's probably the most or second most used plugin.

I was wondering at first what this might be used for, but for testing it makes perfect sense.

I remember reading the BOFH series (Bastard Operator From Hell), where a horrible sysadmin would back up to null, because it was so efficient.

Everything sent to /dev/null can eventually be retrieved from /dev/random, given sufficient time.

Only if you believe that /dev/random is truly random.

Even a truly random die is not guaranteed to ever roll a 6.

“ever” implies the limit of time. Since lim_{t->Inf} p(at least one 6 is rolled) = 1, I think you could say it’s guaranteed.

if it has once, it will again

https://en.wikipedia.org/wiki/Poincar%C3%A9_recurrence_theor...

Though it does eventually roll a 6 almost always.

Which is math speak for 100% of the time.

Yet, it probably will.

I KNOW WHERE the data you read from /dev/random COMES FROM.

But I can't tell you because I would be a dead man if I would.

it's written on the menu of the restaurant at the end of the universe

However, since you're effectively brute forcing a match on an unknown key, the universe may go cold and dark before your content arises, for any content larger than 128 bits or so.

And at some point it gets more efficient to just wait until the universe randomly starts to repeats itself to recreate the content you wanted.

But you’ll restart with it in not-yet-lost-it state and will lose it eventually. There is nothing useful in /dev/random, really, apart from the entropy for crypto. Trillions of trillions of … of trillions unique virtual atoms of no particular value.

Yeah, I remember that, too. :) I think I read that even before I first tried Linux.

Reminds me of MySQL's BLACKHOLE storage engine: https://dev.mysql.com/doc/refman/8.0/en/blackhole-storage-en...

FYI, the name conflicts with FreeBSD’s nullfs [0][1], which does something entirely different.

[0]: https://www.freebsd.org/cgi/man.cgi?query=nullfs&sektion=5

[1]: https://www.freebsd.org/cgi/man.cgi?query=mount_nullfs&sekti...

This looks like it's a Linux kernel module. Not FreeBSD

I can't see how you'd even begin to think that.

I think what they mean is that https://github.com/abbbi/nullfsvfs is a Linux kernel module. Still I don’t know why they bring that up as it’s still true as the parent to that comment said, that it collided with the name in FreeBSD. But I don’t think name collisions matter anyway either, except in the case of trademarks.

People who do not understand mongodb will continue to reinvent it.

I've long dreamed of the write-only filesystem.

Your prayers have been answered!

S4 - Super Simple Storage Service http://www.supersimplestorageservice.com/

« a new innovation in cloud storage. Our advanced write-only storage provides the highest security, lowest cost, and simplest management available. »

Why would you do this instead of tmpfs?

bcs tmpfs stores the data in your memory. The module was simply created from the need to send large data into the void, where the application writing the data also depends on a directory structure to be existant, for performance testing specially. (if you want to eliminate any underlying fs layer as performance bottleneck, it can sometimes be helpful :))

Send data into the void? Consider https://devnull-as-a-service.com/

> Now with Discard protocol support! See Code for details.

I certainly didn't read that right the first time.

Unless I am misreading it, you can configure this to allow some real files to be created, so it isn't all or nothing. But of course those real files will use memory.

Reply on news.ycombinator.com