Is it time to remove reiserfs?
lkml.org
lkml.org
All that said, if it's hurting kernel development and almost nobody is using it, perhaps a deprecation cycle is due. Maybe bcachefs is a good replacement? Or perhaps nobody cares about efficiently storing small files these days at all, and we just all go to XFS, ext4, and ZFS. I think dropping it during a short period is detrimental to users. Maybe a two or three version warning is due.
The advantage of having the creator, designer, and lead maintainer available among free society is definitely real. Regardless of the murder - even if we can separate the man's technical work from the rest of his life activities - he's not doing a lot of work on his creation while he's in custody. He's certainly not going to be furloughed to hackathons or conferences from a murder sentence.
I'm not sure btrfs is a sane replacement, though. It's exceedingly complex, and I try to avoid needless complexity where possible.
I've not looked at reiserfs in many years though, so I could be mis-remembering here.
Yeah... especially if you tend to back up systems when you're rebuilding or replacing them by just backing up the whole disk. It's got everything, you can mount it as a loop device (which you can't do with a compressed image as someone else mentions), etc. I've got a lot of random filesystem images around, and once I trashed a fs entirely with Reiser images on Reiser, I was done with it.
For example, I have a /data drive formated xfs, then I mount a /backup formated btrfs, rsync the 2, take a btrfs snapshot, and unmount /backup.
There used to be major performance variances where xfs was better in certain workloads and ext4 in others, but those appear to have been smoothed out and now their performance seems very similar across all workloads.
> For example, I have a /data drive formated xfs, then I mount a /backup formated btrfs, rsync the 2, take a btrfs snapshot, and unmount /backup.
https://btrfs.wiki.kernel.org/index.php/Production_Users
Facebook deployed it on millions of servers. Is that production enough? Synology NAS devices also use it.
What I do know is that ZFS recently released a feature specifically for the hobbyist/frugal community. The feature allows you to grow an existing RAID array, something a financially sound business would never do. So no customer of anyone supporting ZFS would ever use this, and it took significant effort of ZFS developers to implement this. Not to mention that introducing feature potentially introduces weird behaviour in ZFS that might endanger its (reputation of) stability.
I'm super happy with it, (as my company was not in fact financially sound when we invested in our on-premise storage hardware), but if I was CEO of ZFS I'm not sure I'd sign off on it.
I can give you mine. I was working with a Raspberry Pi 3, and using a USB SSD. It's a USB2 link, so a bit choked, and I figured, hey, filesystem compression can help here, btrfs supports it, great! And it helped - you could get "real world" disk reads a good bit faster than the USB2 bus speed.
Until one day, I rebooted, and it didn't come back up. Analysis on another system was that the btrfs filesystem was just... toast. I've no idea what happened, I found some stuff that said "Oh, uh... don't use btrfs over USB, it kinda breaks in some cases...", the recovery tools couldn't even decide that the filesystem was a btrfs filesystem, and, nope.
I put data on the filesystem, I expect it to come back. btrfs broke that guarantee with a Pi full of data (nothing too important, they're just scratch systems and light desktops), so... I now stick to the boring things like ext4 that have been exceedingly well proven. Is it the best filesystem out there in terms of features? Certainly not. Am I pretty darn sure that I'm not going to trip some edge case and totally scramble the filesystem? Yes, and that's what I care about.
If you do a heavy random write workload, it fills up the disk pretty quickly and require a re-balance _before_ ran out of space.
Of cause you can do nocow on those files, but than it lost all the checksuming/snapshotting features.
I've never lost data to it, I've never tried the soft RAID modes it has though, but I've experienced it making a system almost unusably slow. SUSE out of the box with it automates a lot of it and it's pretty remarkable. Transactional mode if you want it seems like a game changer for some servers and the snapper stuff has saved my bacon a couple times. It's getting there but like I said, it needs some maintenance and just formatting a partition with it is likely the wrong way to experience it.
Well, inclusion in mainline kernels is the big one over ZFS and bcachefs, I guess.
I haven't seen F2FS before, so I'm commenting on the basis of 30 seconds of Googling, here, but it doesn't look like it supports either copy-on-write or snapshots, which are the big selling points I've heard for continuing to use Btrfs on top of a device manager.
This myth is being perpetuated despite btrfs devs (who work at facebook) stating the exact opposite many times over.
Every FS corruption and weird behavior is put aside and investigated. They very much do care.
https://lwn.net/ml/fedora-devel/03fbbb9a-7e74-fc49-c663-3272...
Please read the whole thread before repeating this nonsense, or at least every email sent there by Josef Bacik.
See also:
“Also keep in mind we pay really close attention to burn rates for our drives, because obviously at our scale it translates to millions of dollars. Btrfs has improved our burn rates with the compression, as the write amplification goes drastically down, thus extending the life of the drives.”
As with anything it comes down to money. Yes a machine going down doesn’t impact the cluster but it does impact their wallet. Every failure of a disk costs money and on the scale of the big boys that can add up to big money.
So while “the system” doesn’t care about drive failures the accountants and CFO’s absolutely care.
Just because you and I are using different meanings of the word "care" doesn't mean the point isn't valid. They "care" in that they would like to know what went wrong and study it further. They don't "care" in the sense that they suffered no real harm and no stakes were riding on any one particular server that failed. It's not just a matter of having a backup/redundancy, it's about having automated systems (or even just standard procedures that are being executed on a daily basis at that scale one way or the other) that take care of these failures. So even in production, "regular" btrfs users might have backups so "no lasting damage" would be incurred, but that's hardly the same as openly volunteering themselves for risk.
That's all besides the main point: Facebook is deploying "known good" configurations. They're using a very select subset of features. They're not trusting changed btrfs features/implementations being correct or, as was my experience, worrying about less-used/tested codepaths leading to data loss.
I've heard RAID 1 and RAID 10 modes are safer, but after the FS corrupted my data I haven't really had a lot of trust in it or the people who say again that it's ready for serious use.
I have a filesystem with an absurd number of tiny files in it. I host a statically rendered wikipedia mirror. Tens of millions of gzipped html-files with a filesize in the range 1-5 Kb.
ReiserFS is the only filesystem I know that deals even the slightest bit gracefully with this thanks to tail-packing.
As long as you don't do BTRFS-level raid 5/6. (Just do lvm-level or md-raid-level raid 5/6) don't do many subvolume with quota. (many subvolumes is fine) It's production ready.
I don't know where this "btrfs is not stable" coming from. According to HackerNews, I should have lost my data due to BTRFS 10 times already.
"btrfs is not stable" is coming from people for whom btrfs has not been stable. Why is that so hard to understand?
"The Btrfs code base is stable"
And https://btrfs.wiki.kernel.org/index.php/Status is exactly what I said in the GP. Besides quotas and RAID56, everything is stable.
It is good for most use cases, but you have to do your research
I would rather use other filesystem
Relatedly, it would be pretty cool to be able to build your own filesystem features as layers on each other (such as block-level compression or encryption, block-level forward-error correction, cache promote, striping or other redundancy etc.). A modular FS with pluggable (but also upgradable) components and some sort of structured feature-flag schema metadata, and a way to migrate between variations (such as switching to a different compression or encryption algorithm).
You may also be interested in FreeBSD's GEOM subsystem for storage.
File systems aren't relational databases, but more like NoSQL half a lifetime before it was cool. They're typically backed by the same data structures and solve the same problems.
https://encyclopedia.marginalia.nu/
The data is a bit stale right now, since there's a new dump available and I haven't converted it yet. But it's pretty easy to set up. Just grab a zim-file from here[1] and unpack with an appropriate library with an optional step of parsing and transforming the DOM then save into gzipped html files and set up a server for unpacking and serving those.
The conversion job takes a few days but it's not that bad.
[1] http://wikimedia.bytemark.co.uk/ (use a mirror if it doesn't work)
The systems were hosting map tiles of the entire Earth, and a lot of them were solid blue (ocean) or solid beige (unoccupied land). Those files were something like 60-300 bytes, going from memory. Reiser was basically the only FS that would handle that reasonably.
> ReiserFS is the only filesystem I know that deals even the slightest bit gracefully with this thanks to tail-packing.
Are there any newer file systems that cope with this use-case?
Have you ever tried Kiwix (https://www.kiwix.org ) and looked at the ZIM format (https://en.wikipedia.org/wiki/ZIM_(file_format) ), in particular those for Wikipedia (https://download.kiwix.org/zim ) ?
Ah, but that's all I can think about when I read about reiserfs. For anyone who is unaware, the author, Hans Reiser, killed his wife and hid her body. There was a long and public investigation and he was found guilty. He later produced her body as part of a plea deal.
[1]: https://web.archive.org/web/20060318185107fw_/http://www.nam...
Scientists also generally don't name their discoveries after themselves. The honor of getting someone named after yourself is something that your colleagues are supposed to bestow in recognition of achievements - sometimes long after the person in question is long dead.
Sometimes we have no choice as to whether or not we can use the work of someone who has done bad things - Einstein was quite awful to his wife, for example, but it's not practical to ditch relativity just because he sucked on a personal level. But there's plenty of replacement filesystems - should there be no discussion as to whether or not we should weigh the moral issues alongside the technical ones?
FWIW at some point it also had quite severe stability issues. I'm sure those were solved but that might have contributed to the fact why there was no effort to fork it but instead people went with ZFS, XFS and eventually ext4
If I choose to use a slightly inferor alternative because the name makes me uncomfortable, that is perfectly legitimate.
If using ReiserFS costs me a number of weirdness tokens (like using it in a workplace, and the relevant decision makers are likely to know about the murder now or in the future), then I may choose not to use it to conserve those tokens for other controversial initiatives that I care about even more.
Neither of those decisions may be "rational", but it's also not moralising. I simply have to take a complicated social/emotional calculus into account, something that most do unconsciously 24/7 without thinking "hmm, today I shall get on my moral high horse and decide that this is moral and the other thing isn't".
If you care about those things very deeply, then perhaps that is exactly the right thing to do. Lots of people don't do things because of ideological or personal beliefs, even if it limits their options.
Data loss bugs sure sound exciting, but I'm old, and stable, and I prefer my file systems like myself.
Admittedly, it's not a great signal if the candidate hadn't heard of it or used it depending on their career length because plenty avoided it and happily lived in other FS worlds or avoided certain hype, but it's a fairly good positive signal if someone actually starts talking about it knowledgably in some way regardless of it on positive or critical notes--just have to keep survivorship and confirmation biases actively in mind when weighing these things.
I'm not entirely sure what's gained by asking a candidate about FAT16.
It's like asking about Pamela Jones, from Groklaw, and her red dress. Fun trivia, but it tells you nothing about a person's current technical skills (or even their past ones!), just their ability to remember random names/facts, and excludes newer/younger staff.
Back then reiserfs was too new, and therefore not something I'd consider deploying to production systems. These days? Obsolete in practice. So I guess there would be many like me who have a vague memory that it was gonna be cool, with plugins and stuff, but who never actually used it or tried it.
(Maybe it's different if you're interviewing/grilling somebody for a very FS/Storage-heavy NAS-producing company, but there I'd expect conservatism to be highly appreciated?)
I appreciate the ability to shrink/grow an existing ReiserFS, which I do often enough as I add new storage to my LVM layouts.
I have never lost a ReiserFS filesystem due to corruption, so the tooling seems sufficient for me.
I hope it stays in the kernel for a long time.
When I started in Linux full-time, back in the early 2000s, I was presented with different fs options - the installer did not give me any information on benefits, or on which ones were considered "stable". ext2 was available, but it also had a built-in version number, and I feared that once ext3 eventually rolled over, it might be incompatible. So I chose the fs that came with a name attached: ReiserFS. If someone was willing to attach their name to their code, I figured, that must be some quality code.
ReiserFS served me well, even though it always was a system lurking in the background. I was not a storage specialist. I was just a guy doing work on a Linux desktop, and ReiserFS "just worked". Occassionally, I put in a new disk, and watched the progression of other fs. Eventually, I switched over to ReiserFSv4.
Fast forward a few years, and the whole murder thing happens. Immediately the thing that made ReiserFS stand out for me becomes radioactive - the name is burnt. And I see distros slowly phasing out advertising the option of having a ReiserFS partition. Yes, it was still there when you searched for it, but it didn't feature prominently in the installers anymore. And it is obvious that the code becomes stale - arguably understandable, who would do OSS work on code that is branded the name of a murderer?
Twenty years later, I am still a Linux user. The last ReiserFS disk got shredded a year ago. Feels like the end of an era.
Lessons learned: Naming matters. If you attach your name to something and want that project to survive, maybe do not commit violent crimes.
Add to that some people sometimes decide to give a famouse surname as a firstname for their kids. There is a non negligible number of people sporting the Lafayette, Napoleon, Lincoln as first name for example that could do something bad anytime.
Imagine codenaming a chassis "Drake" today, where even the famed hotel in San Francisco changed its name (now it sounds like a generic chain hotel — "Beacon Grand").
Use Junipero Serra's name? Better hope your offices are vandalism-resistant...
I formed the impression that Hans Reiser's somewhat bombastic approach did not interface well with kernel development anyhow when Reiser 4 came out. I remember reading some rather strange discussion threads at the time, including a - perhaps tongue-in-cheek - suggestion that Linux use Reiser 4 as its VFS and implement all other filesystems as plugins to it.
At one point, employees of his company were continuing to develop Reiser 4 (which they'd been working on for some time) and try to get it merged, even after the original author was no longer able to work on it. I presume those efforts were eventually dropped, which is very understandable but sad for the other developers.
There also was weird artwork to go with all of that:
http://web.archive.org/web/20070219175315/http://www.namesys...
Wow! You weren't kidding. Those really are something else.
I'd love to have this sort of functionality available (ISTR it has potentially other useful properties - e.g. new properties / metadata attached to files can automagically be handled by things like tar).
My memory says that concerns around overlapping file and directory behaviour caused pushback here. I think being able to hardlink to a directory was the issue (if every file can be a directory then how do you avoid directory cycles) but I may be mangling the details.
The issue as I remember is that ReiserFS was developed as a commercial product by Reiser's company, and after the trial started it fell apart, and with that the original developers had to find new jobs.
It's possible the commercial nature discouraged third party contributions -- who wants to effectively be an unpaid employee?
It also was rather troublesome with frequent stories of corruption, and at that time they were working on ReiserFS v4, which meant that anybody interested in filesystem guts was probably waiting for the next version to show up, rather than spending time on a very complex piece of code that was about to become obsolete. Kind of the same problem that Perl suffered from.
And after all was said and done, ReiserFS 4 was still incomplete and extremely experimental. The list of people looking to polish up a very complex piece of software without the original team's assistance couldn't have been very large.
I think all these things added up together and quickly finished it off.
Oh, I don't know, the Linux maintainers who have related to the software rationally and sensibly since Reisers' incarceration?
Silly and lofty "lesson".
The incident was really eye-opening in how fragile was my data: if .so files were not affected, I would have continued to use the system and erroneous data would silently be propagated to backups.
- someone posts a patch to enable refactoring which reiserfs code was blocking
- discussion goes into a deprecation plan, with someone suggesting reiserfs deprecation follow the same as xfs v4. These have formats have year 2038 bugs. Slated for code to be removed come 2030 by starting now
There is pretty much no way to allow people to 'volunteer' to do this kind of work without giving someone an incentive to get as many people into prison and using it as a giant slave work camp.
Does there have to be explicit kernel-level support to support a filesystem? Even if they remove support, does that stop anyone from releasing a kernel module and supporting it?
Your kernel has to have compiled-in support for the filesystem of the drive you store your kernel modules on.
Thats how people boot from ZFS
I think some of the published cases where userspace would have been "broken" and Linus got angry were fairly minor, as in you might not even need to fix the software, the user just has to be aware of different defaults. Whereas updating a kernel to a version where the filesystem is not supported will definitely render the installation unusable.
In some technical sense, the filesystem is just an implementation detail of the kernel for things like 'read', 'write', and 'mmap'. Since this is an implementation detail, it can be considered 'not part of the public API of the kernel'.
Put differently, the promise not to break userspace is aimed at developers and not at actual users. The case of changing defaults is then a problem of developers because their programs are all of a sudden behaving differently.
In userspace, I run "mount -t reiserfs ....", and it mounts a filesystem. If the kernel deletes the reiserfs code, the userspace mount command / kernel 'mount' syscall will fail.
That seems like a very clear userspace thing that breaks.
Is there a reason that the mount syscall no longer accepting a previously valid argument isn't a userspace breaking change?
In any case, the ship has long since sailed on removing filesystems from the kernel. The original extfs is no longer available, nor is xiafs, nor are some virtual filesystems like devfs, etc. "A command that used to do something no longer does because the feature is no longer in the kernel" is commonplace in Linux history and is not considered "breaking userspace".
If paragon can get their cribbed together ntfs driver included in mainline then why can't reiser4 get merged?
Past HN thread: https://news.ycombinator.com/item?id=2131221
I don't know anything about it and whether other contributors are able to give as much input as he could but judging by the comments of unfixed bugs and few commits, it would sound like it is a big risk of stagnation. No?
Let history be. But rename the project to something like ChameleonFS. Hans Reiser's mentions will still be in the git history.
Not that I think that's good rebranding, but it did amuse us.
Then after some management changes, everything that made it great melted away and we were left with a job that underpays, has poor work-life balance, doesn't appreciate employee time/contributions, etc. It was still a pretty difficult decision to leave; management issues notwithstanding, the group of coworkers I actually worked with every day were - and still are - really great.
https://lkml.org/lkml/2012/12/23/75
I was surprised to see that Linux has in fact removed other filesystems in the past (and I had to look up the word "senescent").
So the real news to me here is that a somewhat "major" break in userspace is being considered.
Sure, ReiserFS might not be getting a lot of new installs, but the fact that people have submitted fixes to it within the past few years means it has some. There are installs of it out there. Wilcox is considering breaking userspace for some users.
Linux is famous, famous for keeping backwards compatibility for almost everything forever. The stories of Linux backwards compat are up there with stories of Windows still supporting AUX and CON and other special file names. Is "WE DO NOT BREAK USERSPACE" still worthy enough to be shouted at the top of your email lungs?
AFAIK, BTRFS has everything Reiser has with a far better maintenance and ongoing feature expansion.
Really, the only reason to use reiser at this point is because you are dealing with data that you can't, for some reason, move to a new filesystem yet you still want to have updates to the kernel.
I mean, yes, your filesystem becomes inaccessible, but any programs that you could run if you could access that filesystem by some other means would still function. None of them depend on ReiserFS specifically for their function.
Userspace breakage is more about changing interfaces in such a way that applications using that interface change their behaviour somehow. This does not apply to removed drivers.
If you run a stable distribution, you stay on the older kernel series until you upgrade the entire OS, by which time the release notes would say "Removed support for ReiserFS"
And then there you can do the "tar -c -C /reiserfs . | tar -x -C /ext4" dance (or whatever other preferred copying mechanism), before the OS upgrade.