The BeOS file system: an OS geek retrospective | Hacker News Reader

The BeOS file system: an OS geek retrospective

arstechnica.com

125 pointsfogus32 comments

BeOS was the greatest experience I've ever had with an OS.

It felt faster booted from a ZIP Drive on a 160MHz PPC with 192MB of RAM than my machine feels today, and this is not an overstatement. There wasn't a single click that didn't instantly produce a response.

Everybody points to iOS's design, but to me, one of the reasons for its success is that a system that fits in your pocket feels faster than the one at your desk.

If more people had been exposed to it back then, maybe we wouldn't put up with today's software as it is.

This is something that has bothered me for a long time. I ran BeOS full time for a year or two on a 100Mhz Pentium with 40MB of RAM and it flew. No matter how high-end I go with hardware running a 'modern' OS (Windows, Linux, or OS X), nothing manages to compare. That's a large part of the reason I stole many concepts from BeOS for my own OS, although that project is largely dead at this point (too much work, too little time).

I really miss BeOS. I remember being blown away by how fast it could rip mp3s, and with how many simultaneous audio or video tracks it could handle. That old machine with it's p3 processor felt as fast and perhaps even faster than my core-i7, tri channel ram, tripple sli, raid 0 ssd workstation running *nix or windows.

Do you have a solid state drive? The amount of time I spend waiting for my computer to do what I tell it to has decreased to almost imperceptible levels since I made the switch.

I don't, but I know how big a difference they make. However, while program loading was a huge benefit to BeOS, it was a small one for me. The realtime audio stack still beats everything else, multitasking was insanely fast, etc. I haven't seen anything that can compare, to this day.

There are alternative schedulers (like BFS) for linux that can decrease interactive latencies at the cost of overall throughput.

Dominic Giampaolo, one of the authors of BeFS, is now working on Apple's Spotlight file search engine.

BeFS rocketh, check out the book:

http://www.letterp.com/~dbg/practical-file-system-design.pdf

I worked with the Be guys when they were bought by Palm. Never in my career have I experienced such an obsession with solving problems no one has. To me its kind of sad so much obviously raw talent wasted on such frivolous effort.

What problems are they solving that no one has? And what effort is frivolous?

The file system stuff they were doing was incredible and highly useful. I'd really like to have a file system where I have indexed metadata committed directly with my file writes about now. Alas, since BeFS is dead, I pretty much have to roll my own with SQLite indexes alongside flat files. At the time, sure, it may look like no one has the problem, but if it would have become viable I think that we'd now be wondering how we lived without it.

I dunno... I've never seen a system in which having substantial amounts of non-recreatable data stored "outside" of the file ended up being seamless and seemed like a net gain to usability [1]. I'm thinking about years of annoyance at OS/2's extended attributes, Classic Mac OS's resource forks, and I think WinFS would have had some similar external metadata too. In practice, it seems to me like they end up being fragile and frustrating even in a world of single-user computers (they sometimes get lost when moved to a non-native file system or when the file is manipulated by a program that doesn't respect them [2]), and become easily more trouble than they're worth in a networked world where you want to share documents and other files with users on other systems/platforms. A UNIX-style unstructured-array-of-bytes model seems to be the baseline on the Internet and any extra structure is iffy at best.

Also, I note that when Giampaolo himself was implementing the subsequent metadata filing system that he worked on (Mac OS X 10.4+'s Spotlight feature), he went with a system-level metadata harvesting/indexing model where essentially all data (except for user-added Spotlight Comments, I think?) is wholly derived from the file itself by importers that are invoked by the system to populate its external indexed cache of metadata. That way, even if that central metadata database is blown away, it can be recreated at any time. This also allows interchange processes to only rely on what nearly always works across systems: the data itself and very basic metadata such as filename and creation/modified dates. And programs never have to worry about updating metadata aside from that which is internal to the file format.

[1] doesn't mean they don't exist (perhaps even BeOS, which I haven't used much) and I'd really like to hear about them, just talking about my own experience

[2] in this case, it can sometimes be even worse than a program wrongly blowing away the "resource fork" - a badly behaved program can update the file without updating its metadata, causing the metadata to be wrong!

What I gathered from discussions on the Haiku mailing lists was that -- sometime in the far far future -- file metadata will stay but the indexes will move out of the filesystem into an external location. This makes sense, since it allows you to index complex filetypes such as PDFs and web pages without affecting file system performance. I once worked on an index_server for Haiku that worked somewhat like Spotlight's daemon. A new and much improved version of index_server is already in the Haiku repos, but it's disabled for now. This will form the basis for moving indexes out of the file system.

On losing metadata: every modern filesystem in use supports extended attributes, so this shouldn't be a problem when copying files between BFS and NTFS, EXT4, btrfs, HFS+ or ZFS. The only filesystem that will cause problems is FAT32, which most people still use with USB drives. I don't know how that will be handled in Haiku.

I don't quite follow: when you say an "external location" do you mean in file(s) in the file system but not in the file system's internal structure itself?

With regard to most file systems supporting extended attributes: that's true (though FAT32 not supporting them is a huge caveat, and I also wouldn't be at all shocked to see some issues during conversion across FSes... I don't have personal experience though), but email, HTTP downloads, peer-to-peer, etc don't tend to support them, essentially guaranteeing you'll lose them if you transfer them across the Internet.

Also, it's really nice to not have to worry about, say, an EA-unaware POSIX app losing metadata... how many old-style Mac resource forks were lost during a simple file rename before mv was made resource fork-aware!

> I don't quite follow: when you say an "external location" do you mean in file(s) in the file system but not in the file system's internal structure itself?

That's exactly what I mean. My version of index_server stored the index in a directory called "index" in the root of every volume it indexed. I'm not sure what the new one does.

Note that moving the index out of BFS was merely an idea that was bounced around the ML.

Maintaining consistency is an issue. In many cases, though, the external attributes are generally extractable from the file content itself, so you can do a rebuild if the data gets corrupted.

The BeOS mail application used the extended attributes extensively. Its mail store was basically a bunch of flat files containing the messages (perhaps in folders corresponding to the mail folders), and the extended attributes were used to index the date, sender, etc. These indices were then used to generate mail views, respond to searches, etc. - the mail app could effectively be a custom file system browser that used the extended attributes to speed itself up and quickly access and search mail metadata.

I've been wondering about this! What happened to that team? Has it been rebuilt or partly rebuilt anywhere? Did they have any secret-sauce techniques?

The company pretty much scattered. Probably the biggest chunk of Ex-Be ended up at Danger Research.

BeOS (and QNX) amazed me with how much could be done with so little hardware.

There was a book, "Practical File System Design" that went over the implementation details of the BeOS file system. Good reading for anybody interested in more detail.

PDF of said book can be found here:

http://www.letterp.com/~dbg/practical-file-system-design.pdf

It's on the author's site, so I presume it's legitimate.

That appears to have been a primary source for this article.

Leads me to think why Amiga and BeOS were deemed "ahead of its time". Is it because they never had to worry about hardware compatibility? After all they were "selecting" their own hardware and concentrate solely on software.

If compatibility weren't an issue, we might have been very advanced right now. Seems like compatibility is what stopping everything from properly advancing, except hardware maybe?

I think Amiga and BeOS were both over-fitted to their times, so they were #1 for a short time but ultimately didn't/wouldn't scale to different/faster hardware. BeOS also had the advantage of having zero cruft because it was so new, but that situation can't be maintained.

haiku-os is amazingly good for an alpha. If you are feeling sentimental give it a shot.

Also, a major new version of it was released just last week.

I've tried it out in a VM for a bit, and I must say I'm impressed, but I wonder; is there any reason to use haiku (yet?) over any linux distro for any real-world work? Does it work better on shitty hardware?

If your hardware is supported, haiku is fantastically fast on even extremely old pentium hardware. I tried it on an ancient celeron with 256m ram and it was amazing even running off of a USB 1.1 drive. The mail app starts in less than a second. The browser starts in about 2 seconds and feels as responsive as safari on my i5 macbook pro with gobs of ram.

The coolest feature, however, is how well it preforms under stress. There is a 3d demo that maxes processing resources and displays the performance in FPS. As soon as the demo starts, the CPU is pegged at 100%. Even with the demo running, the system is still fast and stable. Haiku, like BeOS, won't let one process slow the whole system. Apps still load in roughly the same times and remain responsive. I haven't tested how well it handles memory hogs, but I've read that its nearly as good there too.

As long as you are only using apps that come with the distribution you might could use it as day to day desktop, but I think it would probably be a diservice to the users and the project to recommend Haiku in its current state to anyone other than tinkerers and devs. Finding apps would be a problem, even if you didn't have to worry about whether or not they would run. A bad app will crash the whole system, although stability has come a long way in the last couple of years. You just can't forget that this is alpha software.

I think things will rapidly improve though. There is a package manager on the way which will be finding software easier. Also, I've been reading about some fantastic advances in programming language support which should increase the amount of software made available.

Edit: I'm trying to walk the line here, I'm clearly a fanboy.

Haiku hardware support is patchy. I doubt anything would have better coverage than linux on shitty hardware.

I find UI to be the driving force - you get a GUI that interacts with the filesystem much more directly than you can in any other GUI.

A problem I've found for workstation usage - the terminal has some problems. For example, I can't connect over ssh to a remote linux system and have GNU screen run there usably. There's a problem with the way colours are implemented. I understand they're workign on a tty rewrite before the beta but am unsure what the scope of that is, will it deal with this.

I received a haiku programming book from amazon just today. I'm interested in exploring this and in digging towards the OS API - seeing what they do differently to unix. Maybe I'll find cool mechanisms for prototyping that aren't available in linux but I'm skeptical there'll be much.

I thought for sure this was going to be a Siracusan epic article. Instead it's a measly 3 pages from last year.

:-) great article though about an OS I missed the boat on.

Great, now a feel sad. I miss BeOS.

Inexplicably dead comment, copied below:

dawson 1 hour ago | link [dead]

Me too :( If you haven't already, checkout The Haiku Project http://haiku-os.org/ (inspired by BeOS).

For a great tutorial on how indexed attributes make organizing large amounts of information easy and render photo organizers, book collection managers, DVD collection managers etc. obsolete, read http://haiku-os.org/docs/userguide/en/workshop-filetypes+att...

If you're running BeOS/Haiku, you already know about People and Mail. I also recommend you check out Caya and HaikuTwitter.

The part about extended attributes reminded me of my time as an OS/2 programmer, it had EAs as well but they were awkward to use, sounds like BeOS made much better use of them.

Reply on news.ycombinator.com