HFS Origins: The Turbo File System (2017) | Hacker News Reader

HFS Origins: The Turbo File System (2017)

macgui.com

66 pointsLammy17 comments

"I don't have any documentation yet to firmly support this statement, but my impression is that the Macintosh File System was based upon the Lisa file system which in turn was similar to a file system developed by Xerox for its Star workstation."

MFS uses a linked-list for file blocks, similar to MS DOS. This is not a coincidence according to folklore:

https://www.folklore.org/StoryView.py?project=Macintosh&stor...

"Our first file system used a simple data structure that didn't scale well to large drives (in fact, it was suggested to us by Bill Gates in July 1981)"

> Moreover, the Finder had to maintain the catalog in RAM to keep track of folders, which were illusory. The Macintosh File System was not a true hierarchical system; it only appeared to be in the Finder. Folders in fact were kept as "Finder Objects" in the Desktop file.

That is so frigging cool. JBOF, just a bunch of files. The data about structure being not the responsibility, it being a flat file store.

I'm curious how something like Ceph bluestore works, what it's design is. There's metadata nodes here too I think, it's some kind of dumb-ish blockstore, for limited size blocks.

My 5 minute dunning-kruger level look up:

Bluestore is used to directly access raw storage without the overhead of a filesystem. [1][2]

"BlueStore uses the RocksDB’ key-value database to manage internal metadata, such as the mapping from object names to block locations on a disk. [0]

[0] https://access.redhat.com/documentation/en-us/red_hat_ceph_s...

[1] https://blog.acolyer.org/2019/11/06/ceph-evolution/

[2] https://events.static.linuxfound.org/sites/events/files/slid...

That's the way PKZip archives work. The directory is just stored as part of the filename. It's a little bit of a pain because it uses / as the separator character and a lot of Windows zip APIs will "fix" that for you, so there's a difference between what you get from the API versus what's actually stored in the archive entry record in the file. On the other end of the spectrum is the Amiga file system, where every file entry has an entire sector of metadata, and a directory is a linked list of those sectors.

On Apple ][ (/e, /c, etc.) there was some non-standard DOS version (I don't think it was ProDOS, but something else) that was first geared towards using the 80 column mode (PR#3), but then it had the notion of directories.... but it was still storing everything in one big "flat" list, though each "file/dir" would have a parent index, so "cd someDir" and then "dir" would (I guess) just allow files that have in their parent_id as that dir.

But after all, that's how you did any kind of "pointer" programming in language without pointers (like Apple Basic which was my first language) - you'll store things in array, and use indices (much like Databases)...

Maybe it's somewhere in archive.org...

Right? It's actually to my mind a more advanced design than HFS - just store stuff, including the imaginary hierarchy of folders that might go with the stuff. Then, you could programmatically make new folders based on a query of a set of file metadata if you wanted. So many nice possibilities open up.

The design of HFS was protected by United States Patent number 4,945,475, "Hierarchical file system to provide cataloging and retrieval of data." The patent was granted at the end of July 1990, meaning that it has since expired after its maximum 20-year lifetime.

AFAIK unlike Microsoft later on with its FAT LFN patent, Apple doesn't seem to have tried to stop third-party implementations using that.

Was the classic macOS source put out at some point?

> Was the classic macOS source put out at some point?

A large (but apparently incomplete?) chunk of the System 7.1 source code was leaked 10+ years ago, and isn’t hard to find. But AFAIK, there has never been an approved release by Apple. Apple did publicly release the Lisa source code (through the Computer History Museum), so maybe they'll do that with Apple source at some point too.

Reading the article, my impression is the author may have more source code that isn’t widely available. Probably, they know some ex-Apple engineers who have saved code from their time at Apple, and shared it with them on the condition it isn’t made public. I’ve heard of things like that happening before, for other historically important closed source systems.

HFS is in the open-source Darwin/OSX releases. But before that it was well documented in public, and the article links to the patents.

To me the appeal of the old source code is in the comments and changelogs from the original devs as a way to get inside their heads, not in their actual implementation.

I believe most of the open source HFS code was from NeXT rather than Mac OS. A lot of classic Mac OS was still 68k assembly.

eg:

https://github.com/apple-oss-distributions/hfs/blob/hfs-106/...

13-Jan-1998 jwc first cut (derived from old NextStep macfs.util code and cdrom.util code).

That's the wrong place (confusingly enough), look in here.

https://github.com/apple-oss-distributions/xnu/tree/rel/xnu-...

ok but still:

https://github.com/apple-oss-distributions/xnu/blob/rel/xnu-...

    (c) 1997-1999 Apple Computer, Inc.  All Rights Reserved
    (c) 1990, 1992 NeXT Computer, Inc.  All Rights Reserved

I think NeXT might've written an HFS Standard reader or something, but they used the Apple code for HFS+ so it ended up as a hybrid.

https://github.com/apple-oss-distributions/xnu/blob/rel/xnu-...

I wonder how they got this working on x86 with Star Trek. Porting to x86 assembly? ouch https://en.wikipedia.org/wiki/Star_Trek_project

Try http://macintoshgarden.org/apps/mac-os-71-source-code in `/OS/HFS/`

Reply on news.ycombinator.com