HFS+ is crazy
liminality.xyz
liminality.xyz
On Windows/NTFS they are called alternate data streams - https://en.wikipedia.org/wiki/Fork_(file_system) - to use specify a colon and name after the filename (eg example.txt:myads).
On Unix, Linux, OS/2 etc you can find extended attributes - https://en.wikipedia.org/wiki/Extended_file_attributes - which allow storing key value pairs on a file. Restrictions exist and vary.
As for an example of them being helpful - on Windows when you download a file from the Internet using a browser an extended attribute is used to mark that. Trying to execute the file from Explorer then explains that it was downloaded and asks if you really want to proceed.
On Linux selinux can store labels in the extended attributes.
Older ignorant tools aren't going to know about this, but don't substantially harm anything. Modern tools do know about them and do the right thing. (eg copying a downloaded file elsewhere on Windows will still give the warning). The Linux GNU cp command does require a --preserve xattr flag to copy extended attributes and does not do so by default. Dropbox does support them by default and cross platform.
Full documentation is available at https://developer.apple.com/legacy/library/documentation/mac...
Or you could attach authorship, review, dates, keywords and similar metadata which would work for any file type, not just those whose format explicitly has that support.
If you want to get a handle on what filesystem design is like then I highly recommend Dominic Giampaolo's book on the design and implementation of the Be filesystem (he wrote BeFS). The book is freely available from his website as http://www.nobius.org/~dbg/practical-file-system-design.pdf and includes information about the design of other filesystems too. It isn't exhaustive, but does give a very good grounding in filesystems and does cover extended attributes.
So why wouldn't a stand-alone font file consist of a Resource fork with a single resource of "FONT" type? Otherwise, the OS engineers have to develop two entirely different ways of reading in font data. Why duplicate the effort?
The system made perfect sense, both then and now. It bothers me that so few people know anything about Mac Classic, it really was an amazingly well-designed OS for its time.
On Windows you can do the same thing with .rsrc in an .exe/.dll file.
It's possible you would ship a font as a PE .dll with a font resource, but for a single file the encapsulation is a little unnecessary (unless your application architecture already makes it easy to deal with PE encapsulation).
None of this is true for the resource fork in the Mac case.
It had a few interesting ideas. But no desktop OS based on cooperative multitasking can be called 'well-designed', almost anything could hard-lock the entire system at any time.
"In Mac OS X Snow Leopard 10.6, HFS+ compression was added. In open source and some other areas this is referred to as AppleFSCompression. Compressed data may be stored in either an extended attribute or the resource fork.[13] When using non-Apple APIs, AppleFSCompression is not always completely transparent."
http://arstechnica.com/apple/2005/04/macosx-10-4/7/#extended...
Today, resource forks are actually exposed through the extended attributes interface:
http://arstechnica.com/apple/2013/10/os-x-10-9/9/#tags-imple...
You can see the source code for all this in Apple's Darwin open source repository. Example: http://opensource.apple.com/source/xnu/xnu-2782.40.9/bsd/hfs...
Please read this, written by the creator of resource forks, to learn why they exist:
http://www.folklore.org/StoryView.py?story=The_Grand_Unified...
Extended attributes are just that... attributes. They are not the main "data" of the file. Each and every unix file system has some attributes to begin with (permissions, timestamps etc.) and they vary from filesystem to filesystem (e.g., setuid bit, immutable bit etc.). User supplied attributes just extends this concept in a natural way. All the standard tools expect to be able to call open() on the file and start reading a stream of bytes assuming there's sufficient access. That's what a "file" is.
Moreover, it's not just unix as an isolated systems. Those expectations are baked into the structure of the entire Internet. When you receiving a "file" over any medium like email, web etc., you're expecting to receive the aforementioned stream of bytes. The attributes (or extended attributes) are not expected to accompany the file data as a general case.
Resource forks on the other hand just completely work against reasonable user expectations. The example given in the OP's post is one such instance. A font file that shows up as having zero bytes to every tool that works with files including tools that expect to transfer "files" over the internet. It's just broken by design.
Resource forks are basically legacy from the original MacOS, and something that's being retained for compatibility, not something that's really a current design. The current replacement of resource forks is bundles, where a directory masquerades as a file in the GUI.
There are lots of reasons to hate on HFS+, but I wouldn't consider this the most important one.
One maybe lesser-known use for ADS: FlylinkDC++ (and derivatives) have an option to store TTH hash data in the file's own ADS, instead of in a central hash store. It means that the hash data could be used by multiple applications, but, it's less I/O efficient to make thousands of these <4KB blocks everywhere.
If you think it's crazy that a file system supports multiple data forks/steams, then you don't know very much about file systems.
If you think it's crazy that an operating system maintains backward compatibility with an older system, then you don't know very much about operating systems.
If you arrogantly make these pronouncements on your website then you look like a complete idiot.
He even says "I'm sure there's an Apple-y reason for the existence of this feature, but I can't imagine what it might be." That's literally a "i'm not sure why this is" right there!
You would expect more of those types on HN, but instead there's seemingly a crowd of predominantly hyperopinionated tinkerer-types.
Judge people less, it doesn't do you any good, and it doesn't breed better discussion.
You're doing the exact same judging by going "I'm tired of having discussions on HN for this exact same reason..".
Just do your best to contribute and don't fuel the flames when things start going downhill, it's (imo) the best way breed an overall better environment.
But GP's point gets to how the author handled that new knowledge: post an essay of cranky, self-assured tone to their website instead of following that thread to understand how and why OS X resource forks came to be. I.e. they followed the path of "anything I don't already know must be wrong":
I'm sure there's an Apple-y reason for the existence of this feature, but I can't imagine what it might be.
It's hard not to bash this kind of rhetorical mess. "I took the time to research everything up to here, then EPIC FLOUNCE!"
If you're going to write an article critical of HFS+, perhaps making an argument for a successor, great! It's a topic I'll get wholeheartedly behind. But this article is just lazy bitching on the internet, which certainly doesn't deserve to be voted up.
The central idea of Unix is that everything is a file. Files are just streams of bytes on disk. Most happen to be text. Text is just streams of lines. And then you write a bunch of simple tools that handle files, bytes, and lines, and they will combine really well for more complex tasks.
However this was (and in many corners still is) a radical idea. The natural tendency in virtually every other system, from the Macintosh to IBM mainframes, was to store data in various structured records. Whether we're talking resource forks or records, you always, always, always add structure of various kinds. The idea of just scanning through bytes to find, say, the end of a record is shockingly inefficient.
So we have the point of the article. HFS+ breaks Unix. If you don't see that, then you don't actually understand Unix. The filesystem ignores one of the founding central concepts, with the result that the entire Unix toolkit and way of thinking about the world doesn't work. Standard utilities, scripts, etc don't know that resource forks exist, and will do the wrong thing with them. If you walk into OS X and are told, "It's Unix under the hood", well that is a lie. It really isn't. You can work within it and only do Unix things and it will work, but as soon as you access things from elsewhere in the Apple ecosystem, things break in ways that they wouldn't in, say, Linux.
But we also have your point of view. There are very good reasons that HFS+ works the way it does. And it is integrated into the UI in ways that date back decades. And THIS is true. OS X was an attempt to put a Macintosh UI on a fork of Unix. And preserving central concepts of Unix really were not as important as making the UI work right. With the same features. With data being reasonably easy to port between the two systems, and programs not unnecessarily different.
Both points of view are valid. Which one matters depends on what you're trying to do. Furthermore it is natural, not stupid, that a person who has just had a major plank pulled out from under their way of understanding the world tends to have a strong emotional response.
"OMG, you broke everything! The tools that I rely on don't work and I have no idea what else is broken!" This is an extremely common response. This is one of the reasons why it can be hard for programmers to switch languages and environments.
As it happens, I personally understand both points of view. I have over 20 years of experience with both Macs and Unix. This is typed on an Apple laptop. But fundamentally I agree with the article. Apple broke Unix. I recognize that there is no solution at this point, and I mostly confine myself to living within the Unixy parts of the system. But HFS+ got it wrong and is not well integrated with the command line. I mean look at this. If you have myfile you can look at myfile. Then ls myfile/..namedfork and get an error. Then ls $file/..namedfork/rsrc and see more stuff??? And I can only know to do this if I know it is there to be seen???
That's just broken.
There were many disconnects between the two systems. See https://www.usenix.org/legacy/event/usenix2000/invitedtalks/... for some of them.
Remember that MacOS users spend much more time working directly with the filesystem, because (recent developments with Launchpad aside) there's no analog to Windows' Start Menu. Literally everything is done by navigating directly through your hard drive in Finder. So keeping related files nicely bundled together and tidy is a bigger priority.
(You can also do neat things, like record whether a file was downloaded from the Internet, and from where -- and use that data to display a security warning when a foreign file is first opened. This would be very cumbersome to implement without extended attributes or resource forks.)
For a practical example of the metadata, albeit not one from MacOS: BeOS had a similar concept of file attributes as key/value pairs, and some MP3 players would store ID3 tags in files. In a directory window, you could list any custom attributes as columns. When I was a BeOS user, that made it trivially easy to use the file system itself as my answer to iTunes: windows would show artist, album, song title, track number, and playing times (or whatever I wanted), and because you could search on custom attributes and then save those searches as virtual folders, you could easily set up the equivalent of smart playlists.
I never got to use BeOS, and Haiku seems to be stuck perpetually in the "almost there"-stage (Haven't been paying attention in the last couple of years, though). But this helps me understand why people worked and work so doggedly on reviving BeOS.
G.K. Chesterton on the matter:
In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.”
This paradox rests on the most elementary common sense. The gate or fence did not grow there. It was not set up by somnambulists who built it in their sleep. It is highly improbable that it was put there by escaped lunatics who were for some reason loose in the street. Some person had some reason for thinking it would be a good thing for somebody. And until we know what the reason was, we really cannot judge whether the reason was reasonable. It is extremely probable that we have overlooked some whole aspect of the question, if something set up by human beings like ourselves seems to be entirely meaningless and mysterious. There are reformers who get over this difficulty by assuming that all their fathers were fools; but if that be so, we can only say that folly appears to be a hereditary disease.
"Don't just do something. Stand there."
--Marvin Minsky
Only now with your comment it occurred to me it has the same sense with the Chesterton quote, that it's against "just doing something" mindlessly!
Here's a link to Minsky's Law (actually two):
https://edge.org/response-detail/11644
If I recall correctly, it's also mentioned in The Society of Mind.
"I was reading this code, and then I found this line that I didn't understand why it was written, so it should be removed."
There are wide, deep, rich, fertile fields of HFS+ criticism to be had- low hanging fruit just dripping with potential for hundreds if not thousands of snarky blog entries about problems with HFS+ for people to post to HN. This is not one of them. The filesystem situation on OS X has such a long, tortured, and widely documented history of technical and human failure that even the Cleveland Browns should feel sorry for it.
There is at least one extremely old, legacy font file format supported by OS X in which the entire font is in the resource fork. Other than that, resource forks themselves have been deprecated for the entire existence of OS X. This is like stumbling across some ancient vestige of DOS compatibility in Windows and treating that discovery like a smoking gun gotcha moment that you're certain is going to blow everyone's minds.
I might be too optimistic, but I've been assuming that Apple started development on a replacement for HFS+ as soon as the ZFS deal collapsed. I know that was years ago, but filsystems take a looooong time to get right. It's coming, guys, I promise!
I'm not against research, but I'd rather have Apple fund one of these existing efforts.
Exactly. It was also developed for an entirely different non-UNIX operating system. Resource forks were really important and commonly used and understood in Classic Mac OS.
The only crazy thing about the situation is that Apple hasn't made a more typical UNIX filesystem as the default since they retired the Classic environment. What's even crazier is that they used to support one (UFS) but removed that support in Leopard:
So it sure might be confusing (and I see how) but it's absolutely not very common.
Gosh, are you saying that the common use of ADS was a common mac pattern and people very familiar with the history of macs would understand this well?
Do you really think that refutes the point that they're confusing to everyone else? While many OSs have implementations of ADS, almost no one uses them.
It was quite common, yes, and even more extended in the past, but I'm saying something else: that noticing it and having issues with it wasn't that common. It's a leaky abstraction, but you don't often meet that leak.
Case in point TFA's issue. He has a zero-sized font file where all the data are in the resource leak. All fonts I've dealt with in OS X have been proper files, you can copy over to other FS normally.
>Do you really think that refutes the point that they're confusing to everyone else?
No, as I wrote: "It sure can be confusing (and I see how)".
But it's not that often that it has a chance to be confusing (at least in my experience -- but I've also not seen much discussion in support forums, questions from friends/colleagues with Macs etc about such as issues, whereas I've seen for many other issues).
>While many OSs have implementations of ADS, almost no one uses them.
Wouldn't that make them even MORE confusing in those OSs, the times they're finally used? As opposed to an OS that regularly uses them?
All apps designed for Macs by people who actually know what they are doing are resource fork aware, even though resource forks have gone out of fashion.
Like it was said before, resource forks have been around since the first version of the Macintosh (I believe MacOS was called "System" at that time), and it was a rather clever way to keep data such as dialog boxes, message strings and icons out of the executable file while keeping it a single file.
There's a difference between having a feature you want everyone to use, vs. having a feature that only exists for very specific legacy reasons in very specific systems-level backwards-compatibility scenarios, that you don't want anyone to use under any circumstances. Exposing the feature more than it already is exposed would be far more confusing and hurtful to new computer users, most of whom don't understand working with filesystems in general much less specific low-level filesystem features like resource forks.
Remember that you could run Mac OS Classic apps on the first few iterations of OS X, and even after you couldn't any more, Carbonized/Cocoized OS X Apps still could open old files you created, many of which heavily used resource forks.
https://en.wikipedia.org/wiki/Extended_file_attributes
You can do the exact same thing on Windows (on an NTFS volume), just go to a command prompt and run "notepad hello.txt:secret" and see how DIR and Windows Explorer deal with it.
Back when I was a kid I used to use ResEdit to change application & file icons, fiddle with GUI controls, change menu structures and so on. Happy days!
The author clearly has no idea what he's talking about, except pointing out long known issues in transferring files with resource forks to other operating systems.
More info on how it worked: https://en.wikipedia.org/wiki/Plug-in_(Escape_Velocity) (why this is a relevant article in Wikipedia, I have no idea.)
Note to anyone reading this, though: If you're designing a system from scratch, don't even think about using resource forks to store data. Just say no.
From today's perspective, they are just there for backwards compatibility.
http://www.folklore.org/StoryView.py?story=The_Grand_Unified...
If you want to know more about the invention and early history of the Macintosh in general, there's tons more at http://www.folklore.org (or in the book based on it, Revolution in The Valley, by Andy Hertzfeld).
I wonder if Apple can afford to buy out Oracle at this point, take ZFS and wind up the useless parts.
http://www.zdnet.com/article/apple-announces-zfs-on-snow-leo...
Apple could adopt it. They don't. That's probably because the desktop OS is on life support, and there are indications Apple wants to abandon it in favor of a migration to a wholly iOS model.
Oh, and the MacBook line is essentially stagnant as a product line. Basically Apple let's Intel redo the MacBook guts while its hardware teams are hard at work on improving their own architecture for the iPad Pro, which is very clearly a vision of computing that is more like iOS.
What? It has a major release every year. Arguably that's too often.
> continued lockdown of OS X
An indication of long-term investment in OS X. If they wanted to move people to iOS, they wouldn't bother.
> UI convergence, iPad Pro as a "laptop replacement"
There is almost no UI convergence. Do you even use an OS X system? The iPad Pro is only a potential laptop replacement for certain niches; for non-power-users it's more of an iPad replacement.
> the OSX Store having an even worse developer experience than the iOS store
Has nothing to do with the operating system.
> the OSX store letting a cert expire.
Has nothing to do with the operating system.
> Oh, and the MacBook line is essentially stagnant as a product line. Basically Apple let's Intel redo the MacBook guts while its hardware teams are hard at work on improving their own architecture for the iPad Pro, which is very clearly a vision of computing that is more like iOS.
Notebooks as a whole are stagnant. Apple manages to make much more money, on essentially the same hardware as everyone else, simply by investing in their brand by spending a little more time on industrial design and the operating system. They would be fools to change anything about that arrangement.
Oh, and El Capitan's fucked dev experience?
> There is almost no UI convergence.
"Almost" being notification center which is really important and interacted with regularly, I guess.
https://medium.com/backchannel/exclusive-why-apple-is-still-...
"“From the ergonomic standpoint we have studied this pretty extensively and we believe that on a desktop scenario where you have a fixed keyboard, having to reach up to do touch interfaces is uncomfortable,” says Schiller. “iOS from its start has been designed as a multi-touch experience — you don’t have the things you have in a mouse-driven interface, like a cursor to move around, or teeny little ‘close’ boxes that you can’t hit with your finger. The Mac OS has been designed from day one for an indirect pointing mechanism. These two worlds are different on purpose, and that’s a good thing — we can optimize around the best experience for each and not try to mesh them together into a least-common-denominator experience." -- Phil Schiller
http://www.independent.ie/business/technology/tim-cook-apple...
"We feel strongly that customers are not really looking for a converged Mac and iPad,” said Cook. “Because what that would wind up doing, or what we’re worried would happen, is that neither experience would be as good as the customer wants. So we want to make the best tablet in the world and the best Mac in the world. And putting those two together would not achieve either. You’d begin to compromise in different ways." -- Tim Cook
You're unable to develop software using OS X. Other people do not seem to be so impaired. If you think Notification Center is an example of important UI convergence, it doesn't take much imagination to come up with possible explanations.
"Yes, the iPad Pro is a replacement for a notebook or a desktop for many, many people. They will start using it and conclude they no longer need to use anything else, other than their phones."
So I guess maybe we should ask Tim Cook how he really feels? Maybe you should go and write increasingly smug and demeaning hacker news posts at him. I'm sure he'll be as intimidated as I am.
> "“From the ergonomic standpoint we have studied this pretty extensively and we believe that on a desktop scenario where you have a fixed keyboard, having to reach up to do touch interfaces is uncomfortable,” says Schiller.
Just an aside: what a load of absolute horseshit. Just another example of how relentlessly people fall in line with the Apple party line on experience even as experts in UX and UI say, "They are doing nearly everything wrong."
I reach up from a keyboard to a touch surface every day, and it was a revelation when I finally could. You need it maybe once an hour, but the precision of scaling and translation gestures is far greater and the operation way more natural with your hand.
So Phil's statement is either garbage or terribly dated, and independent testing bears that out. But whatever. You're saying that Apple doesn't believe the iPad Touch is a laptop replacement.
> You're unable to develop software using OS X.
No. That's not true. I just prefer not to. The SDK's first substantial change since the days of OpenSTEP was the modernizations brought on by Swift, which themselves feel dated and poorly thought out. Sadly, we don't get many other choices that can match the native look and feel. Its dated and platform locked and Apple's made sure it can't pay out well except in a few categories.
But hey, what do I know. It's not like I've been in charge of shipping successful, highly featured and widely acclaimed iOS apps... What do I know?
> Just an aside: what a load of absolute horseshit. Just another example of how relentlessly people fall in line with the Apple party line on experience even as experts in UX and UI say, "They are doing nearly everything wrong."
So are you saying that they're lying about having done ergonomic research? Also, just like your "indications" comment, this is utterly unsubstantiated. Which experts? What research have they done?
> I reach up from a keyboard to a touch surface every day, and it was a revelation when I finally could.
How could that possibly be comfortable, if your display is at the recommended 25ish inch distance from your eyes and at eye level? Did you notice we're talking about "on a desktop scenario" and not a notebook?
> It's not like I've been in charge of shipping successful, highly featured and widely acclaimed iOS apps... What do I know?
Certainly not Apple's long-term plans for OS X.
The rules you have set up for this debate are pretty one sided. Shouldn't you at least pretend to listen?
Assessing the relative value of iPad Pro vs. a notebook to different market segments is not related to UI convergence at all.
The rules are simple: claims should be substantiated. I have refuted some of your claims with direct on-topic quotes from Apple execs. All you've tried is insinuation, complaint, and appeal to authority (nameless "experts," and yourself, comically enough).
I think it is more likely that:
a) Apple is working on their own new filesystem, optimized for their use. A company that invests in their own chip designs can certainly invest in their own filesystem.
OR
b) Apple feels that HFS+ is not actually holding them back, and plans to stick with it for the foreseeable future.
iOS uses HFS+ in case sensitive mode ("HSFX"), by the way.
That said, I agree, it's a horrible design - but it's existed for 20+ years already.
For me, this is just like saying "hey people! FAT is horrible, it only allows 8 character + 3 character extension file names!"
Actually there's absolutely nothing horrible about resource forks/extended attributes in theory.
We use way worse ideas like "sidecar" files and metadata stored centrally for the same use cases, which are worse ways to handle the issue.
The real problem is the lack of agreement/interoperability in handling them across FSs (and perhaps tooling).
Mac Classic's death knell was the increasing popularity of the Internet. Run by servers that couldn't possibly store Mac Classic files correctly, because guess what? Resource forks/alternative data streams/whatever didn't exist back in 1972.
And yes I am still bitter about this.
It's amazing that we still widely use a language without a string type and memory safety like C for example, instead of something like Rust, Swift and co -- with occasional excursions to unsafety maybe for speed/interoperability with older libs, but not as the default for the whole goddamn codebase. And don't get me started in stdlib and co.
Other stuff too. A common "file resources" standard. X11. All the way to Makefiles and permissions (with stuff bolted on, like ACL). Oh, and the horrible conventions of file paths (dumping everything in /usr/bin and co, splitting an installed app into 5+ different directories for man files, resources, etc).
It's amazing how even a simple improvement like systemd gets tons of negativity from admin types and people who think 70s designs should be set in stone.
That is misrepresenting (perhaps to the point of straw man) the systemd complaints.
Maybe, but not the one's I've seen. Can you point to some collection of systemd complaints that go beyond "this is not how things used to be done"?
Sometimes you DO need to stray off the UNIX way to improve things, namely any time "does one thing well" comes to the detriment of "needs overall overview and cooperation instead of a disparate set of things that can't be glued properly for the task based on a motto meant for simple text-based input/output programs".
There are some valid concerns too, but nothing that's a show-stopper -- which also explains why the show didn't stop.
On the other hand:
1. it's the sort of mistake you tend to learn quickly not to redo ;(
2. a variety of encoding schemes were pretty common, and usually integrated in Mac browsers, mail clients, ftp clients. See for example this page from the Fetch website (http://fetchsoftworks.com/fetch/help/Contents/Concepts/Uploa...), or the hexbin(1) man page.
My personal take on what killed MacOS is this.
The MacOS was, for the start, a clever pile of kludges, for 68K series CPUs. Hot patching of routines was how you fixed bugs, introduced support for new hardware, etc. (See https://en.wikipedia.org/wiki/Macintosh_Toolbox#Advent_and_i...) (Btw, a System 7.1 source code archive leaked years ago.)
I can imagine debugging and extending it became more and more painful. And core data structure choices (16-bit-friendly) might have been becoming wasteful as architectures came and went...
So Apple had people grinding at Copland (apparently didn't quite make it), and then NeXT was bought, .
TL;DR: IMHO, that's not the main point (not either a notable one, once you're educated about it). MacOS grew to a point where it became a fragile, quite complex, house of cards.
Files showing the wrong size are a very small part of the problem. If people actively start storing critical data in resource forks (as it was done in the case of the font), a lot of other things will break. Just think of the humble HTML upload form or Git.
Almost everywhere a file is considered to be a name + it's contents. Meta-data is wide-spread, but it there's never any real data-loss when it's discarded.
"Completely unpractical" is quite an exaggeration. OS X has used them for 15 years and things are working as they should for 99.9999 of the people in 99.9999 of the cases.
You'll read more complaints/confusion about way more standard POSIX/UNIXY features that you'll see about resource forks -- which are mostly hidden under the hood.
>If people actively start storing critical data in resource forks (as it was done in the case of the font), a lot of other things will break.
You mentioned "reality" but this is a hypothetical scenario.
People don't "start storing critical data in resource forks". Apple uses them for specific things it knowns how to handle and that they don't need to be shared outside HFS boundaries.
Anyone who's ever worked with Macs - including software writers, tool writers, developers, know about resource forks - and it's fine because major tools have been rewritten to be resource fork aware.
Nothing's really gone wrong in the sense of reality and while there may be hiccups, you don't really see users actually complain about resource forks.
It's the hypothetical case where "people actively start storing critical data ..." etc. In this case, those people haven't done their homework about the mac platform. Like the Original Post of this HN thread - this person clearly hasn't read the docs re: resource forks and is hacking around, assuming that the mac should be like any of the platforms they've used before, and complaining when it is different.
There are also extensive - majorly detailed documentation from Apple themselves re how resource forks are used and in which circumstances:
https://developer.apple.com/search/?q=%22resource%20fork%22
That gives a ton of hits. Just read the docs. Really.
Again, I pull out the old Windows example moving from FAT to FAT32
8.3 filenames -> Long filenames contains a backwards compatible way to truncate long filenames back down to 8.3. So what if someone stores critical information in the long filename that is suddenly lost when round-tripping through an OS that doesn't understand long file names? SHOCK!! HORROR! Reality is no one does that. End of story.
https://en.wikipedia.org/wiki/Long_filename
Platforms are different! just learn what's different so that you don't get caught unaware.
> HTML upload form
The idea of a file doesn't change with resource forks: you upload the file as a whole and the thing uploads. Just as humble and simple. The onus is on the browser on the one side and server on the other not to drop the metadata.
The only other change might be in rare cases you might want to upload only a single fork in the file instead of the whole file. The old Mac Classic file picker had a way to this for advanced users, and there would be nothing stopping you from adding such an advanced option to any other file picker.
> or Git
Actually, something like git could probably make good use of something like resource forks if given the chance. For instance, the git pack format is essentially a relative to the resource fork format (a collection of a bunch of smaller objects wrapped into a single file). In a world of practical resource forks everywhere, you could presumably attach something like git packs of files to themselves, which could give you the benefit of a file being its own source control history without having to truck that information along as its own files/directories. (In which case you get the benefit of using that humble HTML upload form to much more easily upload a file and its entire source control history, rather than needing a dedicated git server... admittedly there would be performance trade-offs there though.)
That said, it is a fairly non-transparent and mysterious bit of functionality.
My favorite thing about HFS+ is that about ten years ago at WWDC, I sat in with the Darwin Filesystem Birds-of-Feather, and asked them why it is case sensitive. They gave me a very simple answer:
Microsoft Office.
So, on my personal Macs, I actually run HFS+X, which is case sensitive and was designed for OSX Server. Homebrew works, all Apple apps and native Mac apps work. Office will almost definitely still not, but LibreOffice will.
https://support.steampowered.com/kb_article.php?ref=8601-RYP...
https://helpx.adobe.com/creative-suite/kb/error-case-sensiti...
Although resource forks were just another data stream at the filesystem level, the OS treated them as structured data, with record types and IDs. They were used to store user interface elements--icons, pictures, window definitions, dialog boxes, etc.--as well as string lists, version information, and, yes, fonts. Even the executable was stored as a resource (M68k code that is; PowerPC binary was in the data fork). Applications could define their own resource types too.
As another poster points out, this scheme helped applications manage memory by only loading into RAM the resources it actually needed at any given time.
Resource forks were a clever way to keep things like icons, fonts, text messages and dialog box definitions bundled within a single program file (remember - on Macs the install/uninstall procedure is usually dragging an icon).
Many times I've customized programs this way.
HFS+_is brain-dead for a multitude of reasons. This is not one of them.
I know this feeling well but I don't really get how it causes some people to think "I'll write a blog post complaining about this" rather than "I'll do a quick search for this and find out the answer, which might be nuanced, historically contingent, and/or revelatory."
It doesn't have as many features as ZFS or Btrfs, but it's much better than HFS+. Plus it's BSD-licensed and unencumbered by patents, so Apple could do whatever they want with it.
That's UNIX parochialism for you.
Is there a specific reason that this becomes problematic to even most power users outside of specific files being read differently by older tools? Reading the Linus rant on HFS+ I can understand to some degree and appreciate where this becomes a problem, but it just seems like the author found a small outlier issue with how an old font was handled with HFS(+) and condemned the entire system.
cat /etc/passwd`python -c 'print "\xe2\x80\x8c"'`
This led to a security issue with Git (http://git-blame.blogspot.com/2014/12/git-1856-195-205-214-a...) and one in Apache a few years earlier.
That's not actually true. Most if not all Unix utilities have been updated to deal with resource forks, and they have been integrated into the Unix directory hierarchy by treating the actual file as a single level directory.
The fact that the resource fork is invisible by default is the most reasonable way I can think of. Other options such as concatenating all the forks together or treating the file itself as a single level directory by default would actually break everything.
The solution that they found is to have this interpretation (the file is really a single-level directory) accessible, but make one fork (the data fork) the default "bag of bytes" for most of the Unix tools.
For example:
file /tmp/Euclid/..namedfork/rsrc
/tmp/Euclid/..namedfork/rsrc: MS Windows icon resource
That was a copy of a resource-fork based font I created by typing:
cp Euclid /tmp/
The question becomes why is OSX storing font data in the Resource Fork rather than what everything else understands is the actual content of the file itself? On other file systems that I'm familiar with, alternate streams are for storing metadata. There's important information there, for sure, but I'm not aware of other cases where the metadata stream stores the data in absence of the data being present in the part of the file that everything understands is the file's data. In this case, it appears OSX stores the font in this alternate stream. This seems like an error in design when coupled with the fact that other, normal applications, can't understand the file in a way that makes doing normal file operations on it possible (attaching it to an e-mail or sending it via Skype).
I'm sure there's a reason the content is stored there that my lack of OSX experience would explain. The part I have a problem with is that reading that file provides an inconsistent experience within the operating system. Finder can see the file. 'cp' not only sees the file, but when copying it, renders a copy with the contents in the data, not resource fork[1]. It smells like an API problem; the wrong method is being used to read the file by these other programs (like grabbing a pointer to a symbolic link instead of grabbing what it points to), but I don't write software for OSX, so does anyone know the specifics of why this design was chosen for fonts and other resources vs. storing the data in the actual part of the file that other programs would expect to to reside?
[1] I'm basing this statement only on the author's description of what happened. I do not own a Mac, myself.
Also, the CPU Mac OS ran on did not support virtual memory.
To make such a system run, applications were split into several code segments. For example, no sane program would load its printing code into memory before the user actually tried to print, and individual MacPaint commands might be located in independently loaded pieces of code, too. Resource forks and their standardized format allowed that.
Once that code was in place, using it for all kinds of other data that wasn't always needed such as fonts, drivers, or desk accessories became the logical thing to do.
It just was easier, and put less of a constraint on memory to access a font as a set of resources than to write a similar, but separate piece of code for handling the reading of fonts from regular files.
See http://www.folklore.org/StoryView.py?project=Macintosh&story... for a description by someone who worked on this.
For me, the only weird thing is that they chose to use alternate forks. They could just as well have stored the resource fork data in the only data stream in a file. My guess would be that they did that so that they had the freedom to also write portable file formats to files that also contained resources (it certainly wasn't so that they could write secret messages in the System file. That happened way later, with system 7)
Info here: http://unix.stackexchange.com/questions/96491/why-does-du-re...
The last answer (marked for 0 points of course) is probably correct in this case. The file fits in the extended attributes.
How do you even build applications?
http://manpages.ubuntu.com/manpages/precise/man5/attr.5.html
It's good that we have more than one filesystem. HFS+ functionally works well for a Mac because the Mac is designed around it, but it is part of the reason that OSX falls short as a UNIX at times.
That is an odd claim, considering that the ext3 implementation has been removed from the kernel this year, and various distributions have for several years already used the ext4 driver to mount ext3 filesystems.