ReStuff | Hacker News Reader

ReStuff

restuff.jcs.org

155 pointszdw53 comments

Pretty wild that this is necessary, but I guess that's the cost of trusting archival to proprietary software:

"The StuffIt and StuffIt X formats remain, unlike some other file compression formats, proprietary, and Smith Micro Software charge license fees for its use in other programs. Given this, few alternative programs support the format."

https://en.wikipedia.org/wiki/StuffIt#Overview

Well, any patents would have expired by now, right? So how could they stop a clean-room re-implementation? (That is, one person/team describes the format, another person/team implements a decoder from the description).

Compression isn't super complicated generally. I bet with a good number of input output samples it wouldn't be hard to reverse engineer. In that case, you wouldn't even need the two teams.

Draw a Venn diagram of people who 1. have the interest to do it 2. have the skill to do it 3. have the time to do it.

It doesn't have to be one person. If 1 is strong enough and related to revenue in some way, 2/3 can be solved by throwing money at the right people.

How much revenue are you thinking there is in converting archive files from Stuffit versions 4 and 5 (released 1995-1997) to Stuffit version 3.5 (originally released 1994). As far as I know, neither has been commonly used since the release of MacOS X in 2001 (I mean, older Mac OS machines were surely being used for several years after that too, but it's been a while). It does not seem like a particularly likely revenue opportunity. Like most other archaic orphaned file formats, with a few exceptions.

I'm not totally sure who the audience is for this hobby project even! It says people running Mac OS System 6, which was released in 1988, replaced by System 7 in 1991. (If you were running System 7,or system 8 released in 97, or system 9 released in 99 -- you wouldn't need this converter). I do wonder who these people are running a 1988 Mac OS release, on what hardware, and what they are doing!

A Mac OS System 6 machine CAN connect to the internet, but barely, I think this may have been the first version that did TCP/IP built into the OS, but it was finnicky. It got a lot more solid in System 7.

TCP/IP wasn't bundled with the OS until System 7.5 (in 1994), before that it was sold separately ($60!)

thanks for correction!

This makes me think. Are there consultancy shops (or individuals) that specialize in reversing/opening file formats/protocols etc?

There's a bit of that happening in security research, so in that way, yes. A quick search suggests that they exist independently as well, for example http://www.noat.com/

Nothing to do with proprietary versus open. For instance, a Linux ext4 filesystem image cannot be mounted by the ext2 filesystem. That would be forward compatibility.

There is backward compatibility: ext4 driver mounts ext2 image.

You can have ext4 partitions that are readable by ext2 drivers, you just have to disable certain ext4 features.

But more to the point: If you want to open ext4 on a system that was written before the release of ext4 you can just implement a program that can read ext4. Meanwhile apparently the simplest way to open a StuffIt 5 file on System6 is to spin up a System7 VM on a different computer, extract the file with an installed copy of StuffIt 5, compress those files with a copy of StuffIt 3.5, then move that file from the VM to your System6 computer.

The File Formats wiki seems to indicate The Unarchiver and a few other open source tools support this format:

http://fileformats.archiveteam.org/wiki/StuffIt

Edit: as does the ReStuff page actually.

The Unarchiver has been acquired and is proprietary now, I believe.

The command-line tools are still open source though:

https://theunarchiver.com/command-line

I'm not sure there is any legal basis for being able to charge a license fee for using a file format, only for a specific implementation of that format's codecs (unless it is patented, which I don't believe stuffit is or was).

I really hope that the company behind Stuffit does not come in and send a C&D or something as we’ve seen in so many other heartbreaking cases.

Cease and desist for what? Reverse engineering their file format? Doing that is totally legal.

1) Oracle is the likely winner of the still-ongoing Google v. Oracle case. If APIs are copyrightable, file formats almost certainly are.

2) Even if it were legal to copy or reverse-engineer their file format, the legal team at Smith Micro believes otherwise, and doubtless has amassed a considerable war chest to fight in court for that belief. Do you really want to risk litigation from them as a small-time/OSS developer? It's not enough for something to be legal in order to do it. You have to have reasonable grounds to believe you will spank any challengers to the legality of your actions in court -- or such potential challengers must have been already spanked to the extent that any other challengers will think twice before suing on the same grounds. It's not fair, but it's how the system works.

The OP has a "How it works" section that explains they run a (single) Mac OS System 7.5 VM running actual copies of Stuffit Deluxe versions 5.5 and 3.5.

It doesn't sound like any reverse engineering happened.

Is this illegal in some way? I mean, it's not a copyright violation if those copies of Stuffit were licensed and not pirated. Is it some kind of shrinkwrap license violation to put it on a VM that accepts jobs from the public online? I dunno. Probably not? Does it seem likely that a company that no longer makes software for this format is going to come after someone for providing a translation service online for a pretty niche hobbyist audience, that they are charging no money for? Doesn't seem very likely to me, I don't know why they'd do that.

Some companies (or some people at some companies) have seen similar setups and said “if someone wants to use the functionality of our software, they need their own license”.

I don’t know if the company has any legal right to send a C&D, but that has not stopped some in the past.

Yeah, I don't know if that's legally enforceable or not. Yes, they can still threaten legal action.

It seems pretty unlikely to me they will, for software they don't sell anymore, to a project that is not charging any money, and has a pretty niche audience (there aren't that many people that have use of this service).

No?

> a breaking change in StuffIt 5 makes archives created in this version unreadable in older versions of StuffIt

In the predominant software culture, we don't regard that as a "breaking change". Breaking is when content created with older versions of software doesn't parse under newer versions.

I don't think you're right. When MS Office started using the new formats it was considered a breaking change. I think you're too fixated on one point of view.

This is probably because disruption occurs when an old document is successfully opened by the new application, changed and then inadvertently saved in the new format.

That has never happened and still does not happen today with the newest Office. The document will be always saved in the correct format, and if it's the older one, it will be 100% compatible with old Office.

Windows: backward compatibility at all costs.

Apple: let developers fix our compatibility issues!

StuffIt is third party software (Aladdin, now Smith Micro)

The issue here is lack of forward compatibility: the old version 3 program not being able to read files created by the version 5 program.

This is not so different from a Windows 10 program not being able to run under Windows NT 3.5. E.g. due to using some new API that doesn't exist.

However the zip files created by Windows Explorer can be opened without problems in Windows 95 (with additional software, OS support was only added 19 years ago in Windows XP). Similarly Excel 2019 still has an option to safe your files in the Excel 97 format. When it comes to file formats Microsoft is very dedicated to both backwards and forwards comparability, and companies love them for it.

That's great, but in the case of zip files it's because the file and compression format hasn't materially changed since the 1980s, and in the case of Excel because Microsoft controls the format. Apple does not control the Stuffit format and never has (and AFAIK never distributed their own software using Stuffit). Stuffit's compatibility issues are due to the people who make -- or at this point, I believe made, past tense -- Stuffit.

Sure, the zip format hasn't changed much in 30 years, ok.

The Stuffit format was also never owned or even licensed by Apple, it was just a third-party vendor who's proprietary (non-standards-based) compression format became popular on pre-OSX Mac OS. But yeah, that vendor chose to make some changes to it to improve it, such that archives created by later versions could not be opened by previous versions, unlike Microsoft's practice I guess.

But as long as we're comparing... the target OS for this converter is Mac OS System 6, released in 1988, superseded by System 7 in 1991. In 1988, Windows 2.0 was released, windows 3.0 was released in 1990. Can zip files created by Windows Explorer be opened without problems in Windows 2.0? I mean, sure, if the format hasn't changed I guess so?

Googling though.. it looks like if you are creating a zip archive using WinZip, they recommend you choose a "legacy compression setting" if you want to be sure it'll be compatible with older versions. Otherwise it may not be. I'm guessing Windows 2.0 is about as older version as it gets. https://support.winzip.com/hc/en-us/articles/115011349147-Is...

I wonder who's still running Windows 2.0 to find out! I also wonder who's still running Mac OS System 6!

This leads me to a related question: If stuffit existed to compress and preserve resource forks pre osx, why is mac software these days so commonly distributed as a .dmg instead of just zipping an .app bundle (which works totally fine)?

It is solely so that people can include a shortcut to /Applications and style the dmg window background with a bitmap that alludes to dragging to install?

I know that at least hfs+ still supported resource forks, but I don't think any modern .app bundle has any anymore, or has for years. (Does APFS even still support storing a resource fork?)

> It is solely so that people can include a shortcut to /Applications and style the dmg window background with a bitmap that alludes to dragging to install?

I believe so. The first app I checked, DaisyDisk, distributes as a zip file. Like you said, most often people distribute dmg and it's to give hints to drag it into /Applications. I really like the .app system, but always found the UX of download, unzip/open DMG, drag to /Applications as very awkward. I figure Apple stopped caring about that when they added the Mac App Store in 2011.

Yes APFS still supports resource forks.

But with the death of Carbon, I don't think there are any APIs left to read the actual resources. So it's limited to storage and reading the raw data stream.

App bundles don't use resource forks, but they do often use extended attributes and aliases and things. Although it's possible to make a zip file with those things, not every zip program does it by default.

But mostly I think it's just tradition, and maybe some small secondary features like being able to show a EULA when someone double-clicks the disk image.

I've been hanging on to a Mac G3 Wallstreet (which actually still has reasonable battery life) simply because it's a pre-OSX computer and I've got the Myst CD-ROMs. I don't know if you're making copies of everything that's been uploaded for "restuffing" but I'd guess that Jason Scott would love to have a copy (note that I realize that it's probably not legal to distribute most of it but it would be wonderful to get it archived while we can).

Why doesn't someone port 7zip to old computers and be done with it?

In that case you'd still need a way to convert stuffit files to 7zip - the reason this is needed is because it was common to distribute Mac software as StuffIt files back in the day, so there are large archives in the format that people want to read. There exist no 7zip archives of Mac System 6 software.

It's chicken-or-egg. No one (re-)distributes old Mac software in 7zip format because there's no 7zip port, and there's no 7zip port because all of the software is in stuffit format.

> ReStuff is a SaaS that automatically converts StuffIt version 4 & 5 archives to a format readable in StuffIt 3.

is a SaaS... Dear Lord. Imagine having tar as SaaS or something.

Imagine needing to install something to covert a single file in a prehistoric format. Dear Lord.

It's an one-off need for the vast majority of people who even have one to begin with. So an online converter beats an installable utility hands down in virtually all aspects that matter. And if one needs to convert in bulk, the DIY instructions are on the page.

Why are you so snarky? Oh the horror. You need to have a browser to use this SaaS. You do not necessarily have to install anything to convert a single file in a prehistoric format. It also depends on what you mean by install. Download a portable version of a program that does it for you? Technically no installation required there. Using your already existing package manager? Anyways, this is besides my point. See below. Plus why stop after a single file?

If you cannot see the absurdity of using SaaS for something like this, or for something like tar, I am sorry. Keep using SaaS to archive files, too. Why install tar when you can just install a browser and visit a website where you can upload your stuff to just to have it archived on and by their server and then downloaded by you? I brought up tar because it is pretty similar.

What does an online converter beat and why exactly? Just because it is something you may rarely do, does not mean that doing online is a good idea. I rarely edit photos. I am not going to use an online photo editor for my photos for obvious privacy reasons, for example. If it is more comfortable for you and you do not care about privacy, do it for all I care. I am free to form my opinion about the absurdity of SaaS for everything that is trending. Online text editor!

> So an online converter beats an installable utility hands down in virtually all aspects that matter.

This is just your opinion. I shared mine. You may not want to install tar either and use a website for archiving and extracting tar files, or tar.gz files, or whatever else. I have no problem typing "pacman -S tar" in this case. It is a bit of an unfair comparison because I personally often use tar, but I also often use my browser, so might as well use an online tool, especially if I spend lots of time in the browser.

Go read the "How it works" section on the linked page.

Why not do this locally, why would people simply trust the website to not harvest their data? This reminds me of those conversion websites to convert various documents when it can be done locally in a much more optimized and secure way.

Some people also use online photo editors for their photos. But from what we could gather, even privacy-oriented people can be quite silly, for example recently they just uploaded a photo of their face (to some random, untrustworthy server(s)) to get another, "anonymized" face back. See the problem with that? :D

I brought it up to their attention, hopefully for the better.

Why is this trending #1 on HN? Granted, it's a cool tool and would likely save some people alot of time as they wouldn't have to invest necessary time and effort into setting up a virtual machine with an older version of Mac OS. What concerns me about this Saas is that Encoding/Decoding as a service is about a silly as Encryption/Decryption as a service. The project may as well say: "Give us your valuable information and intellectual property, we promise not to look at it and we promise that it will actually be immediately deleted."

I don't mean to be altogether negative or pessimistic here. I have no beef with Joshua Stein and absolutely no evidence to suggest he isn't anything but a super honest and chill dude. I'm sure he is totally trustworthy and I think this is a really cool addition to his portfolio. I absolutely do not have anything negative to say with regard to him or his fantastic decision to host this Saas tool.

My negativity and critique is solely focused toward the HN community that upvoted this Saas tool when it (to the best of my knowledge) cannot be either paid for as a commercial saas under a contract guaranteeing security considerations and custody of information or alternatively self hosted in either a distributed binary or source form. To be absolutely clear, I am absolutely not suggesting that security sensitive Saas tools should be open source of offer self hosted variants to merit upvotes on HN. I think it's great when creators are able to monetize their fantastic software.

At the end of the day consider this meta comment to represent nothing more than my personal opinion on what any individual or organization should consider safe to outsource. A free Saas tool that decodes a deprecated archive format should never be used with anything sensitive regardless of who is offering the service.

Astute readers should note that my game theoretic motivations with regard to sharing my contrarian views on the top trending link are clearly nothing more than a pathetic effort to seek attention and shouldn't be taken seriously. :)

Perhaps some people are simply happy to learn more about computer history and the existence of emulators which make such projects possible even if they don't intend to use the service.

I think you hit the nail on the head! I wholeheartedly agree that showcasing cool technology is absolutely in the spirit of HN and that ReStuff merits exposure. As you described, I myself enjoyed reading the page and learning about it. :)

In addition to the sibling comment about computer history, a lot of HN content that ends up front page is focused on preservation of aging content whose formats have gone out of vogue or whose content/platform has been replaced, edited or discontinued. I’m usually pretty meh on some of these (like, I found the recent Internet Archive stuff about Flash good news but not compelling to read about in depth), but this particular link struck me as a curiosity just by being an unusual niche for this site and audience... and because I was one of the tiny population who was raised with and (almost) never deviated from the Apple ecosystem. I’m fascinated and a little tickled by the fact that that old island still has someone in the lighthouse. It’s not a platform I have any interest in using anymore, but my inner child and teenager is glad someone’s still tending the garden. (Many apologies for the metaphor soup; credit to sleep deprivation)

I also grew up on Macs, but I swore off Apple in 2009. I keep a few old systems around, though, since I'm a nostalgic person. I just got my Power Macintosh 7100 on the net yesterday. There's still a couple of Hotline servers up. Good times.

I’m curious what in 2009 was the tipping point for you. In no way am I interested in challenging your switch, it’s just really interesting timing (as that was around the time nearly all of Apple’s product lines were growing rapidly even among the general HN audience).

Yeah, it was a weird time to do it. I'd been gravitating towards Linux for a while (including a lot of trying to get it to work on my iBook), and in 2009 I finally had the money for a decent new computer, so I got a Lenovo X200.

Reply on news.ycombinator.com