Hacker News Reader

The other post linked from the one you're commenting on (https://blogs.loc.gov/thesignal/2020/11/an-archivists-perspe...) talks specifically about the problems of preserving and reverse-engineering dual-fork files, so the problem is not what they're unaware of, Charlie. You just aren't aware of the problem they're trying to solve.

The LOC, the Internet Archive, and the UK National Archives don't benefit from proprietary compilers for proprietary operating systems like SuperCard; their remit is to preserve these files for centuries or millennia, not until the next update of macOS X. If SuperCard interprets a HyperCard stack differently than original HyperCard did, or SuperCard on a new version of macOS X displays or behaves differently than it did on the previous version, that's a failure of digital preservation. Unless, that is, you notice --- and to notice, you need to emulate the original HyperCard environment and/or be sufficiently familiar with the semantics of the file specification. If https://news.ycombinator.com/item?id=29555959 is to be believed, SuperCard is 32-bit-only and so will not run on the most recent version of macOS X, which lack 32-bit support, which if true demonstrates the enormity of the problem.

As for streams of bytes, a disk is a stream of bytes, or rather a collection of fixed-size blocks of bytes. HFS is looking at flattened HFS files that have been "squished" to fit on a medium that does not, itself, implement HFS. Therefore anything that you can represent with an HFS dual-fork file can also be represented losslessly in a stream of bytes. (You do of course benefit from understanding the dual-fork structure.) HFS+ actually went further, implementing files with an arbitrary number of forks, like sections in ELF or PE. As GeekyBear points out, NTFS did the same, leading to path traversal vulnerabilities in early Microsoft web servers.

Wouldn’t an effective (and seemingly easier) digital preservation approach be to virtualize an older Mac OS (System 7 even) and just run HyperCard on it?

Yes, that's one of the things they're doing, but in the screenshots they're using MacOS 9. It's less effective if you want to do things like full-text search, copy and paste, or render the vector graphics (though maybe those were MacDraw and not HyperCard?) at a different pixel resolution than the emulated Macintosh supported, or with antialiasing. There may also be metadata or "forensic" data that isn't visible directly in HyperCard itself, which may be of interest to future historians.

Aside from the emulation/undocumented-format problem that the librarians here are suffering from, a really big problem with born-digital archival work for more recent formats is copyright. Although libraries are permitted to make copies for archival work, in many cases the software tools they would need to make those copies don't exist, because those tools would circumvent an effective technological protection measure within the meaning of the DMCA, so writing and distributing them is potentially a crime. So in effect copyright law is privileging the business interests of Harvey Weinstein, Dan Brown, Bob Guccione, and Adam Sandler over the academic interests of scholars, historians, and the Library of Congress.

I don't think there's anything wrong with Pee-wee Herman or Amy Poehler making movies, but I don't think it's worth sacrificing the intellectual heritage of humanity to buy them second vacation homes.

Funny you should mention the DMCA. The Library of Congress is uniquely empowered by the DMCA to grant exceptions to it. So if there are any cases where the law is deemed to be a barrier to this effort then the LOC can grant themselves and their colleagues an exemption.

The Librarian of Congress has the opportunity to grant such exemptions in a rulemaking once every three years, which then expire if they are not renewed, which makes them a risky basis on which to base a product; probably the LoC would have to develop the software themselves, which isn't really a strong competency of theirs, instead of relying on outside help.

It isn't the kind of thing where a summer intern can ask permission to break a DRM scheme for archival purposes and then be successfully preserving material the next week.

> render the vector graphics (though maybe those were MacDraw and not HyperCard?) at a different pixel resolution than the emulated Macintosh supported, or with antialiasing

HyperCard graphics were bitmapped, not vectors. There's no additional resolution available.

> There may also be metadata or "forensic" data that isn't visible directly in HyperCard itself, which may be of interest to future historians.

This, on the other hand, is interesting (and potentially a little scary). The HyperCard stack format is practically a tiny little file system, complete with unallocated space; it's not uncommon for FREE chunks to contain bits of data that happened to be in memory on a computer that interacted with the stack.

Thank you!

Yes, and honestly that's going to be the only effective way of preserving some stacks. In particular, some stacks included XCMDs/XCFN resources, which contained native 68k code and could interact with the OS in arbitrary, unrestricted ways.

The original version of Windows NT Server allowed you to install Services for Macintosh and act as an AppleTalk file server, so support for resource forks was added to NTFS.

https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(AD...

Alternate Data Streams are pretty interesting, cause I'm pretty sure Services for Macintosh weren't the only users - it's also way more powerful (closest thing on Unix would be Solaris-style extended attributes) and pretty fundamental part of NTFS on-disk structures (in fact, "alternate" data streams differ from main one only by having a name - the default data stream simply has no name, and all data streams are equal in importance)

Extended attributes aren't limited just to Solaris. A variety of *nix file systems have generally supported forks/multiple data streams for some time. FreeBSD has support for arbitrary-sized extended attributes in UFS/ZFS, for example, and implements its access control with them. Linux probably has the most limited form, where the xattr are capped at 4 or 64 kilobytes depending on the underlying FS.

I think the big issue for uptake is that it just doesn't fit into the UNIX world. Nothing else is aware of xattrs/forks. The basic file utilities will blindly strip them so you can't just cp or tar such a file, for example. If cp transparently breaks all your files, that's pretty much a show-stopper as anyone who remembers the classic Mac to OS X transition will know.

The reason I mentioned Solaris is because Solaris behaviour is closest to NTFS, whereas linux API is based upon Irix, which is closer in behaviour to OS/2 (OS/2 subsystem was another use case for ADS), as well as implementation on a bunch of other systems where extended attributes were treated as just that, attributes.

Comparatively, NTFS is very... flexible navigational database and a bunch of things are just "this is the default parameter set, not exhaustive".

I wasn't aware that FreeBSD copied Solaris in this (ZFS extended attributes by default create a sort-of directory object allocated linked from the file, linux supports inlining simplified xattrs for speed)

There is a difference between alternate data streams and extended attributes. Basically, the former is data, the latter is metadata. NTFS and ZFS implement alternate data streams, but in eg FreeBSD those are accessed like extended attributes.

NTFS doesn't really have a difference internally - they are all attributes of the FILE object (everything in NTFS is a FILE or its attribute, including filesystem metadata). Arguably ZFS ZPL has "blessed" data element (I dare you to implement ZFS while replacing DMU/SPA with Ceph or S3 XD - yes it's doable)

Runtime Revolution is not discontinued, it is alive and well, it just rebranded to https://www.livecode.com many years ago.

The HyperCard stack's resource fork contains icons, sounds, xcmds, xfcns, and probably some other miscellanea. Most of the stack is stored in the data fork.

APFS doesn't specifically have resource forks. They're handled as the com.apple.ResourceFork extended attribute (which is also one way OS X exposes them on HFS+ but with HFS+ the fork/xattr always exists; on APFS it has to be explicitly created).

It seems like it would be reasonably straightforward to create a hypercard clone on top of js and the DOM. Or maybe that's already been done? (a lazy search did not reveal this to be the case).

> it's a distinctly different paradigm from the familiar a-file-is-a-stream-of-bytes convention of Linux/UNIX/NTFS

Each of those platforms has an analog to HFS forks.