Comment by reaperducer | Hacker News Reader

Comment by reaperducer | Hacker News Reader

Parent Full thread

It's interesting how old analog audio technology still finds itself useful in these days of terabyte drives and satellite internet.

Some retro computer enthusiasts have taken to storing their programs as MP3 files. That way they can be loaded into a cassette interface using a hand-held voice recorder. Instead of carrying around a suitcase of tapes, you can store everything in the palm of your hand. It's also supposed to help protect against bitrot, but I don't know if that's true. Still, bitrot is a serious concern in retro computing circles because one bad bit in a 10 GB Windows game is probably harmless. But one bad bit in a 10K Apple ][ game will likely ruin it.

Also, I remember when I got my Amazon on-demand ordering buttons, they were set up using an analog modem sort of thing. It's been a long while, but from what I remember, when the buttons arrived, I'd open the Amazon app and tell it I was programming the button, then hold down the button on the button thing and the button and the phone would squawk to each other. It sounded very much like a 300 baud connection, but without the underlying carrier tone — just data.

There's a very good chance that a single bad bit in a 10 GB Windows game will prevent it from loading.

The average game might contain a multi-megabyte executable along with gigabytes of assets. Those assets will be stored in some kind of compressed format (for example Quake III used 'pk3' files, which were actually zips). These compressed files with inevitably have checksums, and an invalid checksum will prevent them from loading.

You’re assuming the game engine will panic on a bit error loading what’s probably a texture or model mesh. This probably isn’t the case for most games, especially release builds. You might get a console message.

A 1-bit error in a compressed file will cause every subsequent bit in the entire file to end up wrong (well, 50% of them), and probably the file length to end up wrong, and probably the decompression state machine to get stuck and confused. This is why basically all compressed formats carry a CRC (eg. zip, gz, xz).

If you have something like a Deflate stream, there is a chance that the flipped bit will affect a symbol but change it into a symbol with the same length. Could do anything from changing a single byte in the output, to changing bytes in a bounded range, to corrupting the entire rest of the stream, or making it fail to decompress (not every deflate stream is valid, far from it).

Most games don’t store their assets in a single one-shot-load pack file anymore. They use chunked asset files where individual assets or packs will be compressed and then concatenated with a master index, meaning you don’t have to load the whole file when the engine needs only a subset. This means the checksums necessarily cannot cover the entire data file, because then you would need to read the entire file to verify a single asset anyway, throwing away the benefits of random access. You can also turn the checksums off for “speed,” and some do.

Yup, that's why most physical medium have some form of ECC or FEC[1].

[1] https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_cor...

The Square card readers that plugged into the headphone jacks were also analog modems. Well, the swipe variants were just "mo" (modulators) and the phones were the "dems" (demodulators). The first chip card reader, that plugged into the headphone jack, built a full bi-directional soft modem onto either end.

In theory you should be able to update the firmware on one of the chip card readers with one of these vinyl records and a 1/8" adapter.

I wonder what the largest checksummed file you could manually (well, programmatically) test every single bitflip against, rechecksumming to see if you found the bad bit, in a reasonable amount of time is.

1 kilobit of data seems pretty doable, only a thousand variants to check. Somewhere around 1 gigabit seems like it should be too much for consumer hardware -- if it takes 1 second to re-sum the data, that's ~30 years to test every bit flip.

This is the sad NP-hard part of cryptographically-strong checksumming - the heavier the algorithm, the longer it'll take, and the harder it'll be to exhaust it on even trivial inputs.

Sometimes simply loading both copies of the data into RAM and comparing them can be faster - for scenarios where you have both copies available.

XXHash comes to mind as the (apparently) fastest non-cryptographic general-purpose hash out there at the moment; it runs close to memory speed. It would be interesting to see a microbenchmark-style graph showing the slowdown checking all permutations of 1KB, 1MB, etc.

MP3 files? What do they do to maintain integrity with lossy compression?

WAV or FLAC i could see, but MP3?

If you're going to play it over a speaker, you already have way more signal loss than the compression is going to introduce. The solution is redundancy.

It's a fair question. Tape interfaces had very limited bandwidth. The original Kansas standard used frequency switching (FSK) between four cycles of 1200Hz and eight cycles of 2400Hz. Later faster variations used the same frequencies but cut the number of cycles for faster load speeds.

MP3 compression works by removing frequencies that are (supposedly) too quiet to be audible. If you only have two very loud frequencies in a frame they pass through the compression process unscathed.

The switching hash around them and the tape noise may get munged but that doesn't affect the data stream. So vbr MP3 ends up being an efficient and clean representation, and cbr works too but isn't quite so compact.

Edit: in fact some radio and TV shows in the 70s would transmit software live, and that worked fine too. I never tried it, but I suspect you could probably use a phone line - for early adventures in software piracy.

The fidelity of standard non high bias tape at compact cassette speeds is already shit compared to even low bitrate mp3. It’s not a problem in practice.

Lossy codecs use psycho-acoustic models designed to encode music so it sounds okay to humans, by discarding frequencies that we're less likely to hear due to masking etc. -- Just need to avoid those frequencies.

Having said that the newer lossy codecs (e.g Opus and some AAC variants) which are more efficient (sound better at lower bitrates) and have less "problem samples" than MP3 would perhaps also be better for data use, like they are for music.

Well as long as the mp3 loads just fine for the first time I dont see how bitrot could happen.

Reply on news.ycombinator.com