Cartridge Printed Circuit Boards | Hacker News Reader

Cartridge Printed Circuit Boards

byuu.net

187 pointssgift42 comments

Hi, sorry for the rapid-fire on these articles ^-^;;

I'm trying to build out the site to debut it on an upcoming AMA, as right now it's quite new. By October, new articles will proceed at a more reasonable pace.

I do greatly appreciate the attention here, however! I was afraid with it being a new domain with no links, the content would end up not being found by anyone. HN has helped me tremendously, and for that I am very grateful. It's been a real privilege to have so many people reading and caring about this little niche of mine.

Thank you all very much!

This is the first time I've seen someone apologize for producing too much good content too fast. It's okay, don't worry! :)

On the contrary, thanks for the rapid-fire on these articles :)

Thank you for everything.

What did you do with your old website ? I remember at had quite a few articles as well

A combination of two things: one, I redesigned my site from PHP and raw HTML articles to C++ and Markdown articles, and didn't want to update all of them to the new format. Two, I feel I was a bit too confrontational and opinionated with my viewpoints in the past. I've grown and mellowed out a lot in the 20-years I've had an online presence, and I don't really stand by some of the older pieces I've written anymore. It's always been a struggle to counter-balance the passion/intensity of coding with that of my writing; they tend to be a package deal ^^;

I'm open to suggestions to revise and bring back any specific articles, if there's a technical piece you miss.

You may find it too confrontational, but I always really liked your old article where you highlighted problems in existing SNES emulators. It was quite interesting!

Have you considered re-opening your forums? It was a nice little corner of the Internet.

My understanding is that you did so to purge social media interaction? but you're back here and on reddit, so I suppose you haven't sworn it off completely.

Thanks for all your hard work!

byuu didn't want the hassle of community management on top of emulator development, which is understandable. Most of the old forum regulars now hang out at https://helmet.kafuka.org/bboard/ if you want to talk about emulator development, or thread derailing.

If you prefer more technical content to social, the NESDev forums have a "SNESDev" sub-forum, too: https://forums.nesdev.com/viewforum.php?f=12

I've not been enjoying trying to manage topics via GitHub issues, so I'm thinking about reopening a development-only forum without any of the general discussions, but things are still a bit in flux as I build up the new site. It would be fun to talk to you guys again, of course.

Can't find a better way to contact you: little typo, grammatical issue really. Article says:

> In Pilotwings' case, a fix to one algorithm resulted in the attract sequence showing a plane landing to fail, and the plane to infamously crash.

That's easy to misread. I'd suggest:

> In Pilotwings' case, one of these algorithms failed, so that a part of the attract sequence a plane infamously crashed rather than landing as intended.

Or similar. As originally written it's easy to misunderstand what is failing.

I, for one, really like the idea of a single .zip file per game. Then you can include the unmodified ROM, a separate file with configuration data, any necessary firmware, box art, and any other information an emulator might want. And it saves a little space to!

Also, if two different emulators want two different configuration formats, you could just include both! (As long as the file names aren't the same.)

I very much agree. My vision was a ZIP file that when selected in an emulator, could be extracted for instant access, and any save RAM, cheat codes, save states, etc could be stored alongside the game in said folder. It would be bundled with an extensible markup description of the board, and optional pack-in content like box art and user manuals. For games stored on flash memory that could write to themselves (BS Memory, Neo Geo Pocket), this would also prevent the archived, pristene copy from being damaged. The folder would then act like a physical cartridge, and all your game saves would move along with it.

I wrote about my idea here: https://higan.byuu.org/game-paks

It ... was not a popular idea, to put it mildly. I further shot myself in the foot by trying to force the idea on people. You will never win people over with an approach like that.

Let's avoid mentioning names for obvious reasons, but unfortunately those most capable of adopting and promoting such a format through their database effort are unwilling to use ZIP archives in this way. They will only allow for an uncompressed single-file-per-game approach.

As mentioned in my article, this turns into quite the problem. When you take the 8KB program ROM for Galaxian, and append a character ROM for graphics onto it, but your header can only tell you the size of the program ROM in 16KB increments, it means there's no way of specifying where the program ROM ends, and the character ROM begins. Any emulator that supports Galaxian does it via game-specific workarounds like checksum identification.

I remember when Nesticle and some of the other viable Nintendo Entertainment System emulators first started showing up. It took a while but eventually everybody just ended up settling on the iNES format by Marat Fayzullin. This was a very useful thing.

I've always found it extremely frustrating that later systems had so many competing formats for the roms as you inevitably end up with lots of duplicate games just so different emulators can handle them. It gets even worse for optical media with all the .cue file stuff and distributions that provide the audio tracks outside of the iso in half a dozen different formats. It's almost impossible to assemble a good collection for those kinds of systems.

I honestly think a .zip based distribution makes a lot of sense since most emulators of cartridge based systems can open zipped archives and run the first compatible file they find in them. But all the rest of the information for the game could be "built in" to the zip.

If you look at the Gamebase projects, there's plenty of hobbyists who are willing to pour thousands of hours into assembling games, metadata, manual scans and other such things into those kinds of projects...but they lack the ability to unify all that ephemera into a single archive per game.

If such a thing were to become standard, I'm certain people/groups outside of the "established" database groups would begin to publish collections.

I for one would definitely welcome a format with built-in cheats, patches and so on and would love to download a collection of say, Japanese RPGs that come with scanned manuals, translated manuals and can run the game in the original and patched versions....or to try out all the weird and wacky Mario/Sonic patched game variants. I just can't be bothered to go download each patch myself to try it out, but if they were already present in the .zip :) I think even support soundtracks would be awesome and could see people firing up collections of games just to listen to the music, without having to keep track of a separate soundtrack archive.

I think this would be an even bigger benefit for modern optical media. The .bin/.cue stuff is honestly terrible. More than half the time I've ended up with a .bin or some kind of .something that I need to convert for some reason, or mount and extract stuff and then handbuild .cue files which don't work half the time. I don't even know if the problem is what I've downloaded, the .cue I made or the emulation compatibility. A standard distribution "format" that just solved all that would be incredibly useful.

At some point, I want to talk about CDs as well.

A proper CD archival format would include the lead-in sectors, with which it is absolutely trivial to extract track information from the Q-subchannel of it. Easier than parsing a CUE sheet, or MDF, or CCF, or whatever else have you.

I do understand it's hard to rip this data from CDs (usually requires an older Plextor drive), but worst case, as bad as it'd be, generating the TOC from a CUE sheet is very simple, and if it's marked with a special entry to denote it being machine-generated, I feel that would be a good compromise.

Then you'd have a single CD image format in a single file that captures all data on CDs. If you wanted to, you could then compress that image with either ZIP, or a custom-purpose tool that combines things like Reed-Solomon prediction, subchannel prediction, FLAC encoding for audio tracks, etc, resulting in images not any larger than ISO+FLAC.

But well, https://xkcd.com/927 and all, right?

> I for one would definitely welcome a format with built-in cheats

A thing I do with bsnes is bundle a database with most known cheat codes for all games, so you've got all the cheats nicely labeled and ready to go whenever you play a game.

I kinda wish this idea caught on more for all the other systems and emulators, but it hasn't been all that popular.

I kind of think that part of the problem is that there needs to be a bit of a network effect here for these kinds of .zip packaged formats to take off:

1) An emulator of sufficient popularity has to fully support it.

2) The format has to provide some kind of benefit to the user such that they're willing to convert/duplicate their entire collection.

3) Somebody has to make and distribute collections of packaged roms that include enough extra stuff to make them worthwhile.

To be honest, you might be in a good position w/r to #1 since your SNES emulator is now more or less the default in the community (at least for big multi emulation systems like retroarch).

For #2, I think the benefit really derives from what's in the package: manuals, savestates, cheats, soundtracks, playthroughs (that you can jump to in the middle of), patches, variants, meta-data (publication info, credits, etc.) and so on could all be really interesting to lots of people, but really would rely on the emulator to take advantage of in some way.

For #3, There's probably already enough information freely available to make a utility that would take somebody's existing ROM collection and yank down all the rest of the stuff from various websites all at once to build up an initial package of stuff.

I kind of almost just think you should just go ahead and support something like this and have it slowly percolate through the community. If it achieve some kind of network effect it'll slowly replace the other formats in a de-facto way.

how about MAME, which has a zip-based format containing raw ROM dumps, and the "CHD" format for disk/disc based media? obviously all of the metadata is still contained within MAME itself.

MAME is a kind of special case, where the same group of people write the emulator and catalogue the ROM dumps. For most emulated platforms, there's more than one emulator, and often more than one popular emulator, so deciding on a new archival format is a major political endeavour, on top of the existing tricky technical problems.

Also, even MAME is not immune to political ROM-format problems. The format MAME used was designed to help people maintain and update arcade boards, which often had the ROM chips socketed so they could be replaced by the end user and so "a set of ROMs for a game" was a reasonable thing to talk about. MAME uses that format for every platform they support, but most home consoles were not designed to be end-user modifiable, and sometimes the same game shipped on different sets of different sizes of ROM chips (say, a 4Mbit chip, or two 2Mbit chips) depending on what was cheapest at the time.

MAME as a collective project doesn't have the will to use different formats for different platforms, and home-console emulators aren't interested in following MAME's arcade-based conventions.

MAME follows the internal database approach to emulation. The primary issue with this approach is that it does not support homebrew, fan translations, and ROM hacks. If you modify games, checksum lookups fail. For a ROM hack, you could still boot something where the original was in the database by manually specifying the name and accepting the warning (which is fine), but for a new unlicensed / homebrew title, you could only maybe possibly boot it by faking being another very similar game, which feels quite wrong.

I'm not saying the MAME approach is wrong: every approach has its pros and cons, regrettably.

Now, I'm _really_ not an emulator expert, but as I recall both snes9x and zsnes will read rom dumps from zip files. So it seems to be possible to do this without too much cooperation.

If archivers started including useful metadata in addition to the rom dumps, I'm sure the emulators would start to use it as well.

Or maybe the archivers are the problem?

The latter. I support this, but it does no good when no one uses the format. The most prominent databasing effort will not include any additional files. Not even metadata, they're missing the coprocessor firmware required to play games. I wrote about this a few years back here ( https://higan.byuu.org/firmware ), but fair warning it's a bit rantier than my recent articles ^^;

I've just taken a shotgun approach recently: supporting games split into multiple pieces inside folders, games inside ZIP archives, games in merged files to a single ROM, games with external firmware in a separate subfolder, games played by an internal database, games that have an external manifest to describe the PCBs, games played by heuristic detection, and games that fall back on less accurate high-level emulation instead of not booting at all.

I must be missing understanding on some part of the picture of why many were so against it. Unless I'm missing something everything but "decompress the file and load the ROM in it" is completely optional to implement and would be no different than how things work with raw ROMs today, what was the push back on?

There were a few points of contention:

- for consistency and simplicity, byuu wanted a standard naming scheme for the files inside the archive. The main game data should be "program.rom", the non-volatile storage should be "save.ram", etc. Unfortunately, existing emulators all work by grabbing the first (or last?) file they find whose name ends in .sfc, so that was an incompatibility.

- Many people don't pick their own filenames, they use a database like No-Intro or GoodSNES to rename and compress files automatically. Such tools would mistakenly rip apart structured game folders every time they were run.

- Many emulators have config settings for "where to store save games" and "where to look for patch files". If a game folder contains a patch file, should it be loaded instead of a patch in the "patch files" directory? If a game folder is loaded, should save games be written to the folder instead of the "save games" directory?

It's not just "the ROM in it", it's "the ROMs in it". Single-rom games don't benefit from containers. The whole point of treating all games as containers is to allow multi-rom games to be free of metadata. 30% of NES games, 1% of SNES games, and most arcade games are multi-rom. In order for the emulator to tell contents apart without metadata, you have to have a naming scheme. So, for example, Galaxian.fc.zip would contain "program.rom" and "character.rom". Currently, those pieces are blobbed together with a header in front. Thus you can't open a .NES file in your file browser and see if it has a character rom, what size it is, what sha256 it has. This causes massive tedium with NES dump verification since you can't assume anything at face value. All parts and values are obscured and need extracted.

The other reason metadata is evil is that it's a royal PITA to maintain. It's far easier to update your emulator than thousands of individual files. Do not listen to anyone who tells you that we need to drag licensed games into the metadata mud for the sake of unlicensed games. They are pulling your chain, an unlicensed db for nes is entirely feasible, or even the mapper number applied to the extension ala "Cheetahmen II.224.fc.zip".

MAME did this right with a database and NEVER writing inside the container, but only because their hand was forced into it. So many arcade games have 3 or more roms, so they were never tempted to take shortcuts and blob shit together to avoid zip containers. Then they screwed up their own good fortune with some truly bizarre stuff. Google non-merged vs merged and behold threads and threads of confusion. Now weigh that confusion against the supposed benefit.

The Atari 2600 is an example of a system where games didn't have headers and a checksum/database or other heuristic has to be used to account for the occasional hardware extension.

Z26 implements a heuristic to determine PAL or NTSC mode, which is done by monitoring how long the software takes to begin outputting lines (PAL's vblank is longer than NTSC's).

But this article makes me think of the problem of determining the boundary between emulation and simulation. The reason why the original Atari Pong of 1972 is not in MAME is because Pong doesn't have a CPU, it's all implemented as discrete hardware (and has no ROM). But you can't play any games on any emulator if some hardware isn't simulated, like the video.

There is some project that will take a description of the Pong schematic and render a playable game, but it's very slow. I can't recall the name of it. I also find it interesting that the NES's video chip has been reverse engineered to the point of accounting for every change in waveform it outputs to composite video--even the bad ones; meaning the flickerly color fringes and unsmooth vertical lines of real NES video are now documented and implementible by anyone who cares.

Extremely well written. The Mega_Man_X#Copy_Protection[0] is very interesting. I admire (total understatement) the R&D and sheer passion that goes in to preservation / emulation.

[0] https://tcrf.net/Mega_Man_X#Copy_Protection

I just finished my RSS queue with that article being the last, opened HN and saw it here!

I really enjoyed it, just a minor nitpick: I’d love if you provided the full article text in your RSS feed

Which RSS client do you use, if you don’t mind? I’m on my personal quest to find a good RSS client, without success, for now.

Reeder, both on macOS and iOS. NetNewsWire is pretty good too, but doesn’t support syncing to inoreader

The end of this article is gold

'So unfortunately, I can't end this article with a solution. It's an open problem in emulation.'

Indeed, every approach has significant trade-offs.

I've even had it non-jokingly suggested to provide netlists of the game cartridges. But well, that's great if you don't mind your SNES emulator never reaching 60fps. And good luck asking most emulator developers to simulate full-on netlists and logic chips directly, instead of a single-byte in a header that lets you turn that into a few if((address&mask)==range) checks.

Further, in practice, getting emulator developers to all agree on a solution is like herding sheep. The NES has a long history of very strong opinions on how this should be done, and in my earlier, more boisterous days, I did as well.

These days, I've taken to the more pragmatic yet developer-intensive approach of just supporting as many options as possible and letting the user decide. I do the best I can and when things fall through the cracks due to missing information, it just is what it is.

IIRC the NES had an analog bypass that could fetch signals to the cartridge for processing, meaning your game is actually a full blown hardware extension. Some youtube video talked about that regarding Japanese version of Castlevania (it had a complete FM synth chip embedded)

That is correct. Every cartridge on every system can do this, if you don't care about the cartridge potentially being oversized.

There is no reason you couldn't put a Raspberry Pi Zero inside of an NES cartridge. You would just need to write a communication layer between the NES and the Pi.

In practice, the NES, SNES and MSX had this expansion done the most. The Genesis only really had Virtua Racing. And other systems tended to be pretty much standard, with few exceptions.

This guy did that...

https://kotaku.com/programmer-hacks-cartridge-to-run-snes-ga...

One difference - I think that guy essentially forces his signal on the various data lines instead of having a proper NES "communication layer", which is why it's a bit glitchy

The graphical glitches are the result of timing inaccuracies. This technique is a just higher resolution version of what the MMC5 mapper does to get 8x8 attribute blocks. It's not "forcing" anything.

I was under the impression that he was "just" stuffing the bus, perhaps I need to refresh my knowledge of the project

I planned to do that with a hp48 + an esp32 as grafted coprocessor. (and an eink display and a lithium battery)

Are you replacing the HP48’s display w/e-ink?

I have used quite a few cd-rom drive audio cables to make serial cables for those calculators.

Such a great calculator.

That's just bedroom fantasy at this point. I was just thinking about a 2019 variant of the hp48 classic. Even though a good li-ion battery would probably suffice to run the original LCD for a long time.

I see there is a MAME emulation for the 48G, but I have not been able to find any projects using e-ink for MAME yet.

In the Vectrex dev community, they are putting a Raspberry Pi zero in a cart to also do “reverse emulation.” The cartridge port provides HLT which puts the internal 6809 CPU to idle and off the bus. Meanwhile the Pi can control the hardware to produce sounds and vectors. It's called PiTrex.

http://computernerdkev.heliohost.org/pitrex/wiki/index.php

(alternate web archive) https://web.archive.org/web/20190802005407/http://computerne...

The PC industry came up with various "plug and play" standards to describe the configuration of the hardware and make it easily understandable by software, but as evidenced by the complexity of something like ACPI, it's a hard problem for them too.

Memory mapping is relatively easy to describe; other peripherals on the bus (like a second CPU), not so much.

Reply on news.ycombinator.com