Also, if two different emulators want two different configuration formats, you could just include both! (As long as the file names aren't the same.)
I wrote about my idea here: https://higan.byuu.org/game-paks
It ... was not a popular idea, to put it mildly. I further shot myself in the foot by trying to force the idea on people. You will never win people over with an approach like that.
Let's avoid mentioning names for obvious reasons, but unfortunately those most capable of adopting and promoting such a format through their database effort are unwilling to use ZIP archives in this way. They will only allow for an uncompressed single-file-per-game approach.
As mentioned in my article, this turns into quite the problem. When you take the 8KB program ROM for Galaxian, and append a character ROM for graphics onto it, but your header can only tell you the size of the program ROM in 16KB increments, it means there's no way of specifying where the program ROM ends, and the character ROM begins. Any emulator that supports Galaxian does it via game-specific workarounds like checksum identification.
I've always found it extremely frustrating that later systems had so many competing formats for the roms as you inevitably end up with lots of duplicate games just so different emulators can handle them. It gets even worse for optical media with all the .cue file stuff and distributions that provide the audio tracks outside of the iso in half a dozen different formats. It's almost impossible to assemble a good collection for those kinds of systems.
I honestly think a .zip based distribution makes a lot of sense since most emulators of cartridge based systems can open zipped archives and run the first compatible file they find in them. But all the rest of the information for the game could be "built in" to the zip.
If you look at the Gamebase projects, there's plenty of hobbyists who are willing to pour thousands of hours into assembling games, metadata, manual scans and other such things into those kinds of projects...but they lack the ability to unify all that ephemera into a single archive per game.
If such a thing were to become standard, I'm certain people/groups outside of the "established" database groups would begin to publish collections.
I for one would definitely welcome a format with built-in cheats, patches and so on and would love to download a collection of say, Japanese RPGs that come with scanned manuals, translated manuals and can run the game in the original and patched versions....or to try out all the weird and wacky Mario/Sonic patched game variants. I just can't be bothered to go download each patch myself to try it out, but if they were already present in the .zip :) I think even support soundtracks would be awesome and could see people firing up collections of games just to listen to the music, without having to keep track of a separate soundtrack archive.
I think this would be an even bigger benefit for modern optical media. The .bin/.cue stuff is honestly terrible. More than half the time I've ended up with a .bin or some kind of .something that I need to convert for some reason, or mount and extract stuff and then handbuild .cue files which don't work half the time. I don't even know if the problem is what I've downloaded, the .cue I made or the emulation compatibility. A standard distribution "format" that just solved all that would be incredibly useful.
A proper CD archival format would include the lead-in sectors, with which it is absolutely trivial to extract track information from the Q-subchannel of it. Easier than parsing a CUE sheet, or MDF, or CCF, or whatever else have you.
I do understand it's hard to rip this data from CDs (usually requires an older Plextor drive), but worst case, as bad as it'd be, generating the TOC from a CUE sheet is very simple, and if it's marked with a special entry to denote it being machine-generated, I feel that would be a good compromise.
Then you'd have a single CD image format in a single file that captures all data on CDs. If you wanted to, you could then compress that image with either ZIP, or a custom-purpose tool that combines things like Reed-Solomon prediction, subchannel prediction, FLAC encoding for audio tracks, etc, resulting in images not any larger than ISO+FLAC.
But well, https://xkcd.com/927 and all, right?
> I for one would definitely welcome a format with built-in cheats
A thing I do with bsnes is bundle a database with most known cheat codes for all games, so you've got all the cheats nicely labeled and ready to go whenever you play a game.
I kinda wish this idea caught on more for all the other systems and emulators, but it hasn't been all that popular.
1) An emulator of sufficient popularity has to fully support it.
2) The format has to provide some kind of benefit to the user such that they're willing to convert/duplicate their entire collection.
3) Somebody has to make and distribute collections of packaged roms that include enough extra stuff to make them worthwhile.
To be honest, you might be in a good position w/r to #1 since your SNES emulator is now more or less the default in the community (at least for big multi emulation systems like retroarch).
For #2, I think the benefit really derives from what's in the package: manuals, savestates, cheats, soundtracks, playthroughs (that you can jump to in the middle of), patches, variants, meta-data (publication info, credits, etc.) and so on could all be really interesting to lots of people, but really would rely on the emulator to take advantage of in some way.
For #3, There's probably already enough information freely available to make a utility that would take somebody's existing ROM collection and yank down all the rest of the stuff from various websites all at once to build up an initial package of stuff.
I kind of almost just think you should just go ahead and support something like this and have it slowly percolate through the community. If it achieve some kind of network effect it'll slowly replace the other formats in a de-facto way.
Also, even MAME is not immune to political ROM-format problems. The format MAME used was designed to help people maintain and update arcade boards, which often had the ROM chips socketed so they could be replaced by the end user and so "a set of ROMs for a game" was a reasonable thing to talk about. MAME uses that format for every platform they support, but most home consoles were not designed to be end-user modifiable, and sometimes the same game shipped on different sets of different sizes of ROM chips (say, a 4Mbit chip, or two 2Mbit chips) depending on what was cheapest at the time.
MAME as a collective project doesn't have the will to use different formats for different platforms, and home-console emulators aren't interested in following MAME's arcade-based conventions.
I'm not saying the MAME approach is wrong: every approach has its pros and cons, regrettably.
If archivers started including useful metadata in addition to the rom dumps, I'm sure the emulators would start to use it as well.
Or maybe the archivers are the problem?
I've just taken a shotgun approach recently: supporting games split into multiple pieces inside folders, games inside ZIP archives, games in merged files to a single ROM, games with external firmware in a separate subfolder, games played by an internal database, games that have an external manifest to describe the PCBs, games played by heuristic detection, and games that fall back on less accurate high-level emulation instead of not booting at all.
- for consistency and simplicity, byuu wanted a standard naming scheme for the files inside the archive. The main game data should be "program.rom", the non-volatile storage should be "save.ram", etc. Unfortunately, existing emulators all work by grabbing the first (or last?) file they find whose name ends in .sfc, so that was an incompatibility.
- Many people don't pick their own filenames, they use a database like No-Intro or GoodSNES to rename and compress files automatically. Such tools would mistakenly rip apart structured game folders every time they were run.
- Many emulators have config settings for "where to store save games" and "where to look for patch files". If a game folder contains a patch file, should it be loaded instead of a patch in the "patch files" directory? If a game folder is loaded, should save games be written to the folder instead of the "save games" directory?
The other reason metadata is evil is that it's a royal PITA to maintain. It's far easier to update your emulator than thousands of individual files. Do not listen to anyone who tells you that we need to drag licensed games into the metadata mud for the sake of unlicensed games. They are pulling your chain, an unlicensed db for nes is entirely feasible, or even the mapper number applied to the extension ala "Cheetahmen II.224.fc.zip".
MAME did this right with a database and NEVER writing inside the container, but only because their hand was forced into it. So many arcade games have 3 or more roms, so they were never tempted to take shortcuts and blob shit together to avoid zip containers. Then they screwed up their own good fortune with some truly bizarre stuff. Google non-merged vs merged and behold threads and threads of confusion. Now weigh that confusion against the supposed benefit.