Hacker News Reader

Comment by kevingadd | Hacker News Reader

You can't just start it up and run it for 5 seconds, though. A given game might be a hundred hours long and only exercise certain functionality at minutes 5, 37 and 60. It's non-trivial to automate the playback of the whole package and then verify the whole stream against a known good. So you'd need to identify the key points in each game that are important, and replay those with save states and hope that the save states aren't unintentionally encoding corrupted data that came from an existing bug in the emulator you used to run them.

Also, none of that automation you just did will work w/the actual hardware.

View on HN

> A given game might be a hundred hours long and only exercise certain functionality at minutes 5, 37 and 60.

An infamous bug of that variety exists in the SNES game "Speedy Gonzales - Los Gatos Bandidos", where one specific element in one level would freeze the game on most emulators due to a really subtle hardware emulation issue.

https://mgba.io/2017/07/31/holy-grail-bugs-2/#speedy-gonzale...

> Also, none of that automation you just did will work w/the actual hardware.

You might be surprised. Consider what the TASBot team has done with the NES, SNES, and various other consoles -- http://tasvideos.org/TASBot.html

In no way will TASBot ever be automated. It can require hours (and sometimes dozens of hours) to get a single game to sync on real hardware, and that’s if a cycle-accurate emulator was used. If not, it’s near impossible to playback a TAS.

Not to mention only a very small subset of SNES TASes are done on cycle-accurate emulators.

The input stream for an SNES isn't very complicated - just the state of a few buttons at every cycle, and the transition edges are quite sparse relative to the clock frequency. You could easily record human play sessions and replay them.

There's already a community of people that does this called tool assisted speed runs (or TAS). These players already record their inputs for replay and are comfortable tuning them to consistently hit edge cases.

I wonder how much of a lift it would be to take a bunch of TASes and turn them into regression tests...

The SNES output is non-deterministic across runs.

Not only is there unitialized RAM and I/O registers, and some analog effects, the really big elephant in the room is that the system has two oscillators. A ~21MHz CPU/PPU crystal oscillator, and a ~24MHz SMP/DSP ceramic oscillator.

Given that not only do these exact frequencies change between systems due to margins of error on clocks, they also change slightly as the system runs (and gets warmer, for example.)

Every SNES game has sound routines that synchronize the CPU to the SMP.

It wouldn't be possible to make a literal 1:1 play log unless you a) ran a custom register and memory initialization at system startup, and b) replaced the two oscillators with a much faster single oscillator and then used a clock dividier to drive both the CPU and APU off of it.

(You can TAS certain SNES games anyway, of course. It really depends on how the game is programmed to react when the exact CPU<>SMP communications change. If it seeds a random number generator based around the PPU H/V counters that are polled after a CPU<>SMP sync for example, forget about it.)

> Given that not only do these exact frequencies change between systems due to margins of error on clocks, they also change slightly as the system runs (and gets warmer, for example.)

Maybe it'd be possible to modify some boards to use a CPLD (or an FPGA) for synthesizing those clocks from a common source. This could eliminate most (all?) uncertainty.

It would certainly help a lot. People have talked about doing this, but I don't think there's been enough interest to actually make it happen.

The input steam is half the story, you should also match the whole observable state of the machine for an accurate emulation, because some other test might actually depend on it. Triggered on all the clocks and decide which clock matters every case by hand.

To get that, you would need a set of hardware debuggers plugged into the bus and chips. And a lot of inside knowledge to decide if a deviance is random enough to not have to be emulated.

Reply on news.ycombinator.com