How MOS 6502 illegal opcodes work | Hacker News Reader

How MOS 6502 illegal opcodes work

pagetable.com

156 pointshasheddan40 comments

If you want to play around with those opcodes on a netlist simulation of the 6502, may I recommend:

https://floooh.github.io/visual6502remix/

...which is essentially a WASM version of the famous visual6502.org with a couple more features (like an integrated assembler):

http://www.visual6502.org/JSSim/index.html

(check out Help => About for credits)

Unfortunately the assembler I used (ASMX) doesn't seem to support the illegal opcodes, so you need to enter the opcodes as hex values directly into the memory tab (the disassembler window recognizes the opcodes though).

The stable ones are thoroughly used in C-64 software these days. A more detailed matrix with extra information can be found here: http://www.oxyron.de/html/opcodes02.html

There’s some comments about those funny opcodes from a 2021 post on hacker news, that includes links to this site and other interesting sources

https://news.ycombinator.com/item?id=27402655

What is wrong with me that I see an article about hacking a microprocessor that was released nearly a decade before I was born and I go, “Ooooh, gotta check that out!”

Nothing. Old tech is fun for many reasons:

1) It's still simple enough that you can actually get a full diagram of the processor and actually have hope of understanding it.

2) It's interesting enough to actually produce good things. Blockbusters like Super Mario Bros 3 were based on this tech. The Terminator runs on the 6502. The low-cost CPU was comparatively as ubiquitous as the Intel architecture is today.

3) Limitations breed creativity and ingenuity. When you only have uint8 as your only data type, the kind of tricks you have to do to get a simple physics engine working are very interesting.

https://www.youtube.com/watch?v=9UP7HImbAlA&t=517s

So combined with not-too-complicated but complicated-enough-to-be-useful is basically why old tech is fun.

Most importantly, Futurama's Bender runs on a 6502.

As does the Cyberdyne Systems Model 101 , aka The Terminator!

Bender's head runs on a MOS 6502. His ass appears to run on an AMD Athlon II.

#1 is the main reason I hack on old stuff, #3 is also present.

The 6502 in particular is a nice choice since they're still made and available (Mouser carries them), there's a zillion vintage things that use them if you want something actually old, there's hobbyist kits/preassembled if you want something new, and it's a super easy CPU to interface to, especially for trivial cases.

> The low-cost CPU was comparatively as ubiquitous as the Intel architecture is today.

And you can still get them today... and they're still in fairly wide use. https://westerndesigncenter.com

> The legendary 6502/65816 microprocessors with both 8-bit and 8/16-bit ISA's keep cranking out the unit volumes in ASIC and standard microcontroller forms supplied by WDC and WDC's licensees. Annual volumes in the hundreds (100's) of millions of units keep adding in a significant way to the estimated shipped volumes of five (5) to ten (10) billion units. With 200MHz+ 8-bit W65C02S and 100MHz+ 8/16-bit W65C816S processors coming on line in ASIC and FPGA forms, we see these annual volumes continuing for a long, long time.

> The 6502 is likely the only processor family that has remained loyal to its ISA over the last 45 years. In addition it has served the widest spectrum of electronic markets through those years. For example, it has served and in some cases created markets for the PC, video game, toy, communication, industrial control, automotive, life support embedded in the human body medical devices, outside the body medical systems, engineering education systems, hobby systems, and you name it electronic market segments. I might add the 6502 has served in a highly reliable and successful way!

> As added food for thought, the 6502/65816 microprocessors protect millions of lives annually within embedded heart defibrillation and pacing systems. We are quite proud of what our customers and partners have created and continue to create with the 6502 Embedded Intelligence Technology for the benefit of mankind!

Yep!!

Another fun fact: the most common CPU in use today is the Z80 (or at least it was a couple of years ago, I haven't checked since). 6502s are not rare. In both cases, they may go by different part numbers these days, of course.

Where I work, I'm currently working on a system that uses 6 Z80s.

"Are those Z-80s cache-coherent?" is not a thought I ever expected to have.

lol! There isn't a whole lot of shared data that brings up the issue of cache coherency, but there is some (mostly around the communications with other equipment) and yes, they are.

That's fascinating, are you able to share anything about that system, or what those 6 Z80s are doing within it?

I need to be a bit vague, but they're being used in an industrial control application to control machinery. Each CPU is in charge of a different step in the process. They collectively operate as a single system that also feeds data into a deep learning system used to direct the operations a little further down the line.

Z80s are used here because they're tiny, low-power, cheap, readily available from multiple manufacturers (a particular bonus since the chip shortage remains a problem), and are extremely reliable. A more modern CPU would be more expensive and harder to guarantee behavior in.

Thank you for sharing!

So in theory, there are still plenty of people writing 6502 assembly for a living? How crazy would it be to get paid for writing code for these things that I first messed around with when I was 10 years old via my C64.

People still write games for the C64. Take a look at Reddit.

There’s even a Turbo Pascal-like IDE that can target them. Link: https://retrogamecoders.com/introduction-to-trse-programming...

> ... uint8 as your only data type

Ahem! uint8 or int8 by the programmers discretions that is.

Not a thing. This CPU was from back in the era when this stuff was still fun.

I was just thinking the same thing lol

The question is why were the $AD and $AE instructions encoded in the PLA with don't-care bits (causing both of them to fire for an xxxxxx11 pattern such as $AF, instead of none)?

It could be related to the fact that if an instruction was not handled at all the CPU would lock up (search https://www.righto.com/2016/02/reverse-engineering-arm1-inst... for "kill"), so rather than add extra logic for illegal instructions the designers just decided to add undocumented ones.

The only problem with this theory is that there are in fact several opcodes which will make a 6502 lock up...

They could have wired those instructions as a NOP, rather than aliasing another opcode...

They could, but transistors were expensive at the time. Why spend valuable space making sure all instructions are well defined?

On modern CPUs designed for multi-processing and protected memory, you don’t want some instructions to accidentally cross privilege boundaries (can’t have an ‘illegal’ opcode accidentally be non-privileged and modify some privileged processor state), so you have to do some of that. Transistors also are cheap, so you can afford to.

The 6502 decodes instructions by matching the combination of opcode byte + cycle counter against bitmasks stored in a “ROM”. The ROM outputs signals to the control logic telling it to do things like accessing a register, performing an ALU operation, accessing memory, advancing to the next instruction, etc.

This design meant the designers could efficiently reuse functionality across multiple instructions by assigning them similar opcodes, and creating a decode ROM entry that matches all of those opcodes. This becomes clearer if you look at an opcode matrix: https://www.nesdev.org/wiki/CPU_unofficial_opcodes

For example, from a brief glance at the chart I noticed patterns like:

- All of the instructions matching the bit pattern `xxx111xx` read an operand from memory using the ‘absolute, y’ addressing mode - All instructions matching the bit pattern `xxxyyy01` perform an ALU operation, where xxx specifies an operation and yyy specifies an addressing mode (with the exception of xxx=100, which performs a store rather than an ALU op) - Branch instructions all match the pattern `xxx10000`, where xxx specifies the predicate - … and many more similar patterns.

The decode ROM looks at the opcode and says “if these bits are 1s and those bits are 0s, do this on that cycle”. So if you feed it an opcode that is not valid, then it activates arbitrary bits of logic based on whatever patterns in the decode ROM your opcode happened to trigger. The result is that the processor behaves strangely, combining bits and pieces of logic meant for other, sometimes unrelated instructions. Some opcodes cause the CPU to lock up entirely, because they don’t match any patterns in the decode ROM that trigger the “go to the next instruction” logic.

Understanding how instruction decoding works will hopefully help reframe questions about undocumented opcodes:

- “Why did they add undocumented opcodes?” They didn’t add them, they exist by happenstance — feed the instruction decoder garbage input, you get garbage output.

- “Why didn’t they just make them all NOPs/crashes/aliases of existing instructions?” Because that would have required an enormous amount of extra logic on the decode ROM — now your decoder needs to support 256 opcodes rather than ~160, and it can’t do nearly as many partial-matching shortcuts. On a processor that only has ~3,000 transistors, you’re not going to add hundreds more to decode opcodes that programs will never (intentionally) use.

- “Why didn’t they just document them?” Because 1) they didn’t add them on purpose in the first place, so they may not have thoroughly understood their behavior & it may have changed throughout development/hardware revisions 2) most of them don’t do anything useful, 3) many of them behave in very strange and unpredictable ways, and 4) they wanted to leave the opcode space open so that future backwards-compatible processors could use them to do actually useful things.

With the don't-care allowable, the "load" nets can be tied directly to the instruction decoder (i.e. LDA = BIT0, LDX = BIT1) instead of needing intervening logic (i.e. LDA = BIT0 & !BIT1, LDX = BIT1 & !BIT0). If you can make the opcode illegal, you can save two gates (which matter for cost, yield, power and timing).

This is such a fun website. This guy also wrote something about recreating Apple I Basic: https://www.pagetable.com/?p=35

Is an illegal opcode something that was intentionally added to the instruction set but was disabled by the manufacturer?

Or is it a side effect of calling an undefined operation?

It can be both. Anything not officially defined in the spec is an illegal opcode.

Intel had a couple of opcodes that were clearly supposed to have been functional, but didn’t make any sense to use—I believe one such opcode popped the code segment register, which would have effectively served as a “jump to random memory” instruction as it would run the next instruction per the IP register but in a totally different part of memory, so it didn’t make any sense to document it as there was no use for it. And they had at least one other instruction introduced as a copyright trap, which they obviously wouldn’t document. And there were a few more that were undocumented but were aliases of other instructions due to the way the 8086 handled bit masking.

Yup, that's POP CS, 0x0f.

Since it made no sense to use, it became the way 286 and later processors used to indicate multibyte opcodes.

When a 286 is running the code, encountering a 0x0f means that the following opcode is multibyte, while on an 8088, all opcodes are single bytes.

> while on an 8088, all opcodes are single bytes.

According to a recent article on undocumented 8086/8088 opcodes (https://www.righto.com/2023/07/undocumented-8086-instruction...), there are some two-byte opcodes: "For most of the 8086 instructions, the first byte specifies the instruction. However, the 8086 has a few instructions where the second byte specifies the instruction: the reg field of the ModR/M byte provides an opcode extension that selects the instruction."

The latter. The instructions aren't disabled in the MOS 6502, but their function is unplanned and hence undocumented which is a better term.

You can find here the most up to date exploration in how these opcodes work: https://csdb.dk/release/?id=226987

I recall some been used in copy protection schemes so that a monitor or disassembler wouldn’t process them.

So, I created my own disassembler that did as many were documented.

> illegal

Seriously, stop using that word for things that aren't actually illegal.

It's been a technical term since forever. I don't really see anything wrong with it, outside of it maybe confusing laypeople.

Undocumented is a better term for the MOS 6502 in my opinion, because these opcodes aren't invalid, they can't be trapped and they don't throw an exception.

The 6502 was reimplemented in fresh silicon several times by different manufacturers, its more "undefined" than "undocumented". Some clones use them as actual new opcodes specific to that manufacturer. Some might do what the MOS chip did. Some might throw the processor into an unrecoverable state.

This is why I'm emphasizing MOS 6502. I'm well-familiar with the most popular alternative, WDC's 65C02, where all of the undocumented opcodes execute as NOP with no changes to internal registers/flags or to memory, but with one or two harmless side effects on timing behavior.

Reply on news.ycombinator.com