I won't tell you what to work on next but I would be very interested in an analysis of the 8086/8088 instruction decoding and microcode structure. TBH I watch and read everything you do with great interest.
> This is only a piece of the 8086's complex instruction handling. Other latches hold pieces of the instruction indicating register usage and the ALU operation, while a separate circuit controls the microcode engine...
I've always been amazed at how complex the 8086 instruction encoding is, especially compared to, say, the 6502 (which admittedly is a much less sophisticated microprocessor). I especially think this whenever I play around with a toy 8086 emulator I've written. :) It's never struck me as being greatly elegant from a software perspective, but I assume there are some good electrical engineering reasons. (And I'm intrigued about the mention of microcode... I always assumed that came later in the x86 series.)
As for the 8086 instruction set, it's kind of a jumble due to the need for backwards compatibility (of a sort) with the 8080. The 8086 instruction set doesn't make things easy for the chip implementation. Intel tried to make a nicer instruction set (with the i432 and then Itanium) but it didn't work out.
The i432 instruction set was, most emphatically, not nicer for implementation. E.g. it had bit-aligned variable-length instructions.
Let me repeat that, because it's totally insane:
bit-aligned variable-length instructions[1]
Intel top management at the time didn't have people who understood what was happening, and consequently let the i432 architects "play in the sand".
I always enjoyed that witty pun: "play in the sand" is what young children do in sandboxes; "sand" is mostly silicon dioxide and is often slang for what silicon chips are made of.
[1] https://en.wikipedia.org/wiki/IAPX432#The_project's_failures
I encourage people to read about the i432 processor, to get an idea of how wildly different a processor design can be. Among other things, objects are built into the processor. Object pointers are created by the processor, so you can't make an out-of-bounds pointer. (In other words, many current security issues don't exist.) The processor includes garbage collection, in hardware.
... and it's probably one of the least weird ultra-ciscy instructions that architecture has.
All the chips have a lot of internal structure to their opcodes, since certain bit fields specify the ALU operation, register, etc. Overall, I'd say the 6502 is the most structured, both because its instruction set is simple and because they didn't have backwards-compatibility to worry about. The 8086 has a lot of special-case instructions wedged into random spots, as well as a lot of inconsistency. For instance, sometimes bits 5-7 specify the register, sometimes bits 2-4, sometimes bits 3-4. And then there are the multi-byte "group" instructions tossed into the mix.
* one byte with implicit operands, for example LOOP or RET or ADD AX,imm
* one byte shortcut encoding with registers in bits 0-2 of the only opcode byte, these are 16-bit instruction only: INC/DEC, PUSH/POP, XCHG AX,rr (these instructions are also available as 2-byte encodings)
* two byte with one register and one register/memory operand, both encoded in the second byte (bits 3-5 and 0-2/6-7, with bit 0 of the first byte determining the size and bit 1 of the first byte determining if the source is the register or the register/memory operand)
* two byte with a single register/memory operand, in which case bits 3-5 of the second byte are part of the opcode
Prefixes and immediate operands complicate things a bit but that's it. It's refreshingly simple compared to the 386 and later.
An interesting tidbit is that registers are ordered AX/CX/DX/BX in the instruction set encoding because their function roughly matches AF/BC/DE/HL in the 8080 (BX comes last because like HL it can be used to address memory, even though the encoding of memory operands is completely different on the 8086)