A running system won't really help me since I'm sure it will give the right answer. I need to know what's happening internally, which remains hidden in a running processor.

My guess is that it made things both easier and harder. Harder in the sense that you couldn't just throw together floating point circuits; you had to deal with lots of special cases. But easier in the sense that the tricky design decisions were already made for you. And easier to test against a known standard.

For what its worth, Intel was one of the major contributors to IEEE 754. In a lot of ways the original 8087 behavior became an early IEEE 754 draft.

It's sort of random. My intent is to focus on the 8086, but someone asked about the 386 so I looked at that a bit. Then I saw the Navajo Pentium rug in a museum so I figured I should look at a real Pentium to compare. And then I wondered if I could find the FDIV bug. So one thing leads to another. I don't think I'll be able to go much beyond the Pentium with an optical microscope, though.

Even modern chips have a single layer of transistors, although there is research into 3D transistor stacking. (Flash memory stacks dies for more density.) Modern CPUs have much, much smaller transistors as well as constructing them more vertically (FinFET) but it is still a single layer. Modern processors can have over a dozen layers of metal, while the Pentium I examined has just 3 metal layers. I'm not going to be reverse-engineering a modern chip since you need an electron microscope to see the features; the features are orders of magnitude smaller than the wavelength of light.