Symbolic Execution: Intuition and Implementation

You'd take out your regular processor, and plug the ICE into the socket. The ICE would have a normal processor that used the extra signals on the special processor to control it, and to provide a user interface that would let the user control all kinds of aspects of the special processor's execution that normally are invisible.

Here's a nice table comparing ICEs with other tools used to debug embedded systems: www.ganssle.com/articles/ice.htm

In the '70s and '80s (and probably beyond) an ICE was an invaluable tool for embedded developers. Actually, it was pretty damned useful to any low level developer, not just embedded developers. I had a job once where I was rewriting the whole memory and process subsystems on a port of a swapping Unix to turn it into a demand paged system, running on a 68020. There were some subtle bugs in the interaction between page fault handling and signals that I had not considered [1] and an ICE made figuring them out a lot nicer.

Without an ICE back in those days, even a simple task as figuring out what is overwriting something in memory that should not be getting overwritten could be complicated. Maybe if you had a logic analyzer with enough channels and enough history you could trigger on seeing writes to that memory on the bus, and go far enough back in history to figure out from the instruction fetch bus cycles what code was doing it. But with an ICE you could simply set a trigger to pause when the processor tries to write the monitored location, and then examine the program counter directly.

You don't have to be anywhere near embedded or OS development to have a "who the hell is overwriting this?!" problem.

Nowadays, processors often include built-in debugging features that in many cases eliminate the need for an ICE. For example, the 386 had 4 debug registers that could be loaded with addresses, and configured to trigger breakpoints if those addresses were accessed. The breaks could be configured to be on execution, break on write, or break on read/write, and could be configured to cover 1, 2, or 4 bytes starting at that address.

That gives you a good chance to reasonably efficiently track down those overwriting bugs without an ICE.

I haven't looked at low level debugging in a while, but I'd expect that processors nowadays go far beyond the 386's debug register capabilities, allowing much more to be done without ICE than we could back in the day. Also, I believe many smaller processors nowadays include interfaces that expose some of the information and control functions that the special ICE processors exposed. They expose this over a serial interface that only takes a handful of pins, and this is included standard in the processors. So no more need to replace the processor with a special one...you just need to design your system so those pins are accessible. I don't think you can do everything an ICE could do with this kind of processor interface (at least not in real time like you could with an ICE) but most of the time it should be good enough.

[1] There are a couple ways a processor can deal with an instruction that gets interrupted in the middle. One way, which is what I believe Intel uses, is "instruction restart". The interrupt discards any partially completed work of the instruction, and when you resume from the interrupt it starts the instruction over.

Another way, which is what Motorola used in the 68020, is called "instruction continuation". The interrupt saves any partially completely work on the stack, and when you resume after the interrupt it picks up where it left off.

So suppose the 68020 is cruising along in user mode, and the code generates a page fault. That interrupts the instruction, and the processor pushes a stack frame that contains a bunch of internal processor state. The page fault handler arranges to get the page loaded and mapped, and does a return from interrupt, and everyone is happy. That's the normal case.

But imagine that while the page fault is being processed, the annoying bastard user hits ^C, and that irritating git has got a SIGINT handler. The kernel, when it gets ready to return to the user after the page fault notices that it has a signal that needs delivery to a user signal handler. So it alters the return address in the interrupt stack frame to point to the SIGINT handler instead of the page faulting instruction, and diddles the stack so that when the SIGINT handler returns, it will return to the page faulting instruction.

On an instruction restart processor, that would be fine.

On an instruction continuation machine, that return from the interrupt to the SIGINT handler loads internal state from halfway through the interrupted instruction, and then tries to use it to continue a different instruction from the middle. Who knows what havoc that will cause? And then, if that didn't crash it, when the SIGINT handler returns the page faulting instruction will start from the beginning. If it had been an instruction with side effects (say, incrementing an address register) and those side effects happened before the page fault...they will happen again.

From my point of view, all I was seeing was if I had the system under enough memory load that I was getting a lot of page faults, hitting ^C would sometimes just totally lock things up. An ICE let me get a look at what the heck was actually going on, and realize my mistake.

(The fix? I made it so that if the kernel was returning from a page fault interrupt and there was a signal that needed delivery to a user signal handler in that process, it would alter the saved flags in the interrupt stack frame to turn on the single step debug flag, and would not diddle anything else on the stack. That lets it return from interrupt, the page faulting instruction resumes from where it left off, completes, and then the single step interrupt goes off, getting us back to the kernel. The kernel can then clear the single step flag, and go through the normal interrupt code, which sees the pending signal and does the usual stack frame molesting to deliver it, which is safe because it knows the single step interrupt is a simple interrupt that does not save internal processor state).

(Instruction continuation is also a pain on multiple CPU systems, although we were only making a single CPU system so did not have to deal with that aspect. Since page faults save internal processor state, and since what exactly is saved can vary from revision to revision of the CPU, you have to be careful to make sure that when you resume a process after a page fault, you resume on the same processor that took the fault, or at least on one that is the exact same revision. You want to resume on the same CPU if you can anyway, because that will generally be better because of on-chip caches...but all you risk there if you don't is worse performance. If you resume after a page fault on the wrong CPU you risk locking up the system).

You can reach me at john at common-sense.com.