Basically you overwrite the instruction you want to break at with a breakpoint instruction (e.g. int 3 on x86). This will cause the process to trap and the OS will then let the debugger process know about out somehow, e.g. via the SIGTRAP signal on Unix.
The debugger then replaces the int 3 opcode (which is a single byte conveniently) with the first byte of the original instruction so that the execution can continue.
In a trivial example, the breaking instruction could be a jump to itself, which you’d expect to immediately break into the debugger again.
I thought the debugger had to emulate the instruction instead, but it’s not like I’ve ever implemented one…
I really only implemented a debugger for the esp8266 and it was just good enough for me and my team to get our job done so it didn't handle many edge cases like that
1. Overwrite instruction with int 3.
2. When you hit the breakpoint, restore the original instruction.
3. Single-step over the original instruction by changing the thread's EFlags (Intel).
4. Restore the breakpoint with int 3.
5. Resume normally.
You could also do something like have a clean mapping table (i.e. the code with no breakpoints installed) that you install for just the thread doing the step. You then revert back to the normal mapping table with the breakpoint after the step. As you are only modifying the executable section, as long as you are not using self-modifying code, there should be no data inconsistency with having a multiple copys of the executable transiently.
* Restore the original instruction byte.
* Find the next instruction, and set a temporary software breakpoint there.
* Resume the one instruction
* Restore the original instruction byte at the temporary software breakpoint.
* Set the software breakpoint in the original instruction
* Resume running
The other thing to keep in mind is dealing with JMP, CALL and conditional branch instructions. It can get pretty messy pretty quick, which is why I find low level debuggers on old 8-bit CPUs a marvel as they had to deal with only software breakpoints.