You could have a completely bug-free, constant-time, constant-power cryptographic library running on one of these microcontrollers, and debug access would allow you to reliably extract encryption keys just by examining the execution path.
The amount of processor and system state that you have access to with a hardware ARM debugger is crazy, but that isn't really the problem -- you can extract a ton of state with a minimal debugger too. Just a log of instruction pointer values would get you 90% of the way there.
I think it's reasonable to assume that microcontrollers with exposed debug interfaces simply cannot be made secure, just as people generally assume that it's game over once someone has physical access to a computer.