How do I mock a temperamental motor controller responding to AT commands? I mean, I'm sure it's hard enough to replicate the oddities of their serial port parser, but the temperamental part is gonna be tricky. This is an embedded system I'm talking to, which is parsing bytes coming over a serial port in an interrupt service routine at 16 MHz with 4 kB of RAM. Not an async callback with JSON in a nodejs app with GHz of processing power and 8 GB of memory. Emphasis on: temperamental.
Oh, and don't forget. This is a mechanical system, not a website. We'll have to put torque and strain gauges on the system to make sure it's doing what we want.
The skinny is, the system my python scripts talk to is far too complicated to just "stub out". You may be able to test 90% of the cases, but what you really worried about is that 10% of freak accidents that break things. Look up the Therac 25 and how that failed.
One can separate the I/O code from the application logic, and do data-driven testing of the application logic. Use recorded data from real-life systems to build a (partial) model of the system, at least enough to test individual interactions or short sequences of interactions. Capture this data continiously in production systems, so that when an incident occurs it can be cut-and-pasted to form a new testcase. The latter is critical to build up a library of regression tests for those nasty things one had to find the hard way - so that at least it only happens once.
With this basis you can optionally go to hardware-in-the-loop testing, where some of the same testcases run with real hardware instead of fake responses. This should then be instrumented with neccesary sensors/code to be able to observe its behavior. This ensures that your mock and system stays in sync, and increases coverage for non-deterministic cases. At least one such system should be hooked up to your CI server to test every revision of the software, to catch issues early.
When building the next product, consider shipping the instrumentation neccesary for QA into all products. That way the testing system is closer to real system, and one can integrate self-checking, failsafes and error reporting in production systems. This can be used by final QA (before shipping), by engineers debugging on-site and as part of periodic (or continious) health checks on the equipment.
I've done such things for electromechanical systems, though nothing with a potentially-dangerous motor yet.