I always wondered if they did that, something akin to fuzzing tests in SW. Wouldn't it be useful to detect unexpected situations that'd be catastrophic? Or the benefits from it wouldn't outweigh the cost/time loss?
Also, imagine you're an airline with thousands of pilots and dozens of instructors: you're running an airline and a school at the same time. You need to build a curriculum of training and testing that will standardize your pilots. There's room for thinking outside the box but not too much.
Well Boeing certainly doesn't fuzz their software as evidenced by the major bugs in the 737 NG and 747-400's flight displays. Both had bugs that would black out all instruments under specific conditions. That got fixed fairly quickly on the 747, but apparently Boeing didn't learn their lesson with the NG.
The bug blacked out all six display units. What other instruments are you thinking of?