Exclave: Hardware Testing in Mass Production, Made Easier
bunniestudios.com
bunniestudios.com
At the outfit I work for this is pretty much what they were doing for one of the products when I started. It wasn't that they didn't know better (they had other items in mass production); it was that the product used to be such a low volume that typing in CLI commands to program and test it was (almost) good enough.
I spent months automating that to the point of "plug cable into device and watch for Pass/Fail" indication on screen. Now that we're shipping thousands per month it make a huge difference :-)
Testing is a weird thing. The points you test are a function of what you expect may fail in manufacturing. No point in testing something that's extremely unlikely to fail, unless it's required by regulation.
e.g., FDA (Food & Drug Administration) Good Manufacturing Practices state that if you don't build something using a documented and repeatable process then you must test 100% of product coming off the line. If you have a tight manufacturing process, then you can decide what level of testing meets a good balance of safety and cost.
People spend their entire career as Test & Manufacturing engineers and it is quite a complex and interesting field.
Frankly, this just isn't true. It depends on your expected defect rate, how your defect rate drifts over time, the cost of doing 100% inspection, and effects of product failures. If you're making safety-critical gear, yes you will probably inspect 100%. If you're making injection molded clamshells for connected consumer products, you're probably only doing inspection in batches.
Back in the day extremely elaborate and massive bed of nails in circuit test rigs would poke the leads of a through hole circuit board to do continuity testing and basic functions checks. Now a lot of that can be done with JTAG circuitry, a serial bus many ASICs implement and chain together to provide less invasive way to similar tests. Xray, thermal, optical scanners are often used in various parts of the assembly. And of course plenty of humans. Test rig design is a big part of any manufacturing process and may be used to verify overall system function like button presses and connector fidelity.
As a separate but related thought, in VLSI: verification design, for both manufacturing and BIST, can easily eclipse the rest of the design in some ASICs in terms of cost/effort/complexity.
Software engineering is kind of the odd duck in that it is frequently a shoot first ask questions later pursuit in most organizations (funnily, even commonly in places like semiconductor companies that invent and progress new formal methods). There are plenty of decent tools like Lamport's TLA+ that aren't widely used in addition to more familiar aspects of code test.