>Without commenting on whether or not I believe the Bloomberg story, I very much doubt that "people were tearing these boards apart", doing reverse hardware-level engineering looking for an unknown. Even in the case of open-source software, with full source code and documentation available, the number of people who analyze code is miniscule.
I can tell you people were literally tearing them apart; cutting open capacitors and the like (common place to hide extra components) pulling apart power supplies (as I recall, one of the bloomburg illustrations implied that something was in a power supply; I remember a long discussion of how you could compromise something over i2c from a chip actually in the psu) - I mean, sure, these things could be better hidden to the point where such techniques would be ineffective, but my reading of the bloomburg article was that they weren't particularly well-hidden, and that therefore they were discovered. Hiding that sort of thing from another EE who designs that sort of thing seems like it's probably pretty difficult.
(I mean, it is common to have compromised firmware; that's way easier. but that's also not what the bloomburg article was about)
But... yeah, I guess a lot of my assumption was based on my reading of the bloomburg article implying that this hardware was something tacked on in a detectible way; the sort of thing that could be done by some third-tier subcontractor. I acknowledge that if your attacker has the capability to produce ASICs that look just like the real thing and act just like the real thing out the same I/O pins, except in some rare case? yeah, that's not going to fall to this level of examination. but that wasn't my reading of the bloomburg story.
Furthermore, if an attacker went through that level of effort, they'd probably not just do it for supermicro. Most motherboards of the same generation are pretty similar; pretty heavily based on reference designs. If you were going to do something that sophisticated, that's where you'd attack.
If you were going to attack just one particular rev of one particular brand of a motherboard? well, something tacked on seems a lot more feasible.