Despite how obvious it is with a little outside observer perspective, the "perfect bricks" way of thinking is pervasive because that's how the builders and supply chains are organized.
To leave the analogy behind, any non-trivial component has a testable surface area, but typically has additional modes of behavior associated with internal state, environmental conditions, or other areas that well-meaning unit test writers didn't think about.
I have often found issues in simple caller/callee pairs of two components, both of which are tested, but the caller contains subtle expectations of the callee that the unit tests don't match up with.