IBM was extremely vertically integrated. What actually happened was that Compaq reverse engineered their BIOS and made a compatible system and the courts said it was okay.
The thing destroying the modern world is DMCA 1201. Adversarial interoperability is fundamental.
Rest assured there would be no IBM PC compatibles otherwise, in fact PS/2 and MCA architecture was a failed attempt to regain control.
The difference is that now if someone wants to make a competing store for iOS apps, or a competing client app for Reddit or Twitter, the incumbents use the DMCA and remote attestation and similar anti-competitive levers to lock those competitors out of the market and governments have been letting them get away with it.
It is easy to say that IBM had monopolistic tendencies, because they did. On the other hand, the 5100 sounds like a vertically integrated design and it ended up being extremely expensive. The Datamaster strayed away from IBM components, but was still based upon IBM software. One can blame it's failure on the IBM PC being released a month later, but in a way that would prove the point. The Datamaster had such a big head start that the IBM PC borrowed elements of it's design, while the proprietary software of the Datamaster held back its release date.
Undoubtedly, IBM wanted more control over the PC. Avoiding a completely proprietary design was probably a calculated bet, an intentional course of action rather than a mistake. It just happens to be a bet they lost.