True, you do. What you gain is the ability of small pieces to move individually through API changes.
If your entire codebase is in one repo (as appears to be the case here), and you want to change an API, you must either do so in a backwards compatible way, and slowly eradicate any old callers, or change them all in one fell swoop.
By splitting to multiple repos, you can version them independently. Thus, a project can (hopefully temporarily) depend on the old API, which only gets bugfixes, while another project can depend on the new version.
The tricky bit is when you have one "binary" or something equivalent referring to two versions of a dependency. (Usually indirectly, i.e., A depends on B which depends on D v1, and A depends on C which depends on D v2, and D v1 and D v2 are incompatible.) You can't really do much about this, but if you keep your components small enough (think services with well separated interfaces) you should be able to keep the dependencies small enough as well.
Individual libraries/dependencies get worked on by themselves, with an API that other applications use. Then the other apps just bump a version number and get newer code.
That is, the reason an API changes is because a use site has need of a change. So, at a minimum, you need to make that change and test it against that site in a somewhat atomic commit.
Then, if the change has any affect on other uses, you need a good way to test that change on them at the same time. Otherwise, they will resist pulling this change until it is fixed.
Add in more than a handful of such use sites, and suddenly things are just unmanageable in this "manageable" situation.
Not that this is "easy" in a central repo. But at least with the source dependency, you can get a compiler flag at every place an API change breaks something.
And, true, you can do this with multiple repos, too. But every attempt I have seen to do that just uses a frighteningly complicated tool to "recreate" what looks like a single source tree out of many separate ones. (jhbuild, and friends)
So, if there is a good tool for doing that, I'd certainly love to hear about it.