C++ is not just boost and ICU - there are a handful of C++ libraries with a good ABI compatibility track record, for example Qt and the KDE stack.

The C++ ABI of Microsoft Visual Studio is stable since practically forever, albeit with a severe restriction: the language ABI is stable, but not the C++ (or even C) standard library ABI - every Visual Studio release comes with a new incompatible msvcrNNN.dll. But you can load as many different runtime libraries into a process as you like, the big practical restriction is that you can't pass standard library objects allocated by one runtime to a different one.

The C++ ABI of GCC is stable since version 3.2, released in 2002, on x86 platforms (there were numerous fixes in later releases for other platforms). They even managed to retrofit new incompatible C++11 requirements on std::string with some preprocessor hackery and namespace mangling, without bumping the libstdc++ SONAME.

As usual, on macOS the situation isn't as good because they threw out libstdc++ for some clang reinvention of the wheel.

It's not really true that C++ does not have an ABI advantage over Rust if (by your own comment) they both end up using the C abi. Although pedants will insist that C is not a subset of C++, practically speaking you can write idiomatic C perfectly fine that will compile with a C++ compiler, and much C code in the wild is actually written this way. And most moderately experienced C++ developers are very comfortable with all of these elements and syntax (like pointers, goto, C style callbacks, etc), because they are a subset of their own language. It's very straightforward to have your API be C (and therefore have a stable ABI) but do implementation in C++. Even some C standard libraries do this.

In summary, if you're writing a C API/ABI, source compatibility is still a major advantage, there is no two ways around it.