It's a nice thought but you have to be confident every 3rd party project you use is one whose parallel build you'll maintain as it updates. That's the strength of Cargo - it's the way to write Rust code & the maintenance is done by upstream for everyone.
Additionally Cargo's strength isn't just it's packaging & publishing system. Having a single Rust compiler that everyone uses on all platforms is powerful. C++ might get there eventually with clang but the ecosystem of libraries will need to support that as a baseline. Currently there's too many compiler or platform-specific hacks that are done to regularize the environment a bit whereas Rust generally does all of that under the hood for you (i.e. try writing any cross-platform BSD socket code - it's a total shitshow that basically results in Winsock & all other POSIX OSes basically different net stacks). Rust also comes with a very deep standard library for I/O, path manipulation etc whereas the C++ equivalents for all of this are still surprisingly lacking even in C++20 (especially on the network side).
[1] Based on manually counting the packages listed here https://wrapdb.mesonbuild.com/