It's definitely possible to optimize the rlib format, like representing generic types as a tree of short IDs instead of a string or separating them into an optional debug file. Or do it like Zig with a new IR designed to be compact and easy to assemble into final executables. Even a simple global cache for the times you get lucky and end up with the same hash could help.
The problem is no one on the core team has time to do it because they are always overworked from the amount of bugs that need fixing. They also have a culture of making sure all changes are perfect before they go stable as a overreaction to C++ shipping half baked proposals. I think these are the reasons everything is moving so slow.
It had the nice effect that whoever caused the headers to need to be recompiled was the only one who had to pay the recompilation tax instead of everyone.
You may be cleaving the problem along the wrong plane if you're worried about sharing between multiple projects on the same machine versus across all of the members of a single team.
Artifactory is older than dirt. And its main feature isn't even saving bandwidth, it's ensuring that versioned libraries can never be overwritten in place with a different version potentially containing a trojan. And it can be set up not to download new versions until they've survived long enough for someone to hopefully notice a supply chain attack.
This sounds like a security nightmare - there is a reason we have build servers (well there are many reasons, but build safety is one of them).
But I have to admit the idea is clever.
It was part of the (now defunct) frea project: https://freajs.github.io/frea-website/