I don't really understand what you're getting at. No language should provide a package manager? All languages should use the same package manager? That seems unnecessarily constrained and coupled.
This is all so vague, and that confuses me. Language package managers are "destined to decline in quality" for unstated reasons. Language package managers are "don't work well" because there are multiple languages (and I presume you also mean multiple operating systems/distributions). That argument would seem to apply equally well to any package manager (including those maintained by OSs), so why are we singling any of them out?
(Cargo does have a build script mechanism, with a major use case being integrating with other languages at compile time. Eg, running `make`. The other package managers I'm familiar with have similar mechanisms. I would assume they all do, since it's a problem they all have.)