The rubidium does not deplete (it's a closed system), but does settle on the glass. Supposedly you can heat the glass to turn all the rubidium into gas again, giving you theoretically infinite lifetime. I picked up a used PRS-10 on ebay for $250. It's a small but great oscillator, still made today. The manufacturer SRS claims 20 years lifetime, and mine has had 13 years of runtime, so I should be able to get 7 years out of it hopefully. The 11W1 DSub connector they used on the other hand is stupidly expensive.

Cesium oscillators on the other hand do deplete their cesium (by design), as mentioned by CuriousMarc in the video. The tube in his unit was probably replaced at least twice, at a cost of >80k. The high performance tubes fare even worse because they "burn" through their cesium at a 3x rate.

The 5071 successor by HP-Agilent-Symmetricom-Microsemi is the most frequently used COTS clock that contributes to UTC.