The clutch is much more difficult to replace on my car; (i've done the brake discs 2x -- track days),
If you're relatively smooth on your downshifts, each downshift should be much shorter than your average braking duration.
More generally speaking, if you're in a gear that slows you down, or downshift to it, you're more likely to avoid a full stop and start, which puts much more wear on the clutch than a series of downshifts.
... I also downshift when I drive automatics ... when I can... there are large downhills near my neighborhood (in a school zone) coming from a stop, if I switch to manual mode or short/sport mode, I can be in 3rd, at a controlled speed, and barely touch the brakes (versus others ride their brakes all down the hill)