The missing piece is having CBDCs that your bank accepts on something like Uniswap. Until that happens on/off-ramps are needed, and these can only be centralised by definition.
I’m not sure having ubiquitous CBDCs with all that may entail would be better than having a few trusted centralised exchanges for fiat on/off-ramp though.