Leaving RVA23 support until 28.04 LTS would be FAR too long.
It would be nice to see both RVA20 and RVA23 supported in the same OS but the problem is that it's not actually practical to do runtime selection of alternative functions or libraries for all extensions in RVA23. It is possible and sensible for things such as V, perhaps, but extensions such as Zba and Zbb (not in RVA20, but supported by VisionFive 2) and Zicond, Zimop, Zcmop, Zcb have instructions which want to be sprinkled all through just about every function.
You'd have to either deny your main program code the use of all those extensions, or else completely duplicate every program binary.