As it is, my iPhone unlock PIN is everything that's needed to decrypt the data server-side [1], and I'm not changing to an alphanumeric password on my phone only because of that.
[1] https://support.apple.com/en-us/HT204915 ("You might also be asked to enter the passcode of one of your devices to access any end-to-end encrypted content stored in iCloud.")