But it's not free, after all every packet carries this burden. I know about the annotation but it also makes it very difficult to parse.
Rather than trying to minmax the address length, it makes more sense to sacrifice a few bits per packet to the addresses, wait a week or two on average to get the lost throughput back, and then spend those bits elsewhere to make other things easier. (For example, avoiding NAT is an obvious one, but even just "everything is a /64" removes the need to ever need to think about the size of a network.)
If you can spend a few address bits to make something easier elsewhere, that's a good trade. Maybe start worrying if the addresses were a kilobit long or something, but they aren't even close to that.