Isn't that what we actually do, though? Everyone is taught to subtract or add least to most. Why would we want to flip anything.
Over time we all realised that big-endian systems (IBM, 68k, etc, and our native western languages) were a hack and we've moved to little endian systems (intel, arm, DEC, ...) as sane