> The ordering of digits in Arabic is not obviously relevant, per se, since spoken English ("one hundred twenty one") matches the order of the Arabic numbers, too.

I think it is relevant. It is possible that Western mathematics copied the Arabic notation (with right-to-left numbers), without also copying the correct way to read it (also right-to-left). For a similar situation in language, think of accents and the many different ways you can pronounce the same word.