One reason why Chinese have less difficulty remembering and using these numbers than the western fellows might be the difference in pronunciation. In Chinese, all single digit number has the same structure: a consonant plus a vowel. For example, 7, in Chinese is pronounced as [ch-i:], while in English is pronounced as [ˈsɛvən]. Also, every digit takes the same time to pronounce. For example, a string "123456" is [yi:, er, san, si:, wu, liu]. That, in my experience, makes a long string of number easier to read out. And easier to read out means easier to memorize.
A side node: In China every kids in their elementary school if not kindergarten can recite the "table of multiplication". That is, they can remember the answer from 1 * 1 to 9 * 9. I doubt how many westerners can do so, due to the language difference.
Edit: The asterisk symbol is driving me nut.