I think you do memorize them, albeit not consciously. The results of operations on small numbers are "cached" in your head while operations on large numbers are subdivided into more operations on smaller numbers, and so on.

I guess OP meant that for a base-60 system we would have to do a lot of caching.