I'm not sure what your definition of "odd times" is, but automatic transmissions' decisions of when to shift constantly annoy me. (Most of my automatic experience is in a Honda CRV. Never driven a Toyota.)

That may just be a fault in the transmission. Although it's new enough that it shouldn't be a wear issue, it could be gum in the valve body (do they still have those?) or a fluid problem.