Yes, that's right. The little rollers were on an axle that extended a bit further terminating into a spoked disk which sat in the middle of an optical detector and emitter pair. It used some tricks to figure out when the mouse was going forward and when it was going backward (quadrature), but mostly it just counts the pulses of light it sees and turns that into displacement coordinates.

EDIT: Wikipedia has a pretty good picture of this and a better explanation: http://en.wikipedia.org/wiki/Mouse_(computing)#Mechanical_mi...