The array->pointer "decay" (as the standard calls it) has 3 exceptions, of which two are when an array is the operand of "sizeof", and when it is the operand of "&". (The third involves a string initialiser, which is not relevant here.)
So your reasoning is not quite correct, it should really be that you think of
sizeof( &array[0] )
as being sizeof( &(something) )
where "something" could be of any type T, and so the '&' operator yields "pointer-to-T", to which sizeof will yield the size of a pointer.