Yes, what is called "gravity turn" turns the velocity vector, but if the rocket didn't actively reorient itself (so as to keep in line with the velocity vector), gravity itself wouldn't rotate the rocket (change its attitude), because almost-homogeneous gravity field won't impart (meaningful) momentum. (Satellites can actually use it that way, but the time constants are substantial and not really relevant for an ascending rocket on a timescale of minutes.)
However, this is independent of the presence of the atmosphere. In real life, both scenarios need active control of attitude.