Note: This is completely wrong.

The yard stick has a normal force vector from the hand passing through its center of mass. It doesn't start to rotate unless an outside force is applied.

Of course in the real world it requires active control because the most minute misalignment causes a feedback that moves it further out of alignment. The rocket is exactly the same, if the thrust vector is not directly between the center of mass of the rocket, and the Earth, it starts to tip over.

If it were a stable system, misalignment would push the system back into alignment.