Generally speaking this stiffness is the result of the tube shapes necessary to achieve similar strength and stiffness of a steel frame while weighing less.
A perfect example of this is the Cannondale CAD3. It had a massive downtube and huge (straight) seat stays as well. It was extraordinarily stiff and terribly unpleasant to ride, for most. Cannondale improved that to some degree by using S-shaped seat stays on the CAAD4 making the frame a less-efficient conductor of vibrations.
Modern aluminum bikes use hydroformed tubes with optimized shapes and shouldn't really have these problems. Unless you're talking about something with an aluminum fork. That's just a cheap bike. Aluminum forks are terrible.