I'm out of aerospace for about 15 years and only worked there as a junior engineer, so take this with a grain of salt:
There is always a payload. You could not just launch a rocket with an empty payload fairing. The payload dummies used are not only trimmed for appropriate mass and COG but also for the proper frequency response.
I know this, because as a young engineer I did the FE-models of the MaqSat (H2 and B2 IIRC) dummies for the Ariane 5 and spent days trimming their frequency responses.