Not necessarily, they might have just generated revokation certs which are separate (so that they can be used in case your private key is lost). Keeping a revokation cert would be a responsible thing to do, just in case something like this happens.