Probably a much more common problem - definitely need to ensure that all resources are linked together through the DNS provider so that when you destroy the VM you also remove the associated DNS record if it is the last available resource.