Just a simple misunderstanding/communication problem. The "DNS-only" option seems to be reached by clicking "Pause website", that will "temporarily disable CloudFlare". Seems quite reasonable to still have the certificate available, so it can be used right away when you unpause.
Just checked this for my domain. Looks like CloudFlare requested about 3-4 dozen certificates for my domain. It does worry my that this many certs are going around, but given that they already manage my DNS, I've already given them enough power anyway.