They probably don't have the private key. Cloudflare has it since they are the one managing the certificate.
They likely didn't give clients the private keys before simply to save costs by reducing the number of certificates they had to issue. Now that they're not sharing certificates, it's probably just a way to extract a little more money from customers.
[0] https://support.cloudflare.com/hc/en-us/articles/204144518-S...