I don't think users would notice but the IPFS network might collapse.
Besides, even if you work around that problem somehow, YouTube has an insane amount of videos with very few views. That means there are not enough people to upload content, the centralized server or the original uploader needs to keep hosting them.