I've run a project for 6 years on a single $10/month VPS (I pay even less due to a perpetual discount I bagged from lowendtalk) run by a gameserver-focused VPS provider for about SIX years with about 99.999 reliability if you exclude the one time I fucked up a config and it was down for a whole day because I wanted to do a clean OS reinstall, and one other time when they changed my IP address (they gave me notice).

VPS technology has come a very long way and is highly reliable. The disks on the node are set up in RAID 1 and the VM itself can be easily live migrated to another machine for node maintenance. You can take snapshots etc.

To me, I would only turn to cloud infra not for greater reliability but more for collaboration and the operational housekeeping features like IAM, secrets management, infra-as-code etc, or for datacenter compliance reasons like HIPAA.