Why is the choice always portrayed as if it's either cloud, or on-prem / colocation ? Those are two extremes. At work I use dedicated physical machines from Hetzner. If I need an extra one they deliver it in a few minutes, and thanks to Ansible it's provisioned within a few minutes more. Hetzner keeps an eye on the hardware and replaces disks, PSU's and the like if needed.

I wouldn't often advocate to do colocation or on-prem since that indeed comes with a whole set of headaches, but renting dedicated physical servers like we do offers a lot of flexibility, with very little overhead, at a price/performance ratio that makes AWS look like pure extortion.