In 2008 we had racks of servers we were leasing and that were sitting idle most of the time just so we could stress test our Windows Mobile apps.
I've been developing professionally for 20 years and 10 years before that as a hobbyist. I know what a pain it is to get hardware for what you need when your company has to manage all of its own infrastructure.
Just setting up EC2 instances and installing software on them doesn't reduce the pain by much. Sure you're cutting down on your capex but you still end up babysitting servers or doing the "undifferentiated heavy lifting", I would much rather stand up a bunch of RDS instances.
As far as serverless, why manage servers at all when you can just either create a Lambda function for the lightweight stuff or deploy Docker images with Fargate? That's just one less thing to manage and you can concentrate on development