It is expensive. But the point where it stops being expensive is far above most companies use case. If you're paying less than a developers salary for hosting you most likely won't see all that many benefits from moving.

Renting a server from cheaper hosting providers can be massive savings but you now need to re-invent all of the AWS APIs you use or might use and it's big CAPEX time investment. And any new feature you need, whether that's queue, mail gateway or thousand other APIs need to be deployed and managed first before you can even start testing.

It's less work now than it was before just due to amount of tools there are to automate it but it's still more work that you could be spending on improving your product.