I've felt the same way. Serverless is most appealing when you're starting out and have low traffic. It enabled us at ipdata.co to have the most global infrastructure possible with the lowest latencies at an insignificant cost.
At some point I believe when we're big enough we might switch to using servers in all the regions where we currently run APIG+Lambda.