If your lambda runs very infrequently, you'll see more cold starts which will take hundreds of ms, but warm starts are pretty easily in the single-digit milliseconds (at least for a Go lambda). If you're doing a bunch of compute or sync I/O in a loop then your functions will take longer.
But less frequent invocations are actually an even better case for Lambda versus a VPS because it suggests less wasted time (yeah, you have to pay for cold starts, but with a VPS you're paying for all of that time your service is up irrespective of whether or not it's being used).
> 100's of milliseconds minimum. When your serverless endpoint needs to integrate with other AWS services, the configuration can get complicated fast. VPC endpoints, IAM roles, security groups... it goes on and on.
This is all true for a VPS as well. Assuming you care about reliability, you need to run multiple instances of your VPS behind a load balancer which implies a fair bit of networking. Moreover, you need to manage the hosts themselves, so configuring log aggregation, metrics collection, process management, SSH, deployment, hardening, etc, etc, etc.
Further still, your compute will need to communicate with the comparable services as in your lambda hypothetical, so you still need to deal with IAM and some more networking stuff. Maybe you'd say "gotcha! I would just run my databases on my VPS instances!" which is cool, but now you need to configure monitoring, backups, replication, failover, and so on for your databases versus using DynamoDB (and I would bet a lambda/dynamodb workload than the same workload on a reliable VPS/whatever-database stack).
Of course, if you're running a hobby blog or something that doesn't need reliability of scalability, then a $5 VPS is probably fine (maybe even an S3 bucket behind a CloudFront distribution, which would likely be free).