There is an AWS CDN available that can be placed in front of the s3 bucket. Also allows for custom domain names, TLS, globally distributed for low latency ec.
https://aws.amazon.com/cloudfront/
True. For static only deployments, I would use Firebase or CloudFlare or Vercel or Netlify etc. You get automatic CDN and generous free quota.
It’s pretty easy and free to slap a CDN like Cloudflare on top of an S3 bucket