I initially had issues because I was pointing CloudFront at the normal S3 endpoint. Once that was solved, everything started working smoothly.