(That said, I have very little personal experience with CloudFront, as in my experience it is more expensive for fewer features with less POPs than using a "real CDN" like CDNetworks, or even Akamai.)
(edit:)
For this specific circumstance, I'm not certain at all what CloudFront's behavior will be; it seems like the "redirect" concept is a property of the "static website hosting" feature of S3, not part of the underlying bucket, and CloudFront "normally" (in quotes, as I just mean the default origin options it provides) directly accesses the bucket.
I thereby imagine that if I simply set a custom origin to the ___.s3-website-us-east-1.amazonaws.com URL provided by the S3 static hosting feature I will get the right behavior (where CloudFront forwards and caches the 301 responses), but then I have no clue if it will correctly retry the 500 error responses.
That said, I will point out that I am not even certain if CloudFront retries the 500 requests anyway: it occurred to me that with a small error rate combined with a cache, if you (as I somewhat did at least) expect the potential fix to be S3-specific, you might simply never really "catch" an actual failing request in a test scenario.
It could then be that CloudFront retries all 50x failures (in which case if I set it up with a custom origin to the S3 static hosting URL you'd still get the retry behavior), but I somehow doubt that it does that (and just earlier I saw two requests in a row to S3 fail for these 301 redirects, so it might not even help).