Interesting you mention this. That was the first approach. The problem comes when you want to introduce missing features to S3. Like tokens, or file filtering.
Couldn't presigned URLs work like tokens? And can't you do filtering with s3 select?
It's been a while since I did anything complicated, or much at all with s3 so I am not claiming to be any authority. Just curious!