It’s a good idea to use a reverse proxy like Nginx to handle the transfer through an internal redirect, which will hide the bucket name and authentication credentials.
But this means data transiting your network instead of being directly fetched from AWS infrastructure, which for me mostly defeats the purpose of presigned S3 URLs.
If your server is on AWS, then the egress cost should be identical whether the transfer originates from S3 or EC2. But if your server is on premise and your host charges for egress (or has impactful bandwidth restrictions), I can understand where you're coming from, as you could end up paying twice.
Introducing complexity, failure point, scalability issue just to work around broken billing. You are probably right but I feel it’s wrong architecturally. I always liked S3 precisely because I could offload file serving to them.