I would mount the S3 bucket locally then let smth like nStream do its job e.g. https://github.com/s3fs-fuse/s3fs-fuse