S3 Multi-Part Upload API can be used to chunk an object into smaller parts, which can succeed or fail independently.
https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview....
https://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview....
* chunks need to be >5MB which can be problematic on flaky/poor connections (rural areas, tunnels, clubs/basements, people on the move switching connections all the time)
* your s3 bucket needs to allow write by the world, or you need to deploy signature authentication
* there’s an s3 vendor lock-in some might worry about
* not an open protocol, no chance of advancing it with the community
That said, that still leaves a large audience for direct s3 resumable uploads and I’m thankful aws offers it!
S3 is great and in fact, at Transloadit we deploy a content ingestion network (reverse cdn) of many regional tusd servers, close to our customers’ end users, but they all ultimately save to S3 using multipart. We’re happy S3 customers.
So why the extra layer. Because this let’s us offer resumability below 5MB, lower regional latencies, roll our own auth, and switch to a different cloud provider without introducing breaking changes at the customer facing side (assuming the new cloud bucket provider does not offer an S3 compatible interface, or even just a slightly incompatible one)
Ultimately you’re still locked-in with AWS protocol-wise, and there’s no community platform for advancing it, so addressing any of these issues is going to be hard.