Hacker News Reader

Based on the docs, it appears that this also allows you to upload segments of files without knowing the final number of segments or the final file size.

This will be pretty damn useful for piping the output of some process generating a large file (i.e. video transcoding) and beginning the upload before the file has been fully created.

View on HN

This will be pretty damn useful for piping the output of some process generating a large file (i.e. video transcoding) and beginning the upload before the file has been fully created.

Even better: You can split a video into pieces, transcode each part on a different EC2 node, and upload the parts directly from those respective nodes.

won't work with all codecs, but that same concept can be applied to a lot of areas!

That's an interesting idea. The only problem I can see is that every segment needs to be min. 5MB, so you'd probably need to buffer an extra segment compared to the "naive" implementationto ensure the last segment is big enough.

The last segment can be of any size so you could send the last part and it can be 1 byte. All the other parts need to be 5MB or larger.

Reply on news.ycombinator.com