Hacker News Reader

Comment by whatnotests | Hacker News Reader

All excellent points, especially about cost.

I've considered "faking" the append functionality by making a new file per append action, then performing a periodic compaction.

Even compaction-via-combine-and-delete-old is clunky.

    aws s3 combine --target s3://bucket-name/output-file.txt \
      s3://bucket-name/input-file-1.txt \
      ... \
      s3://bucket-name/input-file-n.txt

I, for one, would pay extra for that.

View on HN

The lack of read-after-delete consistency makes this tricky.

https://aws.amazon.com/s3/faqs#What_data_consistency_model_d...

I've seen "eventually" consistent mean up to 24hrs in the face of problems. Several minutes seems common when versioning/bucket replication is enabled.

I can second that. Personally I'd like to see them support symbolic links so version controlling and rolling deployments of static websites becomes a little easier.

This is actually pretty kewl. You could potentially branch the current CLI from the GitHub repo and add that functionality in. Ideally the flow would work something like the following:

  1. Start a multipart object upload
  2. Issue "Upload Part - Copy" requests for each part of an object ( http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPartCopy.html )
  3. Complete the multipart object upload

Alternative flow:

  1. Enable bucket versioning
  2. Download the parsed S3 objects
  3. Start a multipart object upload to S3 with the specified target object as the object name
  4. Reupload the parsed S3 objects as parts of a single multipart object upload
  5. Delete the previous parsed objects once the multipart object upload is complete (a delete marker should be added to the top of the version stack, but the previously stored version should still be available if you specify its handle/version id).

Edit: changed formatting

Reply on news.ycombinator.com