Hacker News Reader

Comment by hartator | Hacker News Reader

The author seems to be unaware of being able to upload directly to S3 via JS eliminating the need of even the Phoenix app.

View on HN

I'm not sure if it's a recent feature, but I was unaware of it until recently. However, you'd still need an app to sign the upload URL. Otherwise, you'd have to expose the AWS secret key to the client.

It's mostly a matter of setting up an endpoint in your app to sign manifests using the secret key. It's give or take a dozen lines of code on the backend (to do it in an "unsophisticated way"), and was one of the first things I wrote in Elixir! These two links should get you on your way:

1. http://bencoe.tumblr.com/post/30685403088/browser-side-amazo...

2. https://aws.amazon.com/articles/1434/?tag=vig-20

I should make a service that handles this for people. That has to be in demand, no? :P

What type of security implications are there from this? Are things like rate-limiting (by time period, all-time) or access control handled properly?

I clicked on this article because I needed to do something similar this week, and ended up using the JS-based approach. I’ve built something like this with S3 before, but ended up doing it with Google Cloud Storage this time around. The process looked like this:

1) Client-side app receives an image (via drag-and-drop or file picker)

2) Client hits an authenticated endpoint on backend with the name of the file, which returns a “signed URL” (authenticated, with customizable expiration) from Google Cloud Storage

3) (This part is particular to GCS, and I think unneeded for AWS/S3) Hit this signed URL in the client, and receive a final, PUT-able URL

4) Upload the file using the final authenticated URL from Google Cloud Storage

Like I said, I think with AWS/S3 you can just request a signed URL and send data to that URL directly. Delegating as much communication with GCS to the backend was important for my project [1] but you might be able to just manage it with a JS lib.

[1] Streaming music, so trying to abstract the actual files from the client-side as much as possible to discourage downloading the raw audio files

s3 requires a signature as well. Without requiring a signature, anyone could upload anything to your s3 account. With a signature, you choose when and where people can put their files.

If we want to store the object's URL in a relational database, should the browser send another HTTP request to our service? Wouldn't we lose the object if there's a network error during the interval between the two requests?

Yes, one option is to do as you've described, for example, use JS to upload the file, then submit a form as normal with a file url / id in the form.

However, you can also create a Lambda that is triggered whenever a file is uploaded to a given bucket and directory. With this, you can react to the file being received and update your database without the need for a second request.

There's also the fact that Puma has slow client support so this issue isn't even an issue.

Reply on news.ycombinator.com