Upload Images to S3 in GraphQL Using Rails and Paperclip | Hacker News Reader

Upload Images to S3 in GraphQL Using Rails and Paperclip

graphqlme.com

34 pointsmengledowl19 comments

(Background: nearly 2 years of working with GraphQL in production)

I tend to think uploading files via GraphQL is an anti-pattern. The downsides outweigh the benefits over just using presigned URLs.

Advantages:

* Possibly simpler to get a simple case up and running

* No need to use a file store that supports presigned URLs

* Can perform processing of files during upload.

Disadvantages:

* No progress events during upload, so you can't show a progress bar

* A GraphQL server wants to be as lightweight as possible, and not keep requests open for long. The common resolution strategies tend to involve creating a lot of file descriptors, dealing with files in the form of base64 strings or attachments is asking for trouble.

* Simplicity benefits don't last long.

* Perceptively slower for clients.

Do the downsides still outweigh the benefits when you consider having multiple clients, especially when they might include mobile apps? If something changes about how I'm doing file uploading (eg. moving to a different file hosting provider), I really don't want to have to deal with the implications of making that change in multiple places, especially when it comes to a mobile app where you can't control what version people are running.

This isn't viable for large files. First uploading to your server, then through to S3. It will take a long time. Have your api return the upload url and then when you switch providers, instead use their signed url? AWS and Google you just PUT to the url, shouldn't change anything. Then hit your own api after it finishes

The pattern i'm suggesting is how most mobile apps i'm familiar with work anyway. Though some have made the mistake using an AWS SDK in the client, rather than just using presigned URLs.

Basic pattern is:

Client: Hey API server, I want to upload a file

API: Sure, here's a URL you can PUT the file directly to. It will expire in 10 minutes, you can always ask for another if you run out of time

Client: Thanks!

-- Client uploads file directly to URL --

This seems unnecessary at first pass. Why not just upload directly to S3 using their REST API and then persist the returned data via GraphQL (if so desired). Am I missing something?

That was my first thought when I read the headline... API or, hell; just use the CLI directly. Works great, I don't get it.

That said, I have no idea what paperclip is and I wouldn't even consider using a framework like rails for such a simple task.

The premise of the post is having a Rails application with a User model and the requirement of letting users upload profile pictures to S3. The post should not be much interesting outside Rails development.

The question is why doing it by base64 encoding those images through the GraphQL API, but not all tutorials show useful examples.

BTW, paperclip is a gem that allows to add files to a model, with extensions to handle several backends (S3 among them) and process those files (convert formats, crop images with frontend integration, etc). It's probably the oldest gem in that genre, the main alternative being Carrierwave.

Yeah that's fair, I should have... read the article.

I think the OP is simply trying to do a How-To File Upload with GraphQL article, the S3 is secondary.

Alternatively, if you needed to allow some user to upload files, you might want to guard against abuse and generate expiring links.

YES. This is basically proxying to S3. Why would you ever want to do that? You want to give clients a presigned URL to do the upload. Lots of ORMs support "file fields" though which encourage you to do this or something similar.

OP here.

There are a few reasons you might want uploads to go through the server, and they basically amount to having more control. Imagine that you have an API that's being used by multiple sources - maybe a web app and a mobile app. The initial work to create uploads to S3 directly for each might not be too bad, but what happens if you want to change something later? Maybe you decide to move from S3 to something else, or maybe you decide that each upload should actually generate 3 different sizes of the image. If you do it through the server, this is all relatively simple. On the other hand, if you do it directly to S3, then not only do you have to change it in two places now, but you also have to navigate the issue of app versions - your original app version is uploading to S3 but your new one is uploading to some other hosting service, so how do you really get off of S3? And how do you keep them in sync? Definitely possible, but much more painful than changing a few configuration options on the server and migrating the existing data, etc.

I think you are mistaken. Currently best way to handle images for user is to upload directly to S3 via JS -> Save the S3 id to db -> Resize on the fly via aws lambda.

Not an expert on this.

This doesn't seem secure. How are you confident you don't allow people to misuse your s3 bucket?

The S3 SDK allows you to generate a URL that can only be used to upload to a specific path (+ there are other constraints you can enforce, file size / mime type etc)

You create a policy [1] for each upload and sign it. The users browser has to provide the policy which can set the path (or prefix), limit file size etc.

[1]: http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOS...

Thanks! TIL and I will actually need this info next month :)

"Best" is relatively subjective here, I think. Some people may agree that your approach is "best", but plenty of people would also prefer to keep as much of their logic in one place as possible.

Surely Multipart is a more standard way to achieve this?

This also needs to use Go, Rust, and Haskell at a minimum.

Reply on news.ycombinator.com