Hacker News Reader

Comment by zepolen | Hacker News Reader

I've been doing the stuff in this article for almost a year now but instead of x-sendfile, I use nginx's proxy_cache which does the 'get from S3, save to disk, serve subsequents from disk' for you.

That way deploying more photo servers for speed just requires nginx and a few lines of config.

A couple notes:

1. Make sure you partition your data on your local disk properly so that there are never more than ~5000 files per directory, you can use nginx rewrite rules to do this for you (and make nginx save the actual url to nested folders).

2. If possible, make thumbnails smaller than 4kb.

View on HN

1. Good point - I didn't bring this up in the article. When we store the images locally, we create divide end of the unique ID of the image into two levels of subdirectories. This gives us "random enough" distribution in subdirectories, and an easy way to look up the files on disk. This logic exists in our rails app, but we didn't want to expose this pathing out to end users. This allows us to change it later if we need to create further subdirectories.

2. Good tip - we try to use our judgement here balancing good design/good performance. We tend to opt for a pretty website first, then go back and make it faster with optimization.

Regarding "This logic exists in our rails app, but we didn't want to expose this pathing out to end users". I'm not sure if you understood, but I'm saying exactly this. nginx will take a /whatever request and convert it to /w/wh/wha/whatever on disk transparently - you can (and I have needed to) change this whenever you like with no front facing changes.

Oh, that's cool - I didn't know you could use matches from a location in an alias. Is that a new feature in nginx?

I think it's always been a feature of nginx:

    location ~ "/(.*)" {
        root /whatever/$1;

You could even do:

        set $subdir $1;

so that it can be used in another location handler.

Reply on news.ycombinator.com