AWS Case Study: Parse (YC S11)

Was it a strange experience to teach a company how to use its own product?

> I realize the diagram is likely heavily over-simplified, but what's the reason for the nginx layer

probably to load balance between unicorn instances in the particular amazon EC2 instance

I can't speak for Parse, but I've come up with something similar in the past. Nginx/HAproxy as a combo is far more flexible than the ELB alone, you might want to use it for rate limiting, better load balancing algorithms, better logging, tweaking headers, handling errors, or controlling buffer sizes for example.

We use nginx for serving static files and doing some trivial top-level routing. Haproxy is for load-balancing among families of unicorn app servers.

Squid handles external http requests that are made from cloud code.

I can't speak for Parse, but I use both of these technologies in one site. Memcached is, well, for any kind data that you want to cache: results of SQL queries, blocks of the site (HTML), etc. Redis, on the other hand, it's a final data storage on its own (like MySQL, because it's also persistent), but it's faster than MySQL for certain things. For example, for things like "List of profiles I've visited / have visited me recently on Facebook". This seems like a very simple MySQL table and query, but once it becomes big it starts to get slow. On the other hand, you can have one key per user on Redis, like "<id_user>_visit_history" which is a sorted set of <time, id_user>. You could do the same with Memcached (although it doesn't support the same data types), but then it wouldn't be persistent.

We use redis because we use resque extensively for the push notifications architecture. Memcache is superior to redis for simple caching in front of data sources.