Hacker News Reader

24 servers? Can you elaborate on your architecture a little more? What tools do you use to help you manage them?

Most of the servers are workers which fetch feeds. Parallel downloads across a number of low-level commodity servers. Additionally, there's a master and slave Postgres database, a small Cassandra cluster, 2 redis boxes, an elasticsearch cluster, front-end servers, caching servers, etc. Shoot me an email if you want to talk in more detail.

and one server to rule them all...

One server to serve an RSS reader to the 5k users, 23 others to host 23 different stats and monitoring applications to make sure that one server is working OK?

I can't speak for him, but the RSS reader I am working on at the moment is similarly split into multiple parts:

* Crawler - retrieves RSS feeds and parses articles

* Database server

* Analyser - performs semantic analysis and various other statistical processing on posts

* Propagator - pushes new posts to everyone's feed

* Cacher - serves a cached copy of the user's feed

* Web server

Based on loads I would just add more nodes depending on what was needed.

Pretty close :)

One server (machine), 23 servers (waiters) to cater to that one servers' (machine) every needs?

Closer to 50/50. 50% of the servers do actual tedious work. 50% cater to users' needs.

Reply on news.ycombinator.com