I'm kind of disappointed by the cynical tone that has been increasing on HN over the past year or so. This is a straightforward blog post that is transparent about a wide range of technologies used in several areas (not just the user-facing portion). I found it informative. For example, it validates some of my concerns about scaling using PostgreSQL. They're using a single write server in a top 2000 site (according to Alexa).
Yeah as am I. These cynical comments can be completely replaced by a downvote if they honestly don't think that this is interesting or appropriate.
On a separate note, it was great to hear that PostGIS (which is amazing btw) scaled well and can be used on a site with this much traffic!
Interesting to hear they were so dissatisfied with Celery/RabbitMQ that chose to write their own queuing system. I'm looking in to implementing a queue for background tasks right now, and was down to choosing between Celery/RabbitMQ and beanstalkd. Perhaps I'll check out their new "Taskworker" ("Kale"?) app that they have open sourced. (https://github.com/Nextdoor/ndkale)
I know it's early days, but I'm excited about Salvatore's new project, Disque. He certainly knows how to find a technical niche with a big need.
Python/Django is still lovely for putting together a quick project, but I don't think anyone would pick this stack (Python/Django/Tornado/RabbitMQ etc) for a scalable server in 2015 anyway. It was a good choice in 2011-12, when Nextdoor started. As of today what's posted in this blog is pretty common knowledge within Python backend developer community. Of course the comfort level of team members in picking up other technologies matter.
What do you think they would use now? I'm not sure Node/Mongo etc is without it's problems.
File under: "Things I Never Wondered About".
some kind of queuing as well
I'm guessing there's a web stack of some type, backed by a database, perhaps with an API mid-layer. Probably some type of FTP or other content upload system for articles, and an image origin for a CDN. Also, perhaps a data ingestion and ETL system.