"until Twiddla went big"
How big to have to change?
"half dozen boxes ... for S3stat"
What kind of boxes do you use and how much do you need to process?
(I use 2 of your services and I love it)
How big to have to change?
"half dozen boxes ... for S3stat"
What kind of boxes do you use and how much do you need to process?
(I use 2 of your services and I love it)
That one box handled the load remarkably well, especially considering that I hadn't done much backend optimization at all, and that we were still persisting every single message to the database. It went a long way towards solidifying my faith in the MS stack, and making me scratch my head whenever I see somebody's blog fall over from a minor HackerNewsing.
As to S3stat, it runs on EC2 m1.small instances. Towards the end of the month, when each bucket takes longer to process, we'll step up the number of workers we fire up each night, and occasionally pull in a few c1.medium instances to speed things along.
It's a fun architecture, and I've been meaning to write it up for a while now. It uses six distinct AWS offerings to do its thing.