That said, there are so many benefits of running multiple servers behind a firewall off the bat, I can't fathom why people don't do this. It more or less enables trivial failover for many classes of problem, and deploys are no longer as large a source of instability: deploy by launching a new box, route a small amount of traffic to it at first, then route all of it. If there's an issue, route to the previous app servers.
Compared to the time wasted by getting the architecture wrong off the bat (and months of pain while you wait for there to be enough time to do the work, which never happens), this is a massive win.
I suspect if asp.net MVC hadn't been out, it would have been written in webforms.
I seem to remember asp.net MVC had only been out a couple of months when they started writing it or not even fully released, I was learning django at the time because MVC so was so obviously better than webforms. But because c# was so much faster than either ruby or python, when asp.net MVC turned up, and it was very, very good for a v1, I stuck with c#, as did many other .net programmers I suspect.
At the time SO was written rails and django were still new and slowwwwww, php was king and still mainly written by page without frameworks, Java who knows, and the MVC revolution was just starting.
I'm not sure what you think he would have written it in? I have a feeling if you hunt through coding horror you'll probably find a post from the SO start time period saying "use what you know". In Jeff Atwood's case that's C#. I know that Joel's company had written their own language that I think at the time was still in use, so they wouldn't use the fog creek stack. Which was still based on Microsoft stuff anyway.
That said, SO was indeed a complete game changer in terms of the rapidity with which one could get solid information.
It might have been easy to go php > ruby, but C# > ruby meant massive amounts of pain or complety changing your environments. While my company at the time had VMs, they were all windows and it wasn't a case of simply spinning up an EC2 to play. The pain of leaving a MS stack was massive.
I'm using ruby/python a bit interchangeably as I was.playing with both, but even things like gem took a load of effot compared to now. Like a couple of days of fiddling doing nothing interesting, not programming but constantly hitting errors or tutorials not working as you'd got a step earlier in the chain slightly wrong. It was a huge barrier to entry.
The worst tutorials claimed to be for both windows and linux, but would link to an out of date windows installer and all the command line commands would be for linux. I once made the mistake of thinking I'd write little maintenance scripts in Python instead of VB, it was great until I tried to connect to a MSSQL environment, ended up in a world of pain.
Last time I span up node.js on a new windows computer it took me like 10 minutes. Everything has changed.
That said, I see what you're getting at now and it isn't what I thought you were talking about previously. It looks like what you're talking about is it taking a few days (or so) to get a working environment up and running vs. nearly instantaneous now? If so, fair enough I totally agree.
I thought you were talking more about the more nuts & bolts parts, like it taking a few weeks to come up to speed on Python (for instance) instead of several months. I've always felt that claims of it being difficult to change languages or other type things (e.g. changing from Oracle to MySQL) were radically overblown, and these days w/ sites like SO it's pretty much trivial.
These days the landscape is a lot better, especially when it comes to Ruby.
You're correct that ASP.NET MVC was new at the time, but with guys like Scott Hanselmann comparing it to an acoustic guitar production versus WebForms more full-studio approach, and with Phil Haack on the squad, and with the corporate blessing from Microsoft and promises of future integrations, this was not some fly-by-night project that required a lot of faith to invest in. It was a clear case where Microsoft got something right, thanks to a small team of talented devs.
Consider: for the first time in history, you had the ability to write clean C# code in a web context with full control of markup and proper separation of concerns, sans cruft or baggage imposed by attempting to stretch a "desktop" metaphor over an request/response medium. Don't underestimate the psychological effect of this on programmers who'd been spending their days in the trenches builting line-of-business CRUD apps with ASP.NET WebForms and unbridled VIEWSTATE. So to me, ASP.NET MVC was the clear and obvious choice for StackOverflow. If you accept that, what better language than the C# he already knew?
But the specifics are heavily dependent on the actual code executed; to a first approximation, most Rails code is shuffling parameters to and from a database. It's very difficult to speculate. If most of your Ruby code, to take an example, is effectively a wrapper around native code (e.g. regular expressions or string substitution in templates), then the win from using something like C# or Java is probably low.
I think the complexity costs of scaling out are underestimated; or rather, they're just accepted as a fact of the matter, like there isn't an alternative. More moving parts in the name of redundancy etc. usually makes things more fragile, without very careful design and implementation.
Interesting that you bring up GitHub. GitHub was probably the most unreliable service our startup used before we moved to Stash. You could count on it being down for a few hours every month, reason given usually "DDoS". We moved to Stash for contractual reasons relating to security and ISO certification required by our target sector, finance, but I could see us having done it for reliability too.
If they were using a slower language, they'd get much higher latencies (maybe 200-300ms instead of 33ms) that couldn't be fixed no matter how many servers they add. If you read the linked page and Nick Craver's blog carefully, you'll notice that they're deliberately aiming for very low latencies everywhere.
As with all things, you have to measure it for your own application and find your bottlenecks before drawing conclusions.
Beside that, what matters here is not throughput but user-visible latency. And here's where a fast language shines.
If they are capable of rendering any question page in 33ms, it stands to reason they could use just 3 web servers and still serve content in roughly 100ms (which is fast enough that most people wouldn't notice).
My point is, without knowing what is the bottleneck for the web servers (we only know it's not CPU - I suspect it should be I/O) it's hard to make this kind of point.
Aside from that, I think I remember the original development team was already familiar with C#/.NET, and this drove the decision for the original stack.
It doesn't work that way. With less servers, requests would still take 33ms (until the servers are overloaded, at which point they may start to take longer unless they have rate-limiting to reject requests above their capacity), but their overall capacity would be lower.
We know from the provided data that their architecture can compile and serve a page in about 33ms. This includes everything that is not web servers, so we are certain that the actual time spent in the webserver will be lower.
Aside from that, we know that their webservers are nowhere near full CPU usage (they claim 15-20%). This also goes in the direction of not invalidating the use of 33ms as an upper bound. We would need to know more about why they have 9 webservers in the first place to get a better idea of what's going on (mostly how the I/O is behaving, I'd say).
Putting this assumptions together, we can safely assume that by shrinking the number of webservers from 9 to 3 wouldn't cause the page loading time to increase to more than 100ms (and it's likely going to be much smaller in most conditions).
Of course, this may be overlooking some piece of information we may be missing (the prime example still being I/O-related).
edit: I see now that in the original post I stated "roughly" 100ms, while it would have been better to say "at maximum".
You don't need to be CPU-bound before this starts happening because requests don't arrive in evenly spaced intervals. Decreasing CPU load, even on a moderately loaded, I/O bound system, can still reduce latency.
It's not about optimization at all.