Stackoverflow's WISC (Windows, IIS, SQL Server, C#) Architecture
artificialignorance.net
artificialignorance.net
Yes, StackOverflow is a nice success story for a .NET stack, but I think an important piece here is actually mentioned by Joel:
"We are using about 1/10th of the hardware that they are, unfortunately, and maybe that’s because they are not good programmers"
StackOverflow says little about "the power of WISC" and more about Attwood and Spolsky being good, experienced Microsoft-tech developers. We cannot compare the power of WISC vs the power of LAMP (for example) when performance and productivity are metrics. It's hard enough to compare each one on its own, but even harder to mix the two. The purported value is thus a red herring. Compare SQL Server to MySQL or C# to PHP, and then we'll have a discussion.
"Can you beat phenomenal performance, reduced CPU usage and great productivity which now comes at no cost up-front?" I can copy and paste that into any flamewar thread on any side of the argument and it will still make sense and you will still want to agree. The reader here agrees with the question, and never with the thesis---as much of a thesis there can be in this post, which is only implied. An unbiased question to be asked should be, free licenses or not, is WISC worth it? Well, no:
* Windows -- Yuck. The only reason to ever want to develop on Windows is if you are using the AllegroCL IDE[1]. Windows as a server? Is there ever any argument there, or is just acquiescence due to the rest of the stack?
* IIS -- Seriously? Have you ever had to manage a farm of servers all running IIS? There may be tools out there to automate IIS maintenance across servers, but do they beat good ol' text config files?
* SQL Server -- Fine, it may have its merits without needing 10 years of experience, so again, where's the SQL Server vs MySQL practical comparison?
* C# -- Mixed feelings. If you're stuck with the above three, not too terrible of a choice if you know the language well and are in a hurry. Else, learn Boo.
Pardon my sarcasm, but there really is no point in belaboring the merits of WISC in terms of performance or productivity. It has its place, and its values, but this leads to a different discussion.
But here's kind of what I felt[2] when I was developing on the MS stack at my previous job:
"I once saw a nature special in which some insect or other dragged some other dead insect somewhere then turned around and dug a whole for it to bury it but the researchers moved the dead insect a bit while it was digging so it had to drag it back but while it did they filled in the hole and back and forth this insect went indefinitely until a PETA sniper took out the researchers."[3]
[1] The AllegroCL IDE is only "full-fledged" on Windows.
[2] After taking the red pill, of course.
[3] Kenny Tilton on comp.lang.lisp
Here is one: http://angryhacker.com/blog/archive/0001/01/01/mysql-eye-for...
Also point 3 about MSSQL defaults has been moot at least since SQL Server 2000. While it does create a named constraint in the background, it supports a as minimalistic syntax as MySQL for setting defaults.
In fact, this is such a widely used syntax that you almost have to look up the overly declarative one he uses in his example to know that it even exists.
Besides that your link does give at least a light overview of the differences when it comes to programming them, so have an upvote.
Windows -- Yuck
Some people like Windows. They develop on Windows. When they want things deployed, they might prefer a server-architecture which they already have intimate knowledge of, rather than some Linuxy thing which does have its different set of rough edges.
IIS -- Seriously? ... IIS maintenance across servers, but do they beat good ol' text config files?
IIS has since IIS7 had all its configuration in text/XML-files. While .NET on IIS5-6 only allowed you to move some settings from IIS metabase to an actual config-file in the site-root, IIS7 and newer allow pretty much complete configuration of any relevant setting, either trough this file or trough applicationHost.config.
Both are text-files. I'm forced to question how long it has been since you messed around with IIS.
C# -- Mixed feelings
C# as a language, while (IMO) not complete is still a very powerful, rich and expressive language. It does have its lackings, but you are writing your Windows/IIS hosted websites in .NET code and may as you please use any other .NET language.
Personally I'm getting more and more fond of F#, although I admit I haven't used it for web-development yet.
While your criticism about lack of direct comparisons is valid, the rest of your post reeks of a anti-Windows sentiment which has little basis in merit and entirely in preference. I'm surprised your comment is so highly upvoted.
Regarding IIS, though, maybe it was just bad experience with a poor IT department. They would go into each server and manually reboot IIS, instead of writing a batch file or something to iisreset xx.xx.xx.xx for every server. Not to mention the fact SourceSafe was used for deployment. (And yes, my experience was before IIS7 which I only used briefly.)
I give C# credit and don't hate it entirely---it's probably more the paradigm (statically typed, class-driven OOP) which I have an issue with.
SQL Server starts free and it was actually only halfway through last year that I stopped using the free version at Playtomic. It's also the only significantly expensive part - and very easy to not use in favor of something that is free/open source/cheaper/whatever.
I'm glad I went with .NET for my startup - a few days ago I calculated some current numbers: 29 million unique people logged over 3 billion events during a 10 day period. It's holding up well enough for me and my very lean, mostly Microsoft-flavored startup.
Yesterday I came across this writeup on what Quora is built on too:
How much does BizSpark cost after the initial pricing period lapses?
Startups may take advantage of the BizSpark Graduation offer that allows startups to keep the software they have with the option to purchase subscriptions and maintenance if they want to keep it up to date. This also allows them to continue using their Microsoft software at no cost in perpetuity and allows them to renew subscriptions to this software without paying any initial license fees. Startups can:
•Keep their MSDN Ultimate software and optionally renew their MSDN subscriptions at any level
•Keep all their Windows and SQL Servers they are using in production and optionally purchase the first two years of Software Assurance (updates and ongoing maintenance) at a 50% discount
It's remarkably friendly although you could paint yourself into a corner if you went crazy with SQL Server, but there are a lot of options these days for dealing with large volumes of data.
At the end of 3 years the costs are hopefully going to be incidental.
"(...)When I compare our performance to similar sites that are running on the open source stack, we are using about 1/10th of the hardware that they are, unfortunately, and maybe that’s because they are not good programmers. But just in terms of the types of queries we are doing and stuff, the Microsoft stack is actually, appears to be paying for itself – in terms of reduced hardware”
Doesn't really provide any real justifications. I have indeed anecdotally heard good things about mssql, but apart from that I have a hard time believing you couldn't get more or less identical performance from jvm/linux/etc (and maybe mysql/postgres really can give mssql a run for its money, but I admittedly know little about relative database performance)
On the other hand, it seems like Jeff/Joel had lots of experience with the MS stack before, so its not an illogical choice. You want to spend your time building a product, not wasting it figuring a new stack you aren't very familiar with.
The difference in performance per hardware dollar for a .NET setup and a Linux type setup can be quite large. Especially if you compare an ASP.NET site to a PHP site. The .NET stuff is going to be compiled, or at least pre-parsed, and likely jitted. To get comparable performance out of PHP, you'd need to use APC or Zend or something similar. For small sites the difference may not be all that large. But if you're using something like CakePHP, there can be a good amount of stuff for the server to parse on every single request.
Also a dead simple dumb LAMP setup is going to use Apache prefork, and won't have nearly as good scalability for the same amount of memory as an ASP.NET setup would. A naive Linux PHP setup is actually more like what the ancient, pre-.NET world of ASP looked like.
So in short, ASP.NET is going to blow the doors off of a dead simple LAMP setup if you have a huge amount of code and don't use Zend or APC. A more modern setup under Linux would do much better of course, say Python and WSGI use Apache-worker. This setup is going to give much better performance (due to not having to precompilation) and make better use of memory (through threads in apache-worker vs what you get with apache-prefork).
Microsoft's technologies come out of the box building higher performance apps in the default configuration than some of the more naive pure-Linux approaches.
Hitting the database bottleneck seems to be the most painful scaling challenge, and compiled code doesn't really help you there.
I'd be curious to see how a limited number of IIS servers running compiled code compares in price to simply upping the number of dynos on Heroku.
"Doc, uh, my brother's crazy; he thinks he's a chicken."
And, uh, the doctor says, "Well, why don't you turn him in?"
The guy says, "I would, but I need the eggs."
Even if you tend to believe MS-SQL is the best database out there, there is no way, coding/testing/distributing your layers (and trust me, there are plenty of them at SO) in C#/Windows is better than {Python||Ruby}/Posix.
This is my opinion, based on my experience from smaller scale web apps, if at small scale the gap is huge, when things grows to millions of page-views a day, the gap is far larger.
Just try to imagine friend-feed, reddit, facebook, justin-tv or any other large website, you name it, scaling and pushing versions on top of MS platform.
Personally, I can't imagine doing the scaling, but I don't know the platform at all. SO has obviously figured out how to scale well, though.
But that was said as a joke, talking seriously, there is no doubt, the web is powered by the open source software.
Most of the major sites, are using the tools which are not made at Redmond. There are however, some large sites which are exceptionally using the WISC stack, but amongst the largest sites, take the top 100, top 1000, you name it, they are the minority.
root@clonezilla-srv:~# wget -O- --server-response --spider http://www.myspace.com/
Spider mode enabled. Check if remote file exists.
--2011-02-02 13:25:18-- http://www.myspace.com/
Resolving www.myspace.com... 216.178.39.11
Connecting to www.myspace.com|216.178.39.11|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Cache-Control: no-cache, must-revalidate, proxy-revalidate
Pragma: no-cache
Content-Length: 74124
Content-Type: text/html; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.5
X-Server: 896c6f7da640823d4508c54f3b48fc12058736fbc4e59bcf
Set-Cookie: MSCulture=IP=81.218.172.217&IPCulture=he-IL&PreferredCulture=en-US&PreferredCulturePending=&Country=SUw=&ForcedExpiration=634322139193426057&timeZone=0&myStuffDma=&myStuffMarket=&USRLOC=QXJlYUNvZGU9MCZDaXR5PUtmYXIgU2FiYSZDb3VudHJ5Q29kZT1JTCZDb3VudHJ5TmFtZT1Jc3JhZWwmRG1hQ29kZT0wJkxhdGl0dWRlPTMyLjE4MTcmTG9uZ2l0dWRlPTM0LjkzNTgmUG9zdGFsQ29kZT0mUmVnaW9uTmFtZT0wMiZMb2NhdGlvbklkPTA=&UserFirstVisit=1; domain=.myspace.com; expires=Wed, 09-Feb-2011 11:25:19 GMT; path=/
Set-Cookie: SessionDDF2=8Q/cYHD2nDXwG4IfpqngrVTCZxMtlFsPs8sS2KKXpeayPKeoIq6z3xJLZvQ7dssRE48xuBfthlzlt6O7FCHJxQ==; domain=.myspace.com; expires=Sun, 02-Feb-2031 11:25:19 GMT; path=/
Set-Cookie: MYUSERINFO=; domain=.myspace.com; expires=Wed, 19-Jan-2005 08:28:17 GMT; path=/
Set-Cookie: MYUSERINFO=; domain=myspace.com; expires=Wed, 19-Jan-2005 08:28:17 GMT; path=/
Set-Cookie: USER=; domain=.myspace.com; expires=Wed, 19-Jan-2005 08:28:17 GMT; path=/
Set-Cookie: USER=; domain=myspace.com; expires=Wed, 19-Jan-2005 08:28:17 GMT; path=/
Set-Cookie: SECUREUSERAUTH=; domain=.myspace.com; expires=Wed, 19-Jan-2005 08:28:17 GMT; path=/; secure
Set-Cookie: SECUREUSERAUTH=; domain=myspace.com; expires=Wed, 19-Jan-2005 08:28:17 GMT; path=/; secure
X-AspNet-Version: 4.0.30319
X-PoweredBy: Pimple Faced Geeks
Date: Wed, 02 Feb 2011 11:25:18 GMT
Connection: keep-alive
X-Vertical: globalsites
Length: 74124 (72K) [text/html]
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.If what is being said in this post is correct (and my personal anecdotal evidence suggests it is) than Microsoft needs to hire the best Ruby and Python developers they can find. Have a program written in both languages on a Linux stack and then write the same program on a Windows stack. Then open source it all so anyone who knows ruby or python can point to code mistakes that might inhibit performance.
Once that's all done MS should stress test all the solutions and publish the results.
The Razor engine is the most credible thing MS has given the development community in a while so there's no time like the present.
This is lose-lose for MS. If it turns out they do worse or the same then they lose. If they do better, even much better, people just say its MS funded, please ignore.
And for any non-trivial app there are 1000! different ways to build it. Even if they happened to find the optimal non-MS implementation, someone will say it isn't. And unless you implement all 1000! of them you'll simply have people saying, "No one would do it that way".
But you did get me thinking and I wouldn't see anything wrong with them having someone else do the actual tests. Hand it off to CNet, ZDNet, AOL or whoever. Provide them the funding and let them run the tests. I don't think any big media company would say no.
Jan 31, 2011 Microsoft Corp.'s net income for the latest quarter fell slightly from a year ago but the software giant still beat Wall Street's expectations despite the weak personal computer market. Net income was $6.63 billion, compared with $6.66 billion in the same period last year.
I don't think MS is losing to much sleep over the never-ending discussion between programmers whether the MS stack is as scalable as open source alternatives.
There are several live sites that prove it can scale. Most sites chose the alternative. At the end of the day its an interesting conversation, but why do people care so much? If I choose to give a couple grand of MY money to MS, why does that bother some people so much?
I think its doable, but it can't be done by MS, Google, IBM, etc...
In my view, the more status-conscious the sales target is, the more easily MS can sell to them. It's not really the tech that seals the deal.
C# is certainly faster than a Python or Ruby but you pay a price in development speed (or perhaps Joel would argue not - if you're a C# expert but Ruby newb you'll probably go faster with what you know).
But you could use Clojure or Java if that's a concern.
Due to the easy, abundant libs, I would still go to Python for hacking something together. I would probably seriously consider C#, though, for something more substantial.
If you're executing jobs that calculate things or other batch processes, you can compile them to C/C++ pretty trivially and have a 50x speedup, depending on your loads. Plus, PyPy is getting really, really good.
However, I am as biased as it gets, so take this with a grain of salt.
But it does appear C# REPL is coming in C# 5.0.
More people(on .NET & otherwise) should look at F#.
"This entire site is serving 16million pages a month and we’re doing it off of 2 servers, which are almost completely unloaded. So we’ve got a ton of headroom on those 2 servers – one server is a webserver, the other server is running Microsoft SQL Server 2008 and they’re both 8 core Xeons."
They have only one web server and one database server? Does Joel not understand the concept of "single point of failure"?
However Jeff/Joel (from listening to their podcast when they did it) seemed to have a massive black spot when it comes to some areas of web scaling that I would have thought were trivial. e.g. saying it was extremely hard to scale beyond a single webserver as it held session state (they seemed to have not heard of memcache)