The Dream Stack - Is This A Joke?
dobbscodetalk.com
dobbscodetalk.com
* It must be really fast
* It must be really safe
* It must be really cheap
* I must be able to do other activities while using it
* It must be ecological - no pollution whatsoever!
I can't tell you where I'm going (next door or to the other side of the world), but I certainly intend to move from my chair at some point, so I think it's really important that I do so in the best way possible. I might take some other people with me as well, please bear that in mind in your recommendation.
So? Anyone? Hello?
For instance: Hybrid car
- Pretty fast relative to other transport
- Not bad safety but not the best
- Pretty affordable relatively
- Has a radio etc.
- It does better ecologically than most.
It's not going to be a perfect answer, but having those 5 questions you just posted lets me know a) what's important to you (I might not care about ecological qualities for instance) and b) lets me evaluate why I might choose a particular method.
It's exactly analagous to the idea that you can't truly shred until you've assembled the right set of effects pedals, preamps, niobium-plated strings, vintage depleted-uranium core wound pickups and ocelot-tooth guitar pick.
Just play the damn thing.
But we each still like to think about that perfect Gibson Les Paul Custom we would have if we were a rockstar..
The painful truth about web services is that it doesn't matter what tools you use; at least initially. If it does something useful and you are getting revenue you can and will rearchitect and rework the solution as it grows. And as a system grows it tends to become heterogeneous, an all php shop will suddenly sprout a java based search engine, an ASP.NET shop will throw a linux box in front of it's app servers to act as a reverse-proxy filter/load balancer, and so on. The really good thing about the current state of the art is that you have a multitude of tools available to solve any given problem. The issue is you need to have people who have experience with more than one toolset to make sensible decisions and be able to pick the right solution for the task at hand.
Its better being adaptable and rolling with the demands than being stuck with a problem you didn't predict that you can't solve without major surgery because everything else is so strictly defined.
To non-programmers like the author, this sort of article is reasonable. Programming is not a creative art, it's just about buying some tools that you read about in the trade magazine.
First, when you have an average level of programmer (as the sort of corporations that would hire a senior analyst tend to), selecting the right tools is where you can innovate against your competition.
Second, if this wasn't written by a non-programmer, but instead was an Ask-YC "What stack would you use to build the next Google/Facebook/Amazon etc." I don't think this same criticism would be levelled at the author.
Honestly, I think we should feed "analysts" as much bad data as possible so that the companies that rely on analysts are killed off. The analysts will then die too :) (Have you ever poisoned someone to cure their cold? This would be like that. Kill the host, then the virus dies :)
I had to re-read the article (and the above paragraph) several times, and check other articles on the site to make sure I wasn't reading a parody site.
Engineering a web scale service for robustness is a series of tradeoffs, increasingly expensive tradeoffs as you make the system proof against bigger threats. Geographical redundancy alone more than doubles your costs, and (if you're doing it right) only adds a third or fourth 9 to your availability numbers.
But "Architected to never go down." does not guide the implementer, it doesn't communicate anything they need to know about what the limits of system are; and most importantly it ignores the reality principle.
The fact that you are already bringing up all your expertise on redundancy shows that you can think about this better than others and probably give an answer that fits this requirement well.
Unless we completely reinterpret the requirements (as per your suggestion), (s)he really can't. It is impossible to meet the requirement "Architected never to go down."
What actually confuses me more than anything is how wildly the strength of the writer's connection to reality varies. I mean, two of the "requirements" are:
# Architected to never go down
# Employs security features
Eh, I dunno.
If I asked you to recommend a car that is fast and safe you wouldn't reply "all cars are fast" and "no car can be perfectly safe", you would suggest a car that maximises those requirements.
Similarly, the author is looking for what frameworks/stacks maximise (among other things) reliability and security.
Incidentally I'm a fan of the ASP.NET MVC stack (using MS SQL) for web development, but I don't think it's the most scalable and is unlikely to have a large pool of good developers to source from (you would probably want a good C# or non-MS-hating RoR developer).
Yeah, why solve problems that you actually have when you can fantasize about solving problems you don't have instead?
However, the whole topic of this thread is fantasising about the perfect stack for that massive site. Besides which, 30 seconds now could save one year in the future (just think if Netscape had thought ahead).
Add one or two orders of magnitude, then you're talking about scalability.
Moreover, a hit is a rather arbitrary unit of measure. 11 hits per second to the Twitter API is a lot different than 11 hits per second to a static site like example.com.
There's not much point even taking that into account for a read-only affair like SO at such a small scale because the cache is doing the work here and that won't sweat even under 100 or 500 reqs/sec.
Anyways, you point out correctly that discussing scaling-issues is rather fruitless without taking all variables into account. I was just trying to say that "SO (in particular) is handling 1mio reqs/day" does not tell much about the performance of the underlying stack.