On the web server scalability and speed are almost the same thing
antirez.com
antirez.com
However I see a few comments that missed a bit the point, and this is probably my fault because I did not specified some very important thing:
Once you substitute not just an hello world template, but a few templates N times (since you have N comments in the page), the performance starts to be so slow you can't serve more than about 10 requests per second, just because template substitution is so damn slow if you don't preload templates, and the obvious way to do it is calling :erb (you see this in all the Sinatra code around, in the examples, and so forth).
10 requests per second instead of the 1000 you get even with 'N' includes if you perform a faster substitution, or if you preload the templates, or using any other trick, is a huge difference.
In case you want raw rendering speed, you don't choose PHP or Ruby or any other high-level language. You go for straight C/C++ instead -- that's what companies like Google do; that's what Adobe did for Photoshop.com, that's what Yahoo does too for various web services they provide.
We pick PHP, Ruby, Python or whatever because there's a point of diminishing returns. For our apps 10 extra requests per second do NOT matter.
But 1000 reqs per second do matter, but for a real world app, not for a dumb hello world that doesn't do anything -- and even that may be insufficient, as depending on your use-case you may need 100000 requests per second out of a single server.
AND performance on the web == page rendering time, not requests per second. It gets pretty damn hard to have 300ms page loads, and IMHO this is where template rendering could make a difference, but again, not that much.
"you don't choose PHP or Ruby or any other high-level language".
Why? There is no technological limit, the template substitution should be fast, with minor coding effort it can be fast. It's just lame code. We are not speaking of algorithmic code.
Probably language speed is not involved at all, it is just as lame as the framework reloading the template at every page view. And even more lame than that given that even my test loading the template every time was faster.
Databases are much more interesting. You simply can't scale databases up trivially. A N times linear improvement in the code (due to, say, caching) means you don't have to throw N^2 servers (if that's how your database scales) at the problem.
Linear load reductions have superlinear cost reductions for things like databases. Linear load reductions have linear cost reductions for web servers. This is why people don't care so much.
Though yes, it would be nice to see Ruby templating a little bit faster.
In my freshman year (7 years ago) I was an efficiency and premature optimization zealot, despising anything that wasn't written in C/C++.
Then I met a hacker that told me how awesome dynamic language were (in particular Python and Lisp). I told him "but they are some 10-20 times slower than C!". And he replied "Yes, but it takes 10 times less time to code. If the outcome is not fast enough, you can spend the rest of the time optimizing". Since then, Python has been my language of choice, relegating C++ to the optimization of the (few) bottlenecks.
Why is this funny? Because that hacker was antirez :)
"hello", development mode 1620 req/s
erb :index, development mode 1000 req/s
"hello", production mode 1620 req/s
erb :index, production mode 1350 req/s
That said, I think antirez is right that there are lots of opportunities for making ruby faster. I'm particularly hopeful about JRuby's use of invokedynamic and all of the work that is going into Rubinius.
The thing is, once you're able to scale out horizontally then you're back to riding the Moore's Law cost curve. If my ruby app requires 4 machines now, it'll require 2 machines in two years. Should I pay the up-front cost now of writing it in something lower-level? It depends, but in lots of cases probably not.
Btw the PHP code is reloading the template at every request for sure. I pasted the example code I used on the blog.
I guess what I don't understand is: if you're running a site in development, isn't 250 req/s enough? And if you're running in production, do you need to reload the template on each request?
Unless you're just using this as an example indicative of the fact that ruby/rails could surely stand to be faster, in which case I completely agree. As does Ilya Grigorik: http://www.igvita.com/2010/06/07/rails-performance-needs-an-...
I was playing with Sinatra, and my real page involved substituting a few nested templates, with different parts of the site. News box, comments boxes, ...
The result was... 30 requests per second once a few templates started to stack, and with mostly toy code. It is easy how things can get worse just doing a few mistakes along this path.
(BTW I completely agree with antirez's point.)
If the database is the bottleneck then the speed of web page matters much less. A page with a single 50 ms query will take %40 of the hardware (60 ms versus 150 ms).
This is even much less of an issue if you take into account how long it took for his test. PHP served 1500 requests per second vs Ruby's 250 requests per second which is equal to 0.7 ms per page and 4 ms per page. Assuming again you have a single 50 ms database query you are looking at 50.7 ms vs 54 ms which means you will need ~94% as much hardware. This is assuming that the database and webserver are on the same machine.
If one puts them on separate machines then the time of execution does not matter as long as the time it takes to query the database is less than the time it takes to render the page. Now this is bad in terms of page load as 50 ms + 49ms for page rendering is much more than 50 ms + 0.1 ms but in both cases you will be able to serve the same number of requests per second. This of course assumes that this is running in a multi threaded environment which allows one thread to sleep and other threads to start while waiting for a response from the database.
That said, it's a young project, and things can be a bit rocky. Including compilation in your deployment process is also a pain; do not kid yourself about that. It's kind of "industrial strength" in general; unless you care about how many PHP requests you can squeeze out of a unit of hardware, HipHop doesn't have much to offer.
It is as silly as saying anyone programming in a JVM language is just programming in Java, or anyone using a language which compiles to machine code is just using assembler. (implicitly of course!)
obviously it depends on the share of time the one spends tweaking generated code and the compiler's codegen.
Well, actually there are a lot of such examples. For instance consider the idea of caching data in shared memory. This is very fast. But the second you've done it with something that has to remain consistent across requests, you can't have 2 webservers any more. You're fast, but can't scale. (Don't laugh, I know of a number of websites that made this mistake at some point in their history. http://www.kuro5hin.org/ and http://slashdot.org/ are two good examples.)
Concurrency and locking issues provide lots more examples. Having fewer locks over large things is much faster than having lots of little locks. But you get less concurrency. For a toy website, MySQL with MyISAM is fine - lock whole tables. Row level locking as done in Oracle, PostgreSQL or MySQL with INNODB is much slower, but it scales.
I know that you think that this is "just another type of speed", but it really isn't. If a critical lock somewhere deep in a database is operating at 95% of maximum capacity, there is basically no visible effect on performance from the lock contention. At 101% of capacity, your database falls over. The characteristic failure mode isn't that you get slower as the request rate increases. It is that you max out your capacity and then crash if you try to go faster. I've been there, and it isn't fun.
Now with all of this said, there is a large, obvious, connection between speed and scalability. Suppose that you are comparing 2 languages, one of which runs twice as fast as the other. You can scale the slow one - just run twice as many boxes. But now you need twice as many database connections. Holding those connections open consumes resources, so you need a bigger database. And so on.
Therefore the faster environment can frequently push off the point at which you start encountering other scalability issues. But speed and scalability are not at all the same thing.
Step 1: Create two WordPress blogs served by two Apache servers — one with KeepAlive on and one with it off
Step 2: Benchmark the speeds — you'll see that the KeepAlive one is faster
Step 3: Get a link to each blog on Daring Fireball
Step 4: Notice which server is still accessible (hint: not the faster one)
The server with KeepAlive will fulfill requests faster up to a certain number of people within a certain time span, but past that number it will simply start turning people away while the other server keeps delivering pages a little bit more slowly because Apache's KeepAlive trades capacity for speed.
At any rate, in a real production environment, there are measures you can take to make sure your site is both scalable and fast. Toy examples of poor configurations aren't very informative IMO.
Speed at scale and scalability are the same thing, but speed and scalability most certainly aren't. A server that performs really well on a single request, but slows down as requests are added would be considered 'fast', but not 'fast at scale'.
Yes, there are plenty of instances in which this is the case, and most notably, this is a very common affliction with databases. Also, even in scenarios where you can add more web servers to an application stack, you can't necessarily add more database instances, so you scale those UP instead of OUT.
In summary, the statements made in the article are true, but yours aren't really. In the case of web servers, yeah, they're almost the same thing. In the case of other things, they're generally not.
Think about it this way: Speed is the number of requests/second a single server can process.
Scalability is the amount of computing power you lose when splitting a job amongst multiple machines.
Intuitively you can think of it like asymptotic complexity. You can have a really fast implementation of quick sort, but at a large enough input size it will be flattened by a very slow version of radix sort.
Similarly you can have the fastest web server on the planet, but if you scale poorly, at some number of nodes even the slowest albiet well scaling system will out perform you.
Like, if I run a C++ program off a 2X CD-RW drive in a computer with a Motorola 68000 and it's slower than my Ruby site is on my 8-core Mac Pro with an SSD, do I get to write a rant about how slow C++ is?
It doesn't matter if you can serve 10,000 requests per second if the cost of that request exceeds what you're paid for it.
For most sites ruby is profitable, the value add from ruby is decreased cycle time between releases which can increase the profitability of your site more than increasing the pages per second can.
The vast majority of sites don't need to scale beyond a single server, and when you do need to scale I prefer things like drop in support for S3, drop in support for memcached and a whole host of other performance increasing techniques over raw pages per second.
The benchmark benchmarks the very simple case of a single PHP page, what real PHP app have you used that is a single PHP page? The request times for something like WordPress are insane because of the number of PHP files that need to be interpreted per request.
edit: To clarify the jist is that low margin activities where increasing the speed of your page by 5X would have a great impact on your profitability are not the areas where you should focus you efforts. Instead focus on high margin areas where you could write your pages in SnailScript and you'd still be profitable.
2) Regarding "The vast majority of sites don't need to scale beyond a single server," you could make the same point that the vast majority of sites are not profitable either. It's a total non sequitur in either case.
3) Your point that it "costs less" to develop in Ruby which can increase the profitability of your site is an incorrect assertion. It may be true that the time to develop in ruby can decrease your R&D costs, it does not affect your cost of goods sold and has no impact on your margins. If you just say "it costs me $100 less to develop in ruby, thus I've made an extra $100" you need to take an accounting course.
More to the point, if you are building your business with your optimization being focused on decreasing development time, you have already lost sight of the goal.
PHP sucks, but there are a ton of profitable companies that use it because it is wicked wicked fast. In my mind, that makes it suck less. :-)
-David
http://www.investopedia.com/terms/g/grossmargin.asp http://www.investopedia.com/terms/n/net_margin.asp
Lots of software companies have gross margin's north of 60% or even 80% (GOOG, afaik) which is unheard of in other industries.
If you focus your business on growing revenue and having a reasonable gross profit margin, you are focusing on the right knobs. If you are focusing on decreasing R&D as a means of maximizing net profit, you are focused on the wrong things.
I think in this mini benchmark, it's not so much that "PHP" is fast (as a language, it isn't, really), rather that Ruby + its various frameworks are pretty pokey.
In any case though, while I remain a Ruby fan, the grandparent sort of misses the point: by being more efficient, you have a wider range of things that can make you money. If you have to have a huge amount of resources to get some pages up, doing ads or some other low-margin activity might not even be feasible like it would with a faster solution.
If your margin on a page request is 5% your business is probably fucked anyway (unless your serving billions of pageviews per month, there are only a few sites that work on this business model). I'd much rather have a business with orders of magnitude fewer requests and a profit margin on the order of 10s of thousands per page request.
What do you think 37Signals margin on a page request for basecamp is? My guess is at least 10,000%.
That's something I would certainly endorse. I didn't get that out of your post, I completely agree. Engineers who start companies often overlook this fact and focus on the entirely wrong set of problems when building their business.
You could end up having 2011 technology on 2014 servers competing with 1999 tech on 2011 servers.
AMZN and GOOG have proven that pageload speed does in fact
directly relate to profitability.
On the other side, client-side optimization does help your servers: imagine if you cut from 60HTTP requests to just 6—and did that with proper caching policy, so your servers won't be hammered on subsequent page loads just to answer "304 Not Modified".
Ever since I got interested in client-side optimization (a bit over two years now) I am amazed how neglected this aspect is. I may be biased, of course.
I do agree with your point though. Decreasing development time is the means to the end of making more money out of the product and making a greater return for your (money/time) investment.
[1] http://www.watchingwebsites.com/archives/proof-that-speeding... [2] http://googlewebmastercentral.blogspot.com/2010/04/using-sit...
The study you link to shows a decrease of in rev of 5% by adding 2 seconds of load time vs. 50 ms. There are very few businesses whose costs are dominated by servers. Most spend more on a single developer than servers. Yes, Google, Twitter, and Facebook could probably do well by doubling their requests per second but the average companies costs are dominated by employees, that why it's called Ramen profitable and not EC2 Micro profitable.
But website speed is one of those things (among many others!) that has a measurable impact on your business. This isn't just a nerdy pissing contest.
On the other side, profitable or not, users don't like to wait that 200 milliseconds more because the code is not written in the right way.