Apache 2.4 Faster Than Nginx? | Hacker News Reader

Apache 2.4 Faster Than Nginx?

blog.zhuzhaoyuan.com

87 pointsfreestyler43 comments

Apache is faster than NGINX. NGINX is faster than Apache. If that confuses you, then you don't understand http daemons. It's all about how you benchmark.

This particular benchmark is 100% pointless and tells you nothing. He was performing the benchmark on one machine. In other words, all was local. You are never going to do that in production. What would be the point? In production, you have this thing called a network connection. It is always the limiting factor.

Somebody do a benchmark with the client and server thousands of miles apart. And tell me how much RAM and CPU is used by the daemons during the test! I am 100% sure both daemons can saturate the pipe. The relevant part is how they do that.

Yeah... I stated that in a previous comment but the whole software benchmarking news are often bullshit. I mean just look at the setup:

_ Single machine

_ Static files (seriously ?)

_ Incomplete or even incorrect configuration of Apache, I don't know for Nginx

But you know what ? That's always the same thing, people prefer to look at benchmarks, sit back in their chairs and have this immense satisfaction of not having started a project with that "slow/under-performing tool that is Apache, because Nginx is so much better". Whatever... It's really tiring.

If you were to build a real website, time delivery for static files you simply... do not care! And anyway that amount of data per sec, that rarely met. And if it's met, you rent a CDN for static content.

But as they say : "Benchmarkers are going to benchmark" (even if it's useless)

Somebody do a benchmark with the client and server thousands of miles apart.

That is not the right way to load test. You're not trying to determine how many pages a single client can request from a server. You're trying to isolate the performance variable. To do that, you test from a machine on the same network. The resulting values are how many requests the server can handle in a given period of time, regardless of how those requests get to the server.

> That is not the right way to load test.

False.

> You're not trying to determine how many pages a single client can request from a server.

True.

The reason you benchmark a server with a client from a thousand miles away is that you want to know what effect common Internet noise and delays will have on RAM and CPU consumption for the http daemon.

Conditions on a LAN are pretty close to ideal. Your router won't drop packets and will perform fairly well. That isn't anywhere close to true on the Internet as a whole. You do not want to push an http daemon to production that works like a champ on your LAN and has a meltdown in the real world.

You want to benchmark under as close to production conditions as you can. Period.

Hahaha, holy jesus, I didn't realize he was running ApacheBench and the httpds on the same machine. That's insane just from a local resource starvation standpoint.

Whatever the overhead of the benchmark tool, it would be the same for Apache and nginx, presumably, so can be factored out.

How much CPU/memory is ApacheBench using? Is Apache using more CPU/memory than Nginx? Would Apache have had better performance if its resources weren't limited? Was ApacheBench able to perform as fast as possible?

It is good not to presume anything when dealing with technology.

Note in captions: (static file).

He might as well have benchmarked Apache against an FTP server.

Hitting the same HTML page/file over and over is not realistic. There is so much more to a web-server than this.

What you absolutely must do is benchmark a dynamic PHP-driven, with MySQL access, website. Because that's what Apache is mostly used for (rarely for file serving).

And I don't mean the first page only. But the entire website using a stress profile that mimics real world usage (including writing to the database).

Then you'll get results that mean something.

As it is right now, this test has only proven that Nginx can handle more concurrent requests for a single static file vs. Apache 2.4 (which is something we all already know to be true).

"Note in captions: (static file)... This is not realistic."

It's reasonable if you're trying to test concurrent connection limits.

"What you absolutely must do is benchmark a dynamic PHP-driven, with MySQL access, website."

That's something nginx is also used for, and it's far from the only thing apache is used for. I agree that would be a useful benchmark, but it's not the only meaningful one.

"you've only proven that Nginx can handle more concurrent requests for a single static file vs. Apache 2.4 (which is something we all already know to be true)."

No, we didn't know this. This was testing apache 2.4's new event [0] MPM [1]. Arguably we still don't know this, since this apache configuration seems artificially limited and there's no comparison of the resource usage of apache and nginx.

[0] http://httpd.apache.org/docs/2.4/mod/event.html [1] http://httpd.apache.org/docs/2.4/mpm.html

You are dead on. If you're trying to determine which httpd is the fastest, you test it on static files. Take the dynamic part of a website out of the comparison. If you want to test something dynamic, make the test pull from many different files so that you can avoid caching.

Next, if you want to test the PHP interfaces, you can do that, but this test setup was perfectly adequate for testing the speed of the daemon. (once the configuration issues are handled)

Your testing methodology will fail horribly the moment the core component fails to produce results on real-world use cases, or scale up with the included modules that are required for it to even be called a modern day web-server.

You eventually have to test the whole, not the part, unless you want to see benchmarks for useless/no-context metrics.

On the other hand I do realize its fun to see tests like this, and they can even give you a bit of insight... For example, that Apache now comes close to competing with Nginx on static loads, especially for larger files (9KB vs 54KB).

I dunno, we separate out static file vs. needs-to-go-to-Tomcat requests at our HAProxy so I'm actually interested in what simply performs best serving static files. I can't imagine we're the only ones to have this need.

This. Good grief, what a leap of logic this entire benchmark is.

why? just because his use case is different from yours? some people use apache for fronting dynamic requests, and nginx for static serving. It would be interesting to see if apache has become competitive at serving static resources and I could drop nginx from my servers.

Good point, but I don't think people only ever use Apache for PHP and MySQL.

I'm certain they don't, but it doesn't seem like a terrible lowest common denominator to me - certainly it'd be a better one than pure static file serving from the point of view of the workloads I normally deal with in production.

I'd like to know where the reports are that Apache 2.4 was faster than Nginx. The test in the post hardly seems exhaustive; most web servers do more than serve up a 9kb file on the loopback address.

I have to imagine, that the actual 'report' was that 'apache 2.4 is faster than nginx in certain situations'. This would certainly seem to back that up: http://mondotech.blogspot.com/2012/02/apache-24-vs-nginx-ben...

"We also show that as far as true performance is based - real-world performance as seen by the end-user- 2.4 is as fast, and even faster than some of the servers who may be "better" known as being "fast", like nginx," Jagielski said.

Jim Jagielski is the ASF President

Source: http://www.serverwatch.com/server-news/apache-2.4-delivers-m...

Agreed. It's one thing to say "x is faster than y" wholesale rather than considering the endless possible configuration options for each that could affect performance.

Seems like it'd be a lot more constructive to say, "Hey, it's not true out of the box, but here are settings you can fiddle with that, if they work in your situation, could make Apache a better option than Nginx." Anybody want to write that blog post? :)

Well… someone should!

I'm frankly really tired of operating on cargo cult programming when it comes to my server packages. They're totally opaque to me, since they're huge pieces of software.

Is MongoDB fast? What about Riak or Couch or whatever? What does "fast" mean?

I want charts, damnit, and comparison tables with feature matrices and stuff that can used toe evaluate and, hell, I want you to put up vagrant images so I can run these tests myself.

The disclaimer on that benchmark is beautiful - I very much appreciate both the degree of honesty about the applicability of the results and the reminder that the majority of benchmarks we as developers personally run suffer from exactly the same caveats.

I look forward to the results of your testing!

Very happy the author wrote this up in a succinct way, posting the configuration files and all. Other comments point out that there can be a lot of other circumstances in which these results are not indicative (network time, large files, etc) but it needs to be said that the author did the tests, and shared the results with the community. Thanks!

This benchmark was useless.

What's important to realize about Apache 2.4 is it's probably the first version that's really C10K ready and that's a great milestone since it maintains httpd.conf and .htaccess compatibility.

Here are the relevant Apache configuration directives for this sort of benchmark of the event or worker MPM.

ServerLimit: In combination with ThreadLimit, this sets the maximum configured value for MaxRequestWorkers. Default is 16.

ThreadLimit: The maximum configured value for ThreadsPerChild for the lifetime of the Apache httpd process. Any attempts to change this directive during a restart will be ignored, but ThreadsPerChild can be modified during a restart up to the value of this directive. Default is 64.

ThreadsPerChild: The number of threads created by each child process. The child creates these threads at startup and never creates more. Default is 25.

MaxRequestWorkers: The limit on the number of simultaneous requests that will be served. Any connection attempts over the MaxRequestWorkers limit will normally be queued, up to a number based on the ListenBacklog directive. Default value is ServerLimit multiplied by the value of ThreadsPerChild, e.g. 400.

ListenBacklog: Maximum length of the queue of pending connections. Default varies by OS, on linux I think it can't be larger than /proc/sys/net/core/somaxconn

AsyncRequestWorkerFactor: Tunes the event MPM's concurrent connections per process. See http://httpd.apache.org/docs/2.4/mod/event.html Default is 2.

"Faster on localhost" doesn't always translate to faster under real-world conditions, of course. If your actual workload is serving largish files to lots of clients relatively far away, this kind of test setup is essentially meaningless. I'm interested to see how the Event MPM helps Apache behave more like nginx in that kind of situation, might even asymptotically approach nginx's performance when transfer time is the rate-limiting factor instead of the server's CPU, for example.

Considering the web server takes less than 5% of the execution time in my stack, I'm not bothered and 99.99% of you probably shouldn't be either.

It'd be a better performance improvement to rewrite the stack entirely in C than change the web server and that's just illogical.

Benchmarking over loopback on your laptop is like the #1 benchmarking anti-pattern. It is, at best, accurate to within an order of magnitude.

People use Apache for its modules, and because it's nicely integrated with Wordpress (and there's probably other similar reasons). People use nginx because it's easy to configure, and plays nice with async backends (and there's probably other similar reasons).

No-one (who can't afford to do their own testing) picks a webserver purely for speed.

Actually, we use nginx for its modules. Or rather, openresty, since they are kind enough to compile them all in.

The article fails to live up to the expectation set by the headline. First, it's just a simple experiment, which the author later found to lead to the wrong conclusion. Second, it in order for it to really be about Apache 2.4, it would need to contain an earlier version (most certainly Apache 2.2) as a control. Otherwise it's just about the current stable versions of Apache and nginx.

Another problem is that the author just went with the slug WordPress proposed, which doesn't contain the question mark. This could be an oversight, but to any linkbaiters who use this tactic: I'm on to you!

Oh hey, look. He set MaxRequestWorkers to 800.

That means Apache can only process 800 connections at a time.

I wonder if that will affect its performance in the benchmark.

Throw in a big keepalive and on a short test you can make it so it only processes 800 requests! Awesome.

Lets try this with the worker mpm instead.

The event MPM (new in Apache 2.4) is designed to address this:

This MPM tries to fix the 'keep alive problem' in HTTP. After a client completes the first request, the client can keep the connection open, and send further requests using the same socket. This can save signifigant overhead in creating TCP connections. However, Apache HTTP Server traditionally keeps an entire child process/thread waiting for data from the client, which brings its own disadvantages. To solve this problem, this MPM uses a dedicated thread to handle both the Listening sockets, all sockets that are in a Keep Alive state, and sockets where the handler and protocol filters have done their work and the only remaining thing to do is send the data to the client.

http://httpd.apache.org/docs/2.4/mod/event.html

Hell, let's get crazy and use event. I hear asynchronous connection handling is popular these days. http://httpd.apache.org/docs/2.4/mod/event.html

heh, something tells me this guy will be updating his post based on all these comments. At least, I hope so.

Like spending hours digging the doc of apache just to compare it the default nginx installation ? My guess is that he will just forget apache and use nginx with default config in production and read the doc after he has a problem with it.

I would like to see this with the worker mpm on.

And the blog has been HN;FP'd! I have to agree with people above me tho, the benchmarks are less then useful regarding more real-life uses but regardeless of that, he still spent time doing them we have to say thanks for atleast that :)

"lies, damned lies and benchmarks" springs to mind.

my apache is not your apache etc.

I'd like to see how Apache 2.4 benchmarks against previous versions of Apache.

http://i.intma.in/images/2e27787dfaf5c000ae9a9a2ea859b6b3.pn...

yeah, this is a web guy I'm totally going to listen to

Those graphs make my colour blind eyes cry.

Reply on news.ycombinator.com