Pancake HTTP Server faster than nginx running PHP
pancakehttp.net
pancakehttp.net
So...it's written in PHP, so it loads up a PHP application and serves it faster than running through a real webserver to a PHP backend via FCGI. Duh?
Now go ahead and add in high concurrency and static asset serving and add PHP-APC to the workers and don't benchmark on an OpenVZ VM (which doesn't guarantee resources!) and let's see how things shake out.
Pancake seems to be touting this "CodeCache", which is great, except that PHP already has APC, which does this already, and I'll wager it does it better. This smells like a false benchmark, where Pancake wins because it's avoiding work that the poorly-configured nginx stack has to do. If you were to tune both of them to their maximum potential, I would be shocked if nginx didn't eat pancake for breakfast.
I have to admit that benchmarking on an OpenVZ VM probably isn't the best idea. However, at the time of benchmarking, my VM was the only one on the host node and I've made sure that no other service on my VM could create high load.
I believe that CodeCache is more efficient than APC and easier to use. In a future version I will also improve the CodeCache, allowing more code to be cached than now possible. I am also planning on having a feature for automatic configuration of the CodeCache.
Results are here: https://docs.google.com/spreadsheet/ccc?key=0Aqxu-bQf_3O5dFd...
tl;dr: Pancake falls over completely at higher concurrencies. It even starts dropping requests at -c2, and it just gets worse as concurrency gets higher. I even tried running it with 16 workers rather than 4 (which should cover up through -c9 easily), and it was still failing to return valid results. Pancake's throughput goes up as concurrency goes up, but it's because it's just dropping requests rather than serving them.
mod_php easily saturates my two cores with -c2 and remains consistent at ~285 req/sec. Pancake never really achieves better than 90 req/sec, and completely fails to handle high concurrency.
Remember, this is Apache and mod_php, which is renowned for being the slow and creaky way to run PHP. If I were to set up nginx and php-fpm, I'm pretty sure I could push that chart even further.
I call bull on "faster than nginx".
Even though requests won't fail anymore with the fixed configuration, throughput on high concurrency still isn't quite satisfying. This seems to me like a bug as the PHPWorkers seem to process the requests perfectly concurrent and finish each in about the same time, but then the delivery of the result to the client is somehow delayed, probably due to a problem with the internal communication. I've tested Pancake with php-fpm (Pancake supports FastCGI) and high concurrency and it seemed to scale quite good.
I'm investigating on this problem and hope to provide a fix soon.
Regarding static file throughput, I know that Pancake does not really reach the speed of other webservers here. Pancake was designed to execute PHP as fast as possible and I still have a lot of ideas in my mind on how I could further improve PHP execution speed. PHP itself isn't quite fast and therefore it is difficult to develop a webserver in PHP that reaches the same speed when serving static files as webservers written in languages like C. I am currently experimenting with a new feature that will hopefully increase the speed of static file serving but I still don't know if it'll really work.
I fixed the backlog setting and re-ran the test, and pancake doesn't drop requests anymore, but it's not improved at all on speed.
https://gist.github.com/4063785
I was watching top during each test, as well, to get a sense of what was happening; pancake was pegging the CPU at close to 100% user and reached a load average of ~14, whereas the mod_php test was something like 85% user/15% system and reached a load average of 5. This would seem to indicate to me that Pancake is handling something that Apache/mod_php is able to hand off to the kernel, and since it's stuck in userspace, processes end up waiting around for their turn to do stuff.
90 req/sec really isn't bad - especially considering that without APC, my install gets ~35 req/sec - but given that it seems to get most of its boost from avoiding re-evaluation of the source files, and given that APC does the same, I don't think it's really all that fair to compare Pancake to a PHP install not running APC, especially since Pancake requires a specially-compiled version of PHP and has to be run as root, so anyone that can run Pancake can install and use APC. Additionally, APC transparently works across all your source files already - it has no manifests to maintain, so turning it on is literally as easy as "pecl install apc" and enabling the extension in your config. That's a difficult combination to beat. If you can do it, I'd love to see the results, but given what I know of the architecture of these various pieces, I'm skeptical that it can be done with pure PHP.
Finally, I think that trying to contend with nginx from a marketing standpoint is silly, because web applications are composed of much more than just PHP files; I have to be able to serve images, stylesheets, Javascript, and the like for a full web application, so I have to consider the performance of the webserver as a whole in terms of its ability to fully deliver a page to a client to evaluate it properly; a webserver that runs the script end of things, but then struggles to serve up my assets (and even worse, blocks workers serving assets while I have script requests pending) isn't going to be very useful in a practical scenario. I performed my benchmarks with my development Apache install; I can tell you from my own experience that nginx will trounce it solidly, so the gap to overcome is almost certainly substantially wider than in my benchmarks. Considered as a whole, from first-byte to request-fully-delivered, I don't think you're going to be able to approach nginx, let alone beat it, without moving major parts of the application out of PHP and into a lower-level system.
Ideally, a benchmark that produces a chart of response time (Y) against concurrency (X) for each product would be great, because "serves one user really fast" is practically worthless for real world purposes. I want to know what the point is at which each server buckles.
http://pancakehttp.net/license/
I would expect a more sensible license.
P.S, that license page is so well hidden.. only possible to find by going to the github page