HAProxy 2.7
haproxy.com
haproxy.com
This is such a welcome change! I've been in more than one situation where I rediscovered this limitations when trying to scale things while trying to remain monolith (due to costs and performance reasons) but noticing HAProxy not being able to utilize the newly upgraded machine. This will make a huge difference!
It's future-proofing basically.
The page is ancient https://www.haproxy.org/10g.html but they were doing 10Gbits on Core2Duo in '09 (sans SSL but back then AES acceleration wasn't very common either)
Usually the only reason to use many cores is SSL, but since OpenSSL 3.0 that totally collapses under load, even then you're forced to significantly lower the number of threads (or to downgrade to 1.1.1 that about any high traffic site does).
3.1-dev is slightly less worse but still far behind 1.1.1. They made it too dynamic, and certain symbols that were constants or macroes have become functions running over lists under a lock. We noticed the worst degradation in client mode where the performance was divided by 200 for 48 threads, making it literally unusable.
Also, I found it best to work with everything avoiding cross CPU communication; making sure everything happens on the CPU core that handles the RX queue for the incoming connection. My NICs only did 16 queues, so more cores than that wasn't useful for me. Do they make nics with 128 queues now? (Selecting outgoing ports so traffic to/from origin stays on the same CPU is tricky, but also solves the problem of making sure there's no port contention between the processes)
Yes, this is absolutely critical. Atomic ops (including taking a lock) when you have a large number of cores just completely kills performance.
>Do they make nics with 128 queues now?
Yep, the Intel E810 100gbit controller supports 256 queue pairs for just one example.
Hope you can process that traffic pretty fast, the amount of memory dedicated to packet buffers with that many queues is making my head spin a bit.
My use case was just tcp proxy, inspecting the first few bytes to determine destination (http vs https vs proprietary) and add proxy protocol headers to the origin. Setup is somewhat expensive (tcp table manipulation isn't cheap), but data on established connections is simple and easy.
The bottleneck was connection establishment in the kernel, and as traffic decreased it became harder to run meaningful tests without great risk (send all the traffic to one server to see if it can handle it is fun, but unsafe), and we had to keep a fair number of servers to keep the IPs from the hosting provider, so we had way more capacity than needed and optimizing more wasn't a good use of time :(
Connection establishment in the kernel was an issue before kernel 3.8 I think though not sure. The problem is finding a spare port when you use a lot of concurrent connections and at some point it costs a lot. The "source" directive in haproxy was made to overcome this (you can define a per-server range so that the kernel doesn't have to scan its lists). But nowadays more efficient mechanisms were implemented in the kernel, that we support in haproxy (using IP_BIND_ADDRESS_NO_PORT) and we haven't witnessed these painful moments for a while now.
Somewhere my (not very great, but working) patch set is floating around, where I added rss aware port selection to HAProxy, so client and origin connections would both hash to the same NIC RX queue, and HAProxy never had to deal with cross-cpu communication for reading sockets.
I haven't looked at this code in a long time! Haven't gotten to a computer and it's hard to read this on a phone, so... And I left where I wrote it, so I can't confirm if there were any substantial changes after these. My fuzzy memory is this may be all the HAProxy changes, then I focused on kernel level changes.
Of course if HAProxy performs well enough, then that's still a million times easier.
Personally, i also use haproxy as a load balancer to terminate https and proxy traffic to internal services.
it's features, such as logging and debugging are far superior than nginx. While the haproxy config can be a little cryptic at times, you can do a lot more with it too.
I too have fought nginx on upstream health checking and have avoided the cost of Nginx Plus. I'd love a good alternative, but we have a number of lua plugins we depend on for rate limiting, metrics, auth, etc.
I do have an haproxy setup that does the right thing with LetsEncrypt, but it's just a path-based forwarder that works with an acme script on the system, not directly built in.
I know the sentiment has changed over the years. Curious what's the opinion today on which is the best tool for a new project.
(Yes, I realize they do different things but there's also considerable overlap as well)
The only thing it serves, from memory only (need reload to change) are error pages.
"NGINX suffers virtually no latency at any percentile. The highest latency that any significant number of users might experience (at the 99.9999th percentile) is roughly 8ms.
What do these results tell us about user experience? As mentioned in the introduction, the metric that really matters is response time from the end‑user perspective, and not the service time of the system under test."
[1] https://www.nginx.com/blog/nginx-and-haproxy-testing-user-ex...
There were big mistakes in this test, failing on a large number of settings, and even the NGINX tests were not as good as they ought to be in some aspects. This conducting Amir, Libby and I to join efforts on documenting better methodology for benchmarks, that we published under the Data Plane Benchmark project at https://github.com/dpbench/dpbench/ and later to the new test in this article: https://www.haproxy.com/blog/haproxy-forwards-over-2-million...
This was great and productive cooperation between engineers, despite working on different (and sometimes competing) projects, resulting in more reliable methods for everyone in the end.
It's fast, easy to tune/configure and cheap to run.
These days I always go for haproxy for reverse proxying.
OpenResty itself is BSD https://github.com/openresty/openresty#copyright--license
Then again, if you already use nginx (say for serving static files), using it to also front your app is the easiest solution and you only need to start wondering about alternatives once you hit a lot of traffic. Any benefits or drawbacks would only show once you start shoving in massive amounts of traffic, and your app will be bottleneck way before that.
* [1] https://www.haproxy.com/blog/extending-haproxy-with-the-stre...
This. Unless you're serving to tens of millions of users just use the thing that you're comfortable with.
HAProxy support 103 Early Hints since 2018. Nginx doesn't support it.
I really like Varnish's VCL configuration; something like HAProxy's more declarative configuration is easier to get started with, but in more advanced configurations the HAProxy declarative configuration becomes "declarative in name only" where you're embedding logic in the "declarative" configuration. I found it pretty awkward and VCL just becomes so much easier because the logic is expressed "directly" rather than through a declarative configuration. Varnish also comes with some tools to inspect/log requests, which can be handy for debugging Weird Issues™ at times.
Nginx kind of sits in between VCL's full-on DSL and HAProxy's configuration.
I don't know about the exact performance comparisons; Varnish has always been "fast enough" for me. I have a $5/month Linode VPS running Hitch and Varnish and thus far that's been enough for my ~1M requests/day with plenty of room to spare.
Disclosure: I am part of the HAProxy team.
I used to use nxinx plus, but after moving to a new company and trying to purchase it again, not only had the price quadrupled. I also wasn't able to actually buy a license without four meetings, a shitload of mails and a lot of frustration.
In the end I gave up and kept the community edition while I explore the options.
Nginx clearly moved from the cheap and performany segment to the expensive and cumbersome.
Then we've used Nginx behind that as the Reverse Proxy, static media serving, and basic static caching.
Basically HAproxy seems better for well... High availability and as a load balancing proxy. And Nginx has seems to be more suited for the HTTP(s) type pipeline.
You if all you need is basic load balancing, or some types of HA, then nginx will work fine. Last I checked (few years ago) nginx has a lot more http routing tools and http focused settings, headers, static file serving, etc. But HAProxy might have improved in the last 3 years with those.
It still poses us a problem which is that users don't upgrade. For example 1.6 is still routinely met in production despite having been unsupported for 2 years, and sometimes with 3 or 4 years of uptime because users forget about it or just don't want to risk an upgrade for no perceived benefit. I can't blame them honestly, as long as they upgrade before reporting problems or asking for help!
retry count is 3 and you have 5 server in backend and 1 backup server and you have health check.
if all servers are down, request will be forwarded to backup server
but what if all the servers are down, but health check is not updated yet. (extreme timing)
the request will be retried 3 times, servers will be mark down. as all the 3 request were failed, HAProxy will return 503.
I think the request should go to the backup server, even if retry limit was 3, HAProxy was not able connect to any of the 3 server and servers were down actually.
* Retries are one layer. By default set to 3 retries. HAProxy will retry the failed connection or request with the same server.
* "option redispatch" is another layer. If HAProxy can't connect to a server that is reporting as healthy, it will send the request to a different server.
* health checks are a layer. HAProxy removes unresponsive servers from load balancing.
* Backup servers are another layer. If all servers fail their health checks and are down, then backup servers come online to service requests.
All these things can be enabled in combination, and it would reduce the chance of a client getting a server error.
To answer your question, HAProxy will not connect with a server that is down (failed all its health checks). It will not retry with it either.
Disclosure: I'm a community contributor to HAProxy.