Hacker News Reader

Comment by smitty1e | Hacker News Reader

If I have a cloud provider, why might I use HAProxy over the provider's load balancing solution?

You'd have more specific than "every single cloud provider" but in general HAProxy is way more advanced than most other proxies.

Stickiness, rate limiting, gzip compression, configurable logging, caching, Lua scripting, load balancing to domain names, more options for health lifecycle. It even has a control socket to instantly manage the server without restart. HAProxy is actually the inventor the PROXY protocol which a number of other systems have now copied.

The convenience of cloud proxies is no small matter, but on a feature-by-feature basis HAProxy easily dominates.

Well said, to add to this, HAProxy is also closing the gap on convenience with its recent addition of AWS EC2 service discovery [1] and Consul service discovery integrations [2].

[1] https://www.haproxy.com/blog/announcing-haproxy-data-plane-a...

[2] https://www.haproxy.com/blog/announcing-haproxy-data-plane-a...

Wow, cool. HAProxy is coming a long way. Consul integration makes me excited to take another look at using it for our mesh. Using Fabio right now, and it's fine, but I'd love to go back to HAProxy for this if it makes sense.

disclaimer: I work at HAProxy Technologies

More control/flexibility (i.e. cloud provider usually gives you a limited number of "checkboxes" that they want you to be able to modify), generally better observability (HAProxy timing metrics [1] and session state at disconnection codes [2] are very helpful for debugging), and granular control over timeouts [3]. You also get builtin rate limiting [4], which from what I have seen with most cloud load balancers usually requires an up-charge through their "WAF" product.

Better health checking, circuit breaking, and layer4/layer7 retries [5].

Another consideration is whether you have spiky traffic patterns. For AWS load balancers at least, they need time to "warm up" to larger traffic levels. If you manage your own load balancing tier you can scale on demand.

[1] https://www.haproxy.com/documentation/hapee/latest/onepage/#...

[2] https://www.haproxy.com/documentation/hapee/latest/onepage/#...

[3] https://www.haproxy.com/documentation/hapee/latest/onepage/#...

[4] https://www.haproxy.com/blog/four-examples-of-haproxy-rate-l...

[5] https://www.haproxy.com/blog/haproxy-layer-7-retries-and-cha...

I last used HAProxy about a decade ago for fairly simple load balancing and caching - worked really well.

Question about the suitability for a dynamic backend case. I have a system where clients are assigned to backends exclusively for the duration of a session. At the moment I have a pool of backends and each client is explicitly told which backend to use so they can route to it. Ideally I’d make this transparent so it was easier to reassign on the fly. Is this case reasonably supported with HAProxy and which features should I be looking at to get a sense of it?

I’m working under the assumption that I’ll need to build the machinery to maintain the pool and registry of where clients should be routed to.

While I have not tested this specific use case, I would start by exploring map files [1] and/or stick tables [2]. Map files can be updated on-the-fly through the HAProxy Runtime API. With stick tables you can persist a session/client based on anything found within the request headers, which would include a users session id.

[1] https://www.haproxy.com/blog/introduction-to-haproxy-maps/

[2] https://www.haproxy.com/blog/introduction-to-haproxy-stick-t...

Excellent, thanks for the info, will check it out.

We actually use both. We have the cloud provider's load balancers balancing to HAProxy load balancers. We use HAProxy, because it is much more flexible and powerful the the cloud provider's LB, and we use the cloud provider LB because it integrates because it is simpler to terminate public TLS with them and manage the certificates.

More control, maybe less cost? If your cloudy provider has an awful solution like the one I was using, you'd get better uptime by doing it yourself.

I haven't heard many problems with AWS or Google load balancing, but sometimes there's issues where people talk about 'pre-warming'... more control would probably help there.

If you're running HAProxy on AWS, be sure your AWS firewall rules are stateless, otherwise you may well run into the unpublished connection tracking limits.

> If you're running HAProxy on AWS, be sure your AWS firewall rules are stateless, otherwise you may well run into the unpublished connection tracking limits.

This.

I got bit by this the other day, a bunch of servers were having very low CPU usage but new connections to them were failing. Turns out it was because they were hitting the conntrack limit. Changing the security group rules to allow all from all "fixed" it.

Although after a quick test, I think there's another limit, also unpublished, as I still can't establish as many connections as I want on a smaller instance – same number of requests on fewer connections works, and also a bigger instance works with many connections. In every case, the CPU was largely unused.

AWS doc regarding the issue: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/security...

> Although after a quick test, I think there's another limit, also unpublished, as I still can't establish as many connections as I want on a smaller instance.

Any chance this is an OS limit in your instance? Either a connection tracking firewall there, or FD/socket count limits? I don't have experience beyond the AWS connection tracking limits because I ran into them while testing something, but only found out how to avoid them much later (the test didn't go well, because of the limits, so we didn't go forward, and our assigned rep didn't tell us about the workaround either; I only found out by complaining about the limit to enough people that someone knew how to fix it)

I doubt that, because the same exact VM, after changing to a beefier instance type (c5.4xl, instead of the original t3.micro and later c5.large) accepted the connections without any problem.

If it's OS related, it may be some dynamic limit based on RAM size. But the limit was something around 100-200 connections, so nothing particularly crazy.

The way I found out about the limit was that downstream servers couldn't establish connections, and a manual test showed significant packet loss, while the servers were under 10% CPU and accepted connections locally. I opened a support ticket and the guy apparently saw that they were hitting the conntrack limit.

Once I replaced the security group with a blanket allow all as he suggested, connections could be established again. But two days later, when the number of connections increased further (while the security group was still allow all), I started seeing again connections not being established, although the servers were still not particularly loaded. As these are UDP (DNS) servers, I can't usefully quantify the number of different clients.

I set up a quick lab with two isolated EC2 instances and figured there must be another limit to the number of connections, although it seems somewhat higher – or maybe the conntrack limit as shown by ethtool doesn't update immediately. This happened yesterday, Saturday, while I was on call, so I didn't push the tests far enough to reach any clear-cut conclusion. I just doubled the servers and everything went back to normal.

However, the limit does seem to be related to the instance size and per interface. Meaning that adding another interface doubled the connections I could establish. Again, I didn't push the tests any further, though I intend to this sometime next week.

To add to what other comments have already mentioned - cloud lbs send you random resets. If you don’t want to have your tcp sessions fail randomly under normal operation you have to use your own load balancer

They have very nice features that provider's load balancing might not offer you.

First to me is the ability to retain the IP address. When using AWS Load Balancer, the LB scale automatically and replace the IP.

Second is sophisiciated routing. Example route by cookie. What it buy is that by setting a cookie you can make sure the request hit a specific pool. You can also have concept of backup

Third is lots of useful metrics with their stats page.

Fourth is the ability to alter the responses such as removing header. You have to pair with lambda edge for these on AWS. What it helps is imagine you can write back the user_id to HAProxy, HAProxy log it together with the request, and delete it(so client won't see it). It helps to easily tail the log for a specific user.

And a lot of rate limiting built-in features that just take a few lines of config.

Small note - AWS Network Load Balancers maintain the same IP addresses.

Yeah I was confused too. Why on earth will someone provide a load balancer if it keeps changing it's IP address as it scales?

The normal AWS load balancers are set up to stay in service as long as the cache time on their IP address having been served by DNS... (which, FWIW, I have long argued "isn't sufficient", as many ISPs in countries half a world away from the US or using tech like satellites don't honor DNS cache times and then your requests end up getting routed to someone else... I routinely got tons of HTTP requests clearly destined for someone else's product).

Because they're intended to be referenced via hostname, and yes, regular HTTP AWS load balancers do indeed rotate their IPs.

That's true, but functionality-wise, they can do much fewer things.

With an application load balancer, you can set up multiple backends, do basic redirections, do OIDC authentication, etc.

With a network load balancer, you can only spread TCP and / or UDP connections.

We use it at work for hosting an app designed pre-cloud (using one db per tenant) in AWS:

We deploy auto-scaling groups for each major app version (because customers can choose when to upgrade, so we can have anywhere from 1-3 versions live at a time). There's a database that has an entry for each customer system, with one or more domains, the app version and the db name (usually auto-generated but for historical reasons can be set manually). There's a UI to manage all this.

A script takes this data and builds haproxy config, creating backends for each version group and routing rules for domain to the proper group. This part could maybe be done with ALB now, but I am not certain of that.

We also automatically configure SSL for all domains: anything that doesn't have a static .pem file gets LetsEncrypt cert. Most of these are done via HTTP-01 because they're customer-owned domains that just CNANE to us. None of this is doable via AWS built-in stuff.

There's also a bunch of other hacks that haproxy does:

doesn't redirect to SSL for a couple specific (non sensitive) URLs+user agent that doesn't follow redirects;

returns a fake "success" page for a long-gone service called by an obsolete client some customers of (ex-)customers are still running, which effective causes a DDoS attack due to retries if we return 404 or 5xx;

Has some awareness of backend state and shows better error pages than just a generic 5xx depending on situation.

Haproxy instances are behind NLB, but otherwise there's a single (layer 7) hop to the app server.

The end result is you can configure a new system via our management UI, and so long as DNS is setup (using a wildcard subdomain we own, and/or customer's CNAME entry) within a few minutes it will be live (database deployed, app servers aware of connection/domain mapping, proxy configured with SSL).

One potential use case: proxying targets with no fixed IP addresses for services which require fixed IP addresses. For example, exposing RDS databases via privatelink. It would be nice if AWS would address this natively, but HAProxy will do.

Looking at the standard haproxy error pages present when using an Azure App Service, you might be surprised to learn that the cloud providers are using haproxy for you, behind the scenes.

Missing capabilities. Some cloud provider load balancers don't support mutual authentication with TLS, for example.

Cost. I imagine.

To expand on this I believe an aws load balancer is $30 / mo.

For early stage projects, and particularly for personal projects it's more cost efficient to use haproxy.

As an added bonus it's got all the bells and whistles to match demands as you scale, and no (cloud) vendor lock-in

Reply on news.ycombinator.com