Hacker News Reader

Comment by atombender | Hacker News Reader

So AWS didn't have TLS termination until now?

I'm still waiting for GCP to support more than ten (!) certs per load balancer, which seems like a ridiculous limit when you consider the need to serve dozens or hundreds of low-traffic customer domains on a single domain. If you're on Kubernetes, they're basically asking you to split your ingresses up just to avoid the limit.

We ended up going straight to running our own in-cluster load balancer (Traefik, which is meh, but works okay) with Let's Encrypt so we don't need to provision anything at all. It's so much nicer than fiddling with manual cert registration. I really wish cloud providers such as Google would get on board with Let's Encrypt already.

View on HN

They have supported TLS termination for https connections for a long time. This adds support for TLS on TCP connections. The NLB load balancer also allows you to keep the same set of pubic IP addresses for your load balancer endpoints.

We use cert-manager to get letsencrypt certs in GKE, and automatically push them to the Google global load balacer's.

https://github.com/jetstack/cert-manager

Doesn't that still require that you create a maximum of 10 host rules per ingress? That makes it harder to automate things.

With the current system, we can use one consistent IP or CNAME for all new domains. With the 10-certs-per-GLB limitation, we'd have to manage the DNS accordingly so that all the domains are correctly spread out among the N different ingress GLBs.

At the time we started with Kubernetes, cert-manager was listed as alpha quality and not recommended for production. Even today, it seems Gandi (which we use for DNS) still isn't supported; at least it's not in the documented list [1]. LEGO supports Gandi, so I'm not sure if maybe the documentation is wrong here.

[1] https://docs.cert-manager.io/en/latest/reference/issuers/acm...

I don't think this limitation applies when using Kubernetes with cert-manager. When you create Kubernetes Service API objects of type LoadBalancer, you get (by default) get a TCP load balancer on GCP.

SSL termination becomes the responsibility of the cluster. The certificate (and private key) is stored inside the cluster, too, via secrets. To have cert-manager automatically create and renew certificates, all you have to do is update your ingress host/tls YAML configuration.

Using cert-manager, you can continue exposing your app under many domain names with a single IP (via an A record) or abstractly through a CNAME record.

I still haven't added more than 10 domains to manually verify your concern, however. But Kubernetes isn't making any changes to my load balancer as I've added domains. I would re-visit to see if cert-manager is right for your use case: I noticed no mention of it being alpha quality on the README. However, they do point out it is 0.x and there could be breaking changes to the API later.

So the method you describe ends up using a TCP GLB, but the point of using an HTTP GLB is to enjoy all the benefits of the GLB.

With a HTTP GLB, you get a very cheap distributed, effectively global CDN that doesn't require special configuration. You also get features like health checking, logging (which you can pipe to BigQuery), Google's "just works out of the box" edge caching (negating the need for an external caching CDN like Fastly), and so on.

My understanding is that the typical use case is to run cert-manager together with ingresses, where cert-manager's will allocae certs and create Kubernetes secrets for each. If you wire up ingresses with those secrets, and you use the "gce" or default ingress class, then you end up getting TLS termination at the GLB level, very easily.

However, because of the 10-certs-per-LB, you run into the problem I described before. There's no way around it except to create ingresses that contain a maximum of 10 hosts. (It's a hard limit, not a quota; you can't petition to have the limit increased.) If you have 60 domains, that necessarily means 6 GLBs, and 6 different IPs to point DNS to.

We're using Traefik today as a custom ingress controller with a TCP GLB. So we get one external IP, and Traefik handles TLS termination (via Let's Encrypt). So this is effectively the same as running cert-manager plus something like the Nginx ingress controller.

Yeah, I should have clarified in my case I am using the nginx ingress controller, but Traefik would also suffice.

They supported it on ELBs and ALBs, but not NLBs.

I’m not sure about gcp but on aws you can have domain validated acm certs which are arguably more convenient than LE certs.

aws will transparently renew them for you if they are associated with a load balancer or cloudfront. no moving parts required in your infra.

Reply on news.ycombinator.com