I'm not sure the issue here, that sigterm is sent before removal from the load balancer? If you handle the sigterm gracefully doesnt this solve your issue...
I'm no k8s expert so maybe I am misunderstanding the problem
I'm not sure the issue here, that sigterm is sent before removal from the load balancer? If you handle the sigterm gracefully doesnt this solve your issue...
I'm no k8s expert so maybe I am misunderstanding the problem
That's exactly the issue. It'd be better for the orchestration system to handle removing the service from the LB in response to a configuration change than to wait for e.g. health checks to fail.
The SIGTERM method means you have to modify every service to accept SIGTERM (no big deal) for some amount of time (fiddly configuration detail) and in the mean time wait for the LB to see the terminating pod's health check fail so it can stop sending it traffic.
If we flipped it around a bit, having the LB remove the traffic before sending a term signal, we could be more assured that poorly behaving pods (pods that die too quickly or too slowly) don't result in user-facing issues.