The power of two random choices

Cool!

> When simulating load distribution across servers, two-choices does an outstanding job at keeping things evenly distributed. However, I was unable to observe a meaningful difference between load balancing algorithms when measuring request latency from servers under load.

That's very interesting. Were you serving static content? Or running code, too?

Intuitively, I'd expect that NGINX serving static content would tend to scale fairly linearly up to the saturation point, and then get rapidly bad. That's a pretty pessmisic case for load balancer performance measurement, because you won't see effects until you're running the whole system very hot. That 'very hot' modality is important to real-world applications, but certainly makes benchmarking challenging. Does that sound right to you?

In my experience, that tends to change when running complex business logic code, or generating dynamic content. There, the load-latency graph has a higher linear slope. The effects of good load balancing are more immediately visible. It's also more visible in things which explicitly have longer latency with higher load, like some kinds of queue-based and stream-based systems.