Thank you. I am not sure how to introduce transit and the concept of hops in an introductory post without explaining in depth about the networking side of it. Maybe you could help me out with that?
Load balancing isn’t just about server load or congestion, it’s also about network load and congestion. If a web page or video takes longer for a user to download, it ties up the server longer too.[1]
Load balancing algorithms can also consider network paths or round trip times between the user and a server to give users a faster web download or video stream. To do this, they may use information from network routing topology, such as how many “hops” or routers between the user and the server, or may even triangulate actual network performance by assessing measurements from multiple data centers and load balancing to the most responsive.
1. See “snoshy” comment on latency in these comments: https://news.ycombinator.com/item?id=25920284 — roughly, you aim to avoid queuing or connection creep, as you mentioned in the intro, and speed of opening, transmitting data over, then closing the connection, can make a huge difference.