I don't know if my testing was right. If anyone from HAProxy team reading this.

retry count is 3 and you have 5 server in backend and 1 backup server and you have health check.

if all servers are down, request will be forwarded to backup server

but what if all the servers are down, but health check is not updated yet. (extreme timing)

the request will be retried 3 times, servers will be mark down. as all the 3 request were failed, HAProxy will return 503.

I think the request should go to the backup server, even if retry limit was 3, HAProxy was not able connect to any of the 3 server and servers were down actually.