SPDY reduces the number of persistent HTTP connections between client and server from around six (depending on browser) to one. High latency connections increase the time required to create these initial connections, and round trip times have not been consistently declining despite bandwidth improvements.
HTTP requests over each keep alive connection are blocking, i.e. the server must response before another request can be made. SPDY addresses this blocking by multiplexing requests and responses. Note that the head-of-line blocking that jganetsk mentioned is a separate issue.
The gateway can even send the incoming requests between many servers and aggregate the results and send them back to the client out of order, as per SDPY