I've also been thinking about another possible optimization:
Two things have a big impact on connection start-up performance currently: handshakes (3-way for TCP, 4-way for SCTP, any properly-designed UDP protocol implements a similar one to prevent amplification attacks) and slow-start. Currently both of those are on a per-connection (defined as an L4 connection) basis.
What if there was a concept of "endpoint associations" in addition to connections? An endpoint association would be a bundle of connections, potentially from different applications, between the same endpoints. An active endpoint association would include congestion control information (so new connections within an established association could immediately start communicating at full speed) and also means the endpoints are authenticated. That makes the handshake only necessary for creation of associations, not individual connections.
If that was done at the OS level, the process for opening a new connection would be something like:
1. User wants to open a connection to 1.2.3.4:80. (L4 protocol doesn't really matter for the example.)
2. OS checks if there is a currently active endpoint association with 1.2.3.4.
3. If there is an endpoint association, create a connection within that association and start sending data immediately, with no handshake, and using the congestion control information (window size) from the association.
4. If there isn't, open an endpoint association, complete the handshake for that, and open the connection within the newly created association. (Future connections would be able to go through the simplified and quicker process above.)
I think the handshake isn't a big issue, but eliminating slow-start for connections to an already-established association should be beneficial. Thoughts?