Not only is the handshake expensive, but TCP has something called "slow start" which effectively artificially limits the rate of data in the beginning of the connection till the network capacity is discovered.

An ideal N would be high enough where the cost of the TCP handshake and slow-start are minimal, but small enough not to trip the traffic shaper to downgrade the connection from the fast flow to the slower one.