> I would've expected a single TCP connection to be able to saturate a line
There's several factors at play that make this (usually) not the case.
Besides physical latency (which includes those added by any VPNs/tunnels/etc., some of which may be internal to an ISP along the route and outside of your control), there's other things like the TCP window sizes / window scaling option[1] that can affect single stream performance, and those type of parameters can differ by OS/interface type on both ends.
Also for SSH specifically, it has its own fixed buffer size that also limits throughput unless you're using the HPN-SSH fork[2].