This did cross my mind! Specifically, just forwarding to something other than TCP. Doesn't sound impossible if we change interfaces a little bit.
Thinking, we can make dialing the provider responsibility. I didn't want to block the goroutine there, but it could just spawn another temporary one for dialing.
Then the manager can be generic over ReadWriter, and optionally try if whatever it has is also Closer.
Beyond that, I think Docker talks gRPC?
P.S.: Note that I made this AGPL. If you planned on creating some sort of product out of this, best start fresh from the golang.org/x/crypto. Might even be simpler in the end, because I feel like a big part of lazyssh is management of resources, which in Docker is (probably) local, cheap and almost instant.