scp and sftp will multiplex too. Remote tab-completion for file transfers also becomes much more approachable this way.

scp remote:~/err[tab] .[enter]

Also, bastion connections get multiplexed as well so connecting to multiple different hosts behind the bastion goes faster.