Hacker News Reader

Ah yeah. Not sure on that one. scp does have the `-3` option to copy between two remote hosts via the local host, but that can be significantly slower if the remote hosts are in the same network and local host is not.

View on HN

Exactly. If I need to move a few megabytes around, <scp -3> and a coffee or a few simple tickets is a good way. A year ago or so, I needed 600GB moved between two systems ASAP during an outage that'd turn into a money-bleed at 6am. If I piped that through the VPN and my workstation, I'd probably still be waiting today.

Some time take a look at lftp [1] and its mirror subsystem for this. It can break up a batch of files or even one large file into multiple SFTP streams. Another upside is that it can replicate most rsync behavior in a SFTP-Only Chroot account. Downside is that without a corresponding daemon like rsync on the other end directory enumeration is slow which isn't a problem if one does not have a complex directory structure.

Play around with the built in rate limit options total and per thread to keep the network people happy.

[1] - https://linux.die.net/man/1/lftp

Reply on news.ycombinator.com