Hacker News Reader

Comment by mnarayan01 | Hacker News Reader

Or just skip the shell entirely:

  tar -C /dest/dir -xf -

View on HN

Yeah that's the right way to do it, but I always forget which flag it is for tar and this is faster than pulling up the manpages ;)

Could you give a full example of how this would work? Receiver and sender..

    tar cf - src/ | ssh $host "tar -C /dest/dir -xf -"

Do you have any idea about a crazy fast way to transfer files between two machines on the internal network?

No encryption necessary No integrity checks No resume support Just plain and as fast it could get..

I need to try the netcat version but I'm hoping someone could show me a concurrent version of it that is mad fast.

You can use nc (netcat). If you have a 10G ethernet you will not saturate it because you will be limited by disk IO (I get 1Gb/s disk reads for a M.2 SSD). I you can read from disk faster, netcat alone will not saturate a 10G link, probably will hit a 3Gb/s limit (depending on your hw). You will need parallel transfers (xargs, a bit of scripting etc). You can also try rsync, maybe it is good enough for you.

From your source machine:

  < /path/to/source ncat remote-host 8001

On your destination machine:

  ncat -l 8001 > /path/to/dest

Though in most situations with files of reasonable size, you're probably going to be better off running through `gzip -c`.

Reply on news.ycombinator.com