Hacker News Reader

We transfer large files containing raw radar data, and moderate sized files contains databases of target movements and track information.

We use rsync.

When I worked for the local university we had to transfer data between machines to run experimental parallel programs on so-called "big data."

We used rsync.

View on HN

Ahh ok thats good to hear. Have you considered using multipath transport protocols with something like rsync? I am curious if it could benefit this situation. MPTCP sounds like an interesting protocol if you control both hosts. https://www.multipath-tcp.org/

We were/are always restricted by intermediate limits on throughput, so it's never been useful or interesting to consider alternatives.

YMMV, but if you want to improve throughput, consider carefully where your data has to go through. But rsync is rock-solid, well-understood, mature, and just does exactly what it is intended to do.

Reply on news.ycombinator.com