Hacker News Reader

Comment by ketanmaheshwari | Hacker News Reader

I have a few questions about curl:

-- Curl claims that it supports so many protocols but I do not see many examples where they are in action, for instance, how to use ssh protocol to perform transfer between two machines? Where can I find such examples?

-- Are there any examples of a fully functional high-level data motion tool that is based on Curl?

-- Has anyone used Curl to perform data transfers at large scale such as terabytes of data? I would love to read more about it if there are any pointers describing such transfers.

View on HN

> for instance, how to use ssh protocol to perform transfer between two machines? Where can I find such examples?

That's out of scope for cURL, use `scp` for that.

> Are there any examples of a fully functional high-level data motion tool that is based on Curl?

What's data motion?

> Has anyone used Curl to perform data transfers at large scale such as terabytes of data? I would love to read more about it if there are any pointers describing such transfers.

There's a few posts on Stack Overflow about that, but they go up to gigabytes. cURL is not the best solution for transfering terabytes of data. http://moo.nac.uci.edu/~hjm/HOWTO_move_data.html seems to be a good overview of the challenges (and tools) to use to securely and reliably transfer terabytes of data, referencing rsync, bbcp, bittorrent, etc.

> Curl claims that it supports so many protocols but I do not see many examples where they are in action, for instance, how to use ssh protocol to perform transfer between two machines? Where can I find such examples?

SSH file transfers use either SCP:// or SFTP:// urls, not SSH:// urls.

>> curl sftp://public.example.com/file.txt

You could try it yourself! The man page is pretty good.

Reply on news.ycombinator.com