Mastering curl: interactive text guide | Hacker News Reader

Mastering curl: interactive text guide

antonz.org

112 pointsasicsp16 comments

> Curl automatically expands glob expressions in URLs into multiple specific URLs.

> For example, this command requests three different paths (al, bt, gm), each with two different parameters (num=1 and num=2), for a total of six URLs:

    curl --output-dir /tmp -o "out_#1_#2.txt" http://httpbin/anything/{al,bt,gm}?num=[1-2]

Note that it's not just curl that does glob expansion. So do some shells. Try:

    echo http://httpbin/anything/{al,bt,gm}?num=[1-2]

Bash does "brace expansion" on `{al,bt,gm}` to create 3 separate params before they are given to curl/echo. (Although other, stricter bourne shells, like `dash`, do not perform this expansion.)

In the linked article, all the examples use a hostname of "httpbin". To run the examples locally, I had to replace that with httpbin.org.

Is that a standard test host? I'm curious about why it's named httpbin - is it a docker hostname?

Sorry for a weird offtopic question :/

Probably a locally hosted version of the docker image from httpbin.org?

Either way, it's some local network/local machine specific hostname.

Urls in the article would seem to line up with that theory though: https://httpbin.org/#/Dynamic_data/get_uuid

Cool, thank you!

Yes, it is a docker hostname. See the docs here: http://httpbingo.org

Thank you!

Shameless plug for a load testing tool I built that, in my workflow, relies on curl for the request syntax: https://github.com/Steven-Ireland/spam

I love cURL and have been using it for development, POCs, and documentation for years. I’m curious though… are people using it in production much?

Of course they do! Mostly through libcurl. For example, every single PHP http-related library/framework uses libcurl. Many others too.

Yes, a lot. All those zillion shell scripts which run the IT these days use it whenever they need to get to the Internet (surprisingly, it is much more common than wget(1), which in fact does more or less the same).

And, of course, everybody and their dog uses libcurl, directly or through some bindings to their programming language.

Curl actually does far more than wget.

Wget implements these protocols:

  $ man wget | fgrep 'HTTP, HTTPS, and FTP'
       the Web.  It supports HTTP, HTTPS, and FTP protocols, as well as

A full-featured curl (and specifically not the one distributed by Microsoft) implements additional protocols:

  C:\>curl --version
  curl 8.2.1 (x86_64-w64-mingw32) libcurl/8.2.1 OpenSSL/3.1.2 (Schannel) zlib/1.2.13 brotli/1.0.9 zstd/1.5.5 WinIDN libssh2/1.11.0 nghttp2/1.55.1 ngtcp2/0.18.0 nghttp3/0.14.0 libgsasl/2.2.0
  Release-Date: 2023-07-26
  Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
  Features: alt-svc AsynchDNS brotli gsasl HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL SSPI threadsafe UnixSockets zstd

Some of the particular protocols are limited - SMB, for example, is only SMBv1 (and not v2 or v3).

Microsoft's version is currently:

  C:\>curl --version
  curl 8.0.1 (Windows) libcurl/8.0.1 Schannel WinIDN
  Release-Date: 2023-03-20
  Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
  Features: AsynchDNS HSTS HTTPS-proxy IDN IPv6 Kerberos Largefile NTLM SPNEGO SSL SSPI threadsafe Unicode UnixSockets

it's commonly used in CI workflows for downloading assets and sending webhooks between tools.

Whilst not _production_ as such it's an integral tool for building

> I’m curious though… are people using it in production much?

Here's Daniel beating his own drum: https://archive.fosdem.org/2017/schedule/event/curl/

Fun fact, NASA used it for the mars helicopter

https://daniel.haxx.se/blog/2022/11/25/89-operating-systems/

I was looking for a guide, thanks!

Do we need to still learn this or can we all just switch to httpie or resti.sh given that it's 2023?

Reply on news.ycombinator.com