Curl User Poll | Hacker News Reader

Curl User Poll

daniel.haxx.se

78 pointsakerl_29 comments

"Please check the protocols you use curl/libcurl for"

I had no idea Curl could be used for protocols other than HTTP/HTTPS.

FTP I did knew, but oh that list just took me by surprise. Suddenly I'll be using curl even more.

    $ curl --version
    curl 7.40.0 (x86_64-apple-darwin13.4.0) libcurl/7.40.0 SecureTransport zlib/1.2.7 libidn/1.28 libssh2/1.4.3
    Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
    Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets Metalink
    $

Indeed, and they have been a source of some nasty security vulnerabilities in the past. If you're using libcurl, it's a very good idea to restrict redirects to HTTP and HTTPS only (with CURLOPT_REDIR_PROTOCOLS), lest a malicious site redirect you to an imap or smtp URL.

I was wondering if you could write a whole email client in bash using just curl/libcurl somehow, but I do not have enough foo for it.

One day, one day.

Too bad the "worst areas" list didn't include the UI. I cannot recall the number of times I've spent looking through the docs, which flag was it again? -s? -S? -q? Every. Single. Time.

But it's a minor annoyance. Great great tool.

I personally think curl is one of the better examples of CLI tools as it follows the pipe metaphor by default (which wget doesn't), supports both long and short flags (so if -s isn't memorable, then use --silent) and doesn't require a flag for it's core mandatory input (ie a URL). You can even slide flags in after the URL, if you wanted - which is something you generally only see supported by GNU utilities.

I could see curl being a little daunting to someone who is normally uncomfortable with the command line, but that's not really the fault of curl having a bad UI, that's more a paradigm preference of the user.

> it follows the pipe metaphor by default

Completely agree. As a novice 15 years ago, this bugged me, especially coming from arguably simpler tools like wget. But over the years I can to appreciate how it works. And now, wget seems kinda dirty.

as it turns out (just had to look it up again, incidentally!) the correct form when piping it to an output tool is -s -S. E.g.:

    $ curl -s -S http://example.com/some.json | python -m json.tool

I want errors, but I don't want that annoying status bar to mess up the output. I don't use curl often enough to even remember I need two flags at all (e.g. I remember now, but didn't know when I wrote the original comment).

It can follow any philosophy it wants: when a user is consistently confused, I personally consider it bad UI.

EDIT: This is not the only bad / confusing default. Another popular one is -L. Not following redirects should be the exception; I almost never want curl to choke on 3xx. Result, I could practically alias curl="curl -L". Great software, but the UI is unfortunate.

Firstly, every CLI HTTP client I've used outputs a status to console, even when it detects standard out is being redirected / piped. It can be very annoying at times, but isn't a curl specific thing. And it's hardly a great ordeal to remember --silent (or quickly curl -h | grep silent / do a find inside man, should you need a memory jog). Perhaps a little tip here is to force yourself to use the longer flags as they're much more memorable?

Secondly, have you ever considered that there are just as many people who might not want curl to automatically follow redirects? Personally I happen to use curl a lot to check that redirects work (eg sending the right HTTP status code) so I'd find your preference more annoying than intuitive.

Lastly, I get that you might not like some of curls default behaviours, but given curl's command structure follows the same idioms as most POSIX / GNU command line programs do, it's a bit far reached to say curls UI is bad or "unfortunate". Particularly when where's other people that do prefer the default behaviour you dislike. And since curl does follow POSIX idioms, you can just create that alias you discussed to force curl into your preferred default behaviour (you don't get much more user friendly than a tool you can customise the default behaviour of).

This is why I say that curl follows the standard CLI UI pretty well - it doesn't throw much in the way of unexpected behaviour and anything specific that you may dislike can easily be tuned via the usual methods.

What situation were you running into that required "-s -S"? I'm able to pipe the output of curl to something else (or just consume it via IO redirection) just fine without either switch. The status bar is output on stderr.

> Too bad the "worst areas" list didn't include the UI. I cannot recall the number of times I've spent looking through the docs, which flag was it again? -s? -S? -q? Every. Single. Time.

Complete agree. That's why I created HTTPie[1], a modern take on CLI HTTP to address those usability issues.

[1] http://httpie.org

I agree, I use curl once or twice a week and need crib sheet to remember what switches to use and when. I feel it pushes the UNIX tool "do one thing, do it well" thing a bit far.

You can prepare a ~/.curlrc to take of these personal defaults for you:

~/.curlrc:

  location
  show-error
  silent

Additionally

1) if you supply -q as the first switch on the command line, the curlrc file will not be used.

2) you can use -K <config_file> or --config <config_file> to pick a specific configuration.

Survey is down, so I'll just post. I use curl primarily for testing web service apis. It's also pretty awesome that chrome and other devtools can 'copy request as curl'.

The survey is here: https://docs.google.com/forms/d/1uQNYfTmRwF9RX5-oq_HV4VyeT1j...

Being a similar user of curl, I found I had to skip a lot of questions.

The survey calls itself a "curl user poll" but most of the questions seem to be written as a "curl developer poll". It's actually pretty hard to convey what I think is the majority position of curl users: I use it, I'm glad it exists, and I never think about the code it contains.

Shame you can't complete the survey using a cURL request.

The Google machine disagrees

https://journalxtra.com/linux/bash-linuxsanity/bash-filling-...

TL;DR

cURL can fill out forms using the "-d" option.

http://curl.haxx.se/docs/manpage.html#-d

Google, very sensibly from a web security point of view, have a hidden nonce field in their surveys that you'd need to parse out of a prior request in order to submit the form. That means you can't use 'pure' cURL. formfind might well work to grab the value you'd need to send though.

Pipe/heredoc to edbrowse will handle that, too.

I listened to an episode of Changelog last night which featured curl creator, Daniel Stenberg, as a guest: https://changelog.com/153/

It's worth listening if you're at all interested in curl or popular open source projects in general.

He also does weekly videos discussing curl, HTTP/2, stuff at Mozilla, etc; worth a subscribe: https://www.youtube.com/user/danielhaxxse

http://curl.haxx.se/donation.html might not be a bad place to visit

Can do a lot more with Curl than I imagined. Unfortunately for me, when it comes to those kinds of uses, I'm rarely still in Bash, and I've already started using Python or Go's HTTP libraries.

Python also has a curl library.

Go has a few curl packages too: https://www.google.co.uk/search?q=go+lang+libcurl

Sure, but I have personally never found need to use it instead of the stdlib http2.

Reply on news.ycombinator.com