Hacker News Reader

Comment by CobrastanJorji | Hacker News Reader

If you regularly do command line JSON requests, I'm a big fan of HTTPie. It's so much easier to use correctly. https://httpie.io/docs/cli/examples

For example, here's a JSONy POST request with cURL:

curl -s -H "Content-Type: application/json" -X POST https://api.ctl.io/v2/authentication/login --data '{"username":"YOUR.USERNAME","password":"YOUR.PASSWORD"}'

Here's that same request with HTTPie:

http POST https://api.ctl.io/v2/authentication/login username=YOUR.USERNAME password=YOUR.PASSWORD

View on HN

After this change the equivalent would be:

    curl -XPOST --jp username=YOUR.USERNAME password=YOUR.PASSWORD https://api.ctl.io/v2/authentication/login

Which isn't too far from your desired outcome (notably, without relying on argument position for meaning). Although, I guess the argument that "curl is already installed on almost every server", sorta gets moot because I imagine it will take a while for most distros to move to the latest curl that will support --json/--jp

It's a lot easier for a distro to update a default utility than to add a new one though. I suspect having updated curl would happen faster than adding httpie to the list of default utilities.

The exact opposite is more likely true, depending on the nature of the change. A new utility has no current usage so there are no in-the-wild backwards compatibility concerns. The system python on redhat based distros was stuck at 2 for a long time because some of the sysadmin utilities relied on it and all those deps needs to be vetted or converted to either a newer python version or their own shipped version of the runtime that didn't use the general purpose python install.

There's a big difference between a bump to the latest curl version and a jump between two major versions of a language runtime. The Python 2->3 transition took years and many distros kept both versions.

Most dependencies get updated much more quickly. It wouldn't even shock me if this change got picked up mid-cycle.

Unhappily RedHat won't. Heck, RedHat has gone well out of their way to backport updated TLS handshakes (1.2 at least) to their 2013 version (7.29.0) of curl.

I'm assuming that you mean for RHEL? That's a little bit of a special case.

Python is a huge outlier in this case. Very rarely do you have a default tool that can't be upgraded, and this case was only because Python is an interpreted language.

Curl is a command line tool. As long as they only add new functionality, there is very little that prevents an upgrade.

This is only a valid comparison if curl deprecates or moves options, as Python 3 did with language features and standard library functions.

Especially singe HTTPie is a python package. curl exists even in plenty of embedded systems, in many Docker base-images, etc.; but scripting runtimes like Python generally don't.

Except if it's an enterprisey distro, it won't get updated. Specifically RHEL 7 ships 7.29.0 and cherry-picks features (TLS handshakes) to backport to their nine-year-old version.

I wish this were the case. Unfortunately for us grownups that run stable/LTS distros, non-security updates like these don't usually make it to us for like 5 years or something.

There's a happy medium, and we're not in it.

It depends. How long did to take for bash4-isms to get adopted? Sometimes the update time for more basic utilities, for all systems can take a long time. (Apple switching to zsh doesn't help here.)

nit for understanding's sake: is the second --jp missing?

Not per the linked wiki [1]. But it is still being designed it seems.

[1] https://github.com/curl/curl/wiki/JSON#--jp-part

And that -X POST is actually not needed. Adding any payload will make curl do a POST request.

Also see: https://daniel.haxx.se/blog/2015/09/11/unnecessary-use-of-cu...

Shameless plug, we've built hurl [1] to add some syntactic sugar over curl, with plain text. For example, you can write a text file post.hurl:

    POST https://api.ctl.io/v2/authentication/login
    {
      "username": "YOUR.USERNAME",
      "password": "YOUR.PASSWORD"
    }

and it will send the same POST request with a json body:

   hurl post.hurl

You can add asserts on the response too:

    POST https://api.ctl.io/v2/authentication/login
    {
      "username": "YOUR.USERNAME",
      "password": "YOUR.PASSWORD"
    }
    HTTP/1.0 200
    [Asserts]
    jsonpath "$.status" == "LOGGED"

Under the hood, we use libcurl and a Rust binding. The HTTP engine is curl because curl is awesome!

[1] https://github.com/Orange-OpenSource/hurl

>hurl

Awesome name.

It is a good name! https://github.com/EdgeCast/hurl

Hurl probably has many fine uses - but note that curl reads json from file just fine:

  curl -d @post.json http://example.com

I have recently been trying a port of HTTPie in Rust, xh [0] (which is needlessly hard to find in a websearch). I am a Python guy, but love having single executable tools.

[0] https://github.com/ducaale/xh

xh is great, just like HTTPie but incredibly fast.

I alias it to http so it's memorable and the commands make more sense though.

Happy to hear about xh, thanks for mentioning it. I recently ran into a problem where HTTPie didn't support HTTP/2, so I had to fall back to curl. The ticket for HTTP/2 in HTTPie is still open https://github.com/httpie/httpie/issues/692

Happy to see that xh supports HTTP/2 out of the box.

Since htttpie can't do nested JSON assignments I like to use jo, like this:

  $ jo foo=$(jo bar=qux) | http example.com

Piped JSON automatically uses POST

This is amazing! Definitely adding to my toolbox.

(HTTPie maintainer here)

You might be in luck since Nested JSON support is going to be star feature of our upcoming release. Here is a sneak peek:

$ http --offline --print=B pie.dev/post \ search[type]=client \ search[stars]:=50000 \ search[platforms][]=Web \ search[platforms][]=Desktop \ search[platforms][]=Mobile \ search[platforms][]=CLI

{ "search": { "platforms": [ "Web", "Desktop", "Mobile", "CLI" ], "stars": 50000, "type": "client" } }

We are rolling a brand-new mini-language that integrates really well with the existing request building syntax, but also features stuff like JSON type-safety and amazing error messages for basic syntax errors.

That sounds great! Any chance of HTTP/2 or HTTP/3 support?

It will probably be post 3.0, since the underlying HTTP interface we use (https://pypi.org/project/requests) does not support HTTP/2. We are currently discussing how to migrate from that to something like httpx, without causing any change of user-visible behavior.

For longer than I can recall I've had a `post_json` alias like this:

    curl -H'Content-Type: application/json' -d @- << 'JSON'

Invoke with

    post_json <url>

Then just paste or write JSON without caring about quotes etc, finish the heredoc, done.

Don't you need `<<'JSON'` instead of `<<JSON`? Otherwise any $ in the json body would be interpreted as a shell variable reference.

Yes, I mistyped on my phone. Edited, thanks!

TIL you could open (and not close) a heredoc string in an alias.

curl's primary advantage is that its pretty much universal.

I can hand another dev a curl statement without having to worry if they have the requisite software to reproduce the call.

Also Postman, Insomnia, Sentry, & Swagger all support ability export to curl.

This example isn't fair/steelmanning cURL.

  -s: if HTTPie disables progress bar by default, that's just a different design choice, the advantage is in the eye of the beholder
  --data '{"user...: likewise, HTTPie's default to JSON is a design choice. I wouldn't say the default is superior
  -H "Content-Type...: likewise, if HTTPie adds this header by default, thats just getting in my way when I don't want that header
  -X POST: you don't need to specify that in cURL if using --data

HTTPie can be even more compact by using the command https to indicate to use that protocol. This makes the command look like:

https POST api.ctl.io/v2/authentication/login username=YOUR.USERNAME password=YOUR.PASSWORD

I think --data implies -X POST, so that part at least would be unnecessary.

Ahh, I believe you're right. "PUT" would have been a better example, in that case. But I suppose getting the cURL syntax wrong helps my point that I find the cURL syntax confusing.

HTTPie does not support multi-valued keys in query strings (sending several field[]= for example). It is very annoying and, for consistency, I would rather use cURL (instead of mixing syntax with HTTPie).

If I’m understanding your point correctly, HTTPie has support for multi-value keys in query strings.

You can use the `name==value` query parameter syntax [0]:

  $ http --offline pie.dev/get AAA==AAA AAA==BBB
  GET /get?AAA=AAA&AAA=BBB HTTP/1.1

Or simply:

  $ http --offline 'pie.dev/get?AAA=AAA&AAA=BBB'
  GET /get?AAA=AAA&AAA=BBB HTTP/1.1

[0]https://httpie.io/docs/cli/querystring-parameters

I appreciate HTTPie but it can never beat the fact that you curl is almost always installed by default.

There is a different way, without curl or other curl-like programs, without Python, Go, or the like. It is smaller and faster, IME as a non-developer.

   post api.ctl.io/v2/authentication/login < 1.txt|openssl s_client -connect api.ctl.io:443

post is a 702-character shell script. printf is a built-in.

      #!/bin/sh
      (  
      y=Connection;n=0;while read x;do
      x1=${1#*//};x2=${x1%%/*};x3=${x1#*/};
      x=$(printf "%s" "${x%%:*}:";echo "${x#*:}");
      if test x"${x3}" = x"${x2}";then x3="";fi;
      printf "%s\r\n%s\r\n%s\r\n%s\r\n" \
      "POST /${x3} HTTP/1.1" \
      "Host: ${x2}" \
      "Content-Type: application/json" \
      "Content-Length: ${#x}";
      if [ $n -gt 1 ];then
      printf "%s\r\n\r\n%s\r\n" "$y: keep-alive" "$x";else
      printf "%s\r\n\r\n%s\r\n" "$y: close" "$x";fi;
      export n=$((n+1));
      done;
      if [ $n -gt 1 ];then
      printf "%s\r\n%s\r\n%s\r\n" \
      "GET /robots.txt HTTP/1.0" \
      "Host: ${x2}" \
      "$y: close";fi;
      )

Data to be posted may be stored in a one-line file named "1.txt"

    cat > 1.txt
    { "username": "YOUR.USERNAME", "password": "YOUR.PASSWORD" }
    ^D

Examine the request

    post api.ctl.io/v2/authentication/login < 1.txt

POST the request

    post api.ctl.io/v2/authentication/login < 1.txt|nc -vvn 127.1 80

Alternatively, data to be posted can be read from stdin

    echo '{ "username": "YOUR.USERNAME", "password": "YOUR.PASSWORD" }' \
    |post api.ctl.io/v2/authentication/login \
    |nc -vvn 127.1 80

If the data to be posed is JSON formatted as multiple lines such as

    { 
      "username": "YOUR.USERNAME", 
      "password": "YOUR.PASSWORD" 
    }

then

    (tr -d '\12' < 1.txt;echo) \
    |post api.ctl.io/v2/authentication/login \
    |nc -vvn 127.1 80

For TLS we use proxy listening on 127.0.0.1, e.g., stunnel, sslsplit, haproxy, etc.

    cat > 1.cfg
    pid=/tmp/1.pid
    [ x ]
    accept=127.0.0.1:80
    client=yes
    connect=64.15.182.200:443
    options=NO_TICKET
    options=NO_RENEGOTIATION
    renegotiation=no
    sni=
    sslVersion=TLSv1.3
    ^D

    stunnel 1.cfg

There is a different way, without curl or other curl-like programs, without Python, Go, etc. It is smaller and faster, IME as a non-developer.

   post api.ctl.io/v2/authentication/login < 1.txt|openssl s_client -connect api.ctl.io:443 -ign_eof

post is a 702-character shell script. printf is a built-in.

      #!/bin/sh
      (  
      y=Connection;n=0;while read x;do
      x1=${1#*//};x2=${x1%%/*};x3=${x1#*/};
      x=$(printf "%s" "${x%%:*}:";echo "${x#*:}");
      if test x"${x3}" = x"${x2}";then x3="";fi;
      printf "%s\r\n%s\r\n%s\r\n%s\r\n" \
      "POST /${x3} HTTP/1.1" \
      "Host: ${x2}" \
      "Content-Type: application/json" \
      "Content-Length: ${#x}";
      if [ $n -gt 1 ];then
      printf "%s\r\n\r\n%s\r\n" "$y: keep-alive" "$x";else
      printf "%s\r\n\r\n%s\r\n" "$y: close" "$x";fi;
      export n=$((n+1));
      done;
      if [ $n -gt 1 ];then
      printf "%s\r\n%s\r\n%s\r\n" \
      "GET /robots.txt HTTP/1.0" \
      "Host: ${x2}" \
      "$y: close";fi;
      )

Data to be posted may be stored in a one-line file named "1.txt"

    cat > 1.txt
    { "username": "YOUR.USERNAME", "password": "YOUR.PASSWORD" }
    ^D

Examine the request

    post api.ctl.io/v2/authentication/login < 1.txt

POST the request

    post api.ctl.io/v2/authentication/login < 1.txt|nc -vvn 127.1 80

Alternatively, data to be posted can be read from stdin

    echo '{ "username": "YOUR.USERNAME", "password": "YOUR.PASSWORD" }' \
    |post api.ctl.io/v2/authentication/login \
    |nc -vvn 127.1 80

If the data to be posted is JSON formatted as multiple lines such as

    { 
      "username": "YOUR.USERNAME", 
      "password": "YOUR.PASSWORD" 
    }

then something like

    (tr -d '\12' < 1.txt;echo) \
    |post api.ctl.io/v2/authentication/login \
    |nc -vvn 127.1 80

For TLS we use proxy listening on 127.0.0.1, e.g., stunnel, sslsplit, haproxy, etc. This way we only ever have to type a single address and port, i.e., 127.1 80, or short alias for the hostname, e.g., echo 127.0.0.1 p >> /etc/hosts.

    cat > 1.cfg
    pid=/tmp/1.pid
    [ x ]
    accept=127.0.0.1:80
    client=yes
    connect=64.15.182.200:443
    options=NO_TICKET
    options=NO_RENEGOTIATION
    renegotiation=no
    sni=
    sslVersion=TLSv1.3
    ^D

    stunnel 1.cfg

Apologies for unintentional duplicate. Happens sometimes when I forget to change default cache settings in text-only browser.

Reply on news.ycombinator.com