Hacker News Reader

Comment by TekMol | Hacker News Reader

I think this is a bad idea. Trying to make programs intelligent and have them behave in unexpected ways usually is.

It will make curl fail if a 0 byte is outputted within the text. There are many reasons why this will happen. Software errors, UTF-Encoding, special use cases...

In fact, I run into this problem with grep from time to time and it is super annoying:

    > grep somefile.txt stuff
    Binary file somefile.txt matches

So this change makes the code base more complex and the behavior fuzzy.

I prefer elegant software that behaves predictably.

View on HN

A NUL byte can't print in a terminal or show in an editor, so treating it as binary is a sensible thing to do when you are asking cURL to print it to your terminal. It is no longer text if a NUL byte is present (unless you use UTF-16/32, in which case, WHY?!).

I assume that you also inserted a NUL byte into your somefile.txt to make grep treat it as binary.

cURL behaves predictably now as well as before. It will warn you if you try to print unprintable data to your terminal, but can be told to ignore it, and won't affect output redirects.

Many legacy and contemporary programs change their output behaviour based on if they're outputting to a tty or to a pipe. Off the top of my head, ls (as old as you can get) and git (new) do not colourise their output by default for pipes, but do colourise for terminals. This is just curl trying to be a good citizen on the terminal, for which there is plenty of precedent.

Not just colour. The actual formatting and layout of ls output changes, too.

And it's sometimes helpful, but sometimes a big pain because you just wanted to pipe it into less.

this drives me slightly nuts but I get why people do it.. I really wish for some kindof smarter negotiation between pipes.

I often end up doing things like: watch --color juju status --color or "ls -c | less -r", etc.

> Software errors

Which could mess up your terminal, so curl is doing the right thing.

> UTF-Encoding

On Unix, terminals would be completely broken if they used UTF-16. Other Unicode Transmission Formats don't use null bytes.

> special use cases

Such as?

> In fact, I run into this problem with grep from time to time and it is super annoying:

That's not applicable here, because grep only prints parts of the file, while curl prints out the whole file.

Everyone working with binary in the terminal should be told about about stty sane. Very useful.

There is also reset(1).

https://linux.die.net/man/1/reset

My coworkers and I used curl, and the linux kernel, as part of interviews. The candidate was asked to download the source to the Linux kernel during the interview; occasionally they would curl it to the tty. (We don't ding them for this, as I do it all the time myself… In fact, if anything, it provides a great opportunity to see if they know about reset, or the details of their mistake (e.g., why did that have so many odd effects on the terminal? why did the title change? etc.)) Surprisingly, the binary output would screw up tmux beyond repair by reset. (There would be characters printed outside the "terminal"'s area, for example. We suspect tmux has a few bugs because of this.)

Fuzzing tmux's terminal would probably lead to some interesting bugs.

I have to agree. curling binary to shell is awkward, but nothing that can't be fixed with a `reset`.

Changing behavior of something that is as popular as curl to protect edge cases is strange, to say the least.

Are you sure awkward is the worst that can happen?

http://www.openwall.com/lists/oss-security/2017/05/01/13

http://www.securityfocus.com/bid/6940/

I am sympathetic to your comment because it feels like we are babying users. To me, this is no different than when FreeBSD and GNU modified the rm command to prevent people from shooting themselves in the foot.

I thought they changed it to prevent buggy scripts from executing rm with empty variables? That's not necessarily the fault of the user.

> To me, this is no different than when FreeBSD and GNU modified the rm command to prevent people from shooting themselves in the foot.

Which was the right change. It prevents data loss for negligible added complexity.

Reply on news.ycombinator.com