Colorizing Stderr: racing pipes, and libc monkey-patching
repl.it
repl.it
I looked into accomplishing the same thing with URxvt's pre- and post-text rendering hooks, but couldn't get it to work.
At repl.it we don't put it in our .bashrc or anything like that, we call the wrapper program manually when we launch the user's binary.
I think the best solution here is probably to avoid `sudo -E`, and for `stderred` to be patched to not perform any monkeypatching when `uid == 0`.
IMO this should be treated as a bug in initramfs-tools.
- If the error message has ANSI escape sequences (like gcc does), then you get an ugly initially-red output line. - You can do fancier styling with CSS, since the DomTerm escape sequence creates an element explicitly marked as error output, which is more semantically meaningful than "red".
$ somecmd 2> >(someutility)
Should do roughly the same thing. To highlight stderr with red color:
$ somecmd 2> >(perl -ne '$|++;print "\033[31m$_\033[00m"')
I assume you could roll that into a bash function or script easily enough.
Here's another approach too: https://github.com/dmoulding/hilite/blob/master/hilite.c
Ironically it’s one feature I’ve never advertised because I thought it would irritate other users besides myself. I had no idea there was a demand for that kind of feature.
As an aside, I also started work on a feature in the shell to strip all SGR escape sequences (colour and text formatting) for people who prefer their shell output text only. But that proved a little more problematic because it proved impossible to reliably differentiate between processes that expect a TTY (eg top, lynx, etc) and thus would need to bypass the SGR stripping wrapper, from the processes that don't.
The first solution (with the JSON adaptor) is the best solution, the only problem is the implementation (because it doesn't make sure that everything that goes to stdout/stderr goes through the JSON adaptors)
You could fix this problem by using a little C program to do the JSON wrapping.
- The C program would start by setting up two pipes, let's call them pipe_stdout and pipe_stderr.
- Then the C program forks
- The child replaces file descriptors 1 & 2 with the write end of the pipes and closes the read ends.
- The parent process closes file descriptor 0, closes the write end of the pipes, and calls select() on the read end of the pipes
- The child process calls execve to start the interpreter. All output to stdout/stderr now goes into the pipes to the parent process.
- the parent process reads data from the pipes, wraps it in JSON, and sends it to stdout.
Is there a reason why you didn't go this route? This way you should have minimal interleaving, and there's no way the interpreter writes data directly to stdout breaking the JSON stream.
It is really confusing to the user to have the prompt in the middle of their output.
What tricks do you suggest to prevent interleaving? It's not clear it's possible at all.
There's no guarantee of ordering between stdout/stderr, so line buffering them, or displaying them in separate UI sections, or even doing some non-blocking reading to flush one fully before flushing the other, should be sufficient.
That's a non-starter. We think barring a great UX improvement, the environment should look as predictable and as close to a local setup as possible.
If line buffering isn't enough, you can use heuristics around read sizes and non-blocking reading to guess whether a given write to the pipe was intended to be done as a block.
There absolutely is. If the two file descriptors refer to the same open file description or the same pipe/socket/character device, then the ordering of write syscalls is preserved--and that is the default for terminals.
Python by default starts buffering output when it's not connected to a PTY, which causes a lot of issues with interactive output.
besides, what happens to colourisation of output from stderr when a process gives useful color output?
UX is not really my remit (you could maybe guess :) but i’d greatly prefer bold, or stdout as grey, stderr as white - only if no colour control codes are present in the written message.
still, kudos for monkey patching libc :)
It can be quite hard to read deep red text on a black background. You don’t always have enough access to systems to change the default coloring to prevent this.
So I guess my question is, why can't whatever it does to handle the ordering and merge the two be replicated in a small program that reads two names pipes and does the same but with extra colorizing?
Ah, that's what I was missing. Thank you, that makes sense.
* https://unix.stackexchange.com/a/434839/5132
Second: Even as an example that read() implementation is exceedingly bad. It is stack frame corruption waiting to happen.
Third: This is another example of hardwiring control sequences for a particular class of terminal and not respecting TERM=dumb. It probably won't be a problem for the WWW site, but it will for the underlying general-purpose tool.
Wouldn't need any LD_PRELOAD tricks...
We're keeping an eye out in the beta. You can try it by going to your account and adding the explorer role.
Leave it up to the CLI to decide what colors to use.
#include <stdio.h>
int main(void) {
/* Open third stream. Stream must be opened in shell using 3>... */
FILE *user = fdopen (3, "w");
/* If third stream is not open, then print to first stream. */
if (!user) user = fdopen(1, "w");
printf("Output.\n");
fprintf(stderr, "Error!\n");
fprintf(user, "Important message to user!\n");
fclose(user);
return 0;
}
$ gcc test.c -o test
$ ./test
Output.
Error!
Important message to user!
$ ./test 1>out.txt 2>error.log 3>/dev/tty
Important message to user!Secondly, it doesn't matter what's right or not, the reality is CLIs (that you didn't write) generally use stderr for progress so you can't assume everything on it is an error.
DESCRIPTION
Under normal circumstances every UNIX program has three streams opened for it when it starts up, one for input, one for output, and one
for printing diagnostic or error messages. These are typically attached to the user's terminal (see tty(4)) but might instead refer to
files or other devices, depending on what the parent process chose to set up. (See also the "Redirection" section of sh(1).)
The input stream is referred to as "standard input"; the output stream is referred to as "standard output"; and the error stream is
referred to as "standard error". These terms are abbreviated to form the symbols used to refer to these files, namely stdin, stdout,
and stderr. *debug-io* - bidirectional, for interactive debugging
*error-output* - output, for warnings and non-interactive error messages
*query-io* - bidirectional, for asking user questions and reading answers
*standard-input* - stdin
*standard-output* - stdout
*trace-output* - for tracing functions and timing execution
It was IMO a good idea, and I wonder how the world ended up adopting just three standards streams in the end.
Seems to me that you could just look at how SSH does it ?
ssh localhost -t 'echo hi 1>&2' 2>test
ssh localhost 'echo hi 1>&2' 2>test