True Colour support in various terminals | Hacker News Reader

True Colour support in various terminals

gist.github.com

74 pointspmoriarty27 comments

You know I tried to hack to play a gif upon SSH login, /etc/issue.net does not accept ANSI chars, but /etc/motd does.

So I mkfifo /etc/motd, use some monkey patched shell commands to stream gif to /etc/motd, tried SSH login, the animation totally works, adjust the sleep you get different speeds.

I was totally satisfied with this and logged out of my VPS server. Oh wait....

Since my scripts was not in a loop, so it only feeds data into /etc/motd once, after I logout every SSH session, I can no longer login. Even local login fails because Linux would stuck forever reading the fifo /etc/motd.

I had to mount the disk into another machine and restore /etc/motd to normal.

Lessons learned, don't mkfifo /etc/motd.

Isn't motd only displayed on interactive login? You could probably have gotten back in if you did something like ssh foo "sudo rm /etc/motd"

I use this fairly often, when working on the image parts of our API (image caching, map tiles etc).

I tweaked the "img" script linked from the article to download and display an image, so I don't need to leave the terminal.

  img http://sipi.usc.edu/database/preview/misc/4.2.03.png

My result is this, in Konsole (KDE): http://i.imgur.com/34fjSas.png

Sad infodrop: the main reason this is a nonevent is because ncurses encodes fg/bg colors in an unsigned int. :(

I don't understand the problem, could you please elaborate?

Sorry, sure.

Emitting colors to terminals is trivial, but ncurses - the dominant text UI library (since the 1980s!) - needs to hold the color information for the "windows" or frames that it maintains for you so that it knows how to repaint things when you switch from one view to the next (eg, if some kind of popup opens and then gets closed, ncurses maintains state about what was underneath so it can redraw it for you. This was novel in the 80s because of the number of competing terminals; see /etc/termcap for a very good idea of why everyone was happy to let a library do this for you).

* The following is my understanding of the situation; I welcome corrections *

Unfortunately, ncurses encodes the foreground and background as an unsigned int, which only stores 0-255. That bit I said before about ncurses existing since the 80s - unfortunately that's more important than I wish it was; ncurses ("new curses") is merely the OSS variant you find on Linux, it has existed on UNIX (as the "curses" library) in many forms. For Reasons™, the commercial curses implementations of the day all tried to stay binary-compatible with each other. Of course, ncurses did the same.

And so, ncurses is pretty much based on a de-facto ad-hoc standard (AFAIK) that's not really ratified anywhere (it might be, not sure) and which specifies very strict in-memory data structures because of the way the API was originally built. Changing any of this would break binary compatibility.

IIRC, ncurses' author/maintainer, Thomas E. Dickey, hasn't really made any noises about for example adding a new function and doing some kind of backward-compatibility dance to allow ncurses to preserve but ignore the legacy color data. I'm not sure if this would actually be technically impossible or whether it's a case of "eeeeh, that doesn't sound like fun, I don't feel like doing that". I can understand the latter, but I definitely hope the former is not true, as it would be the death knell for truecolor for at least the next 10 years - ncurses isn't going anywhere anytime soon, and it would be the canonical way to get the capability popularized.

In case you were curious to see how much depends on ncurses on your system (uses ldd and assumes "libncurses"; unsure if Linux specific):

  (IFS=:; for x in $PATH; do ldd $x/* 2>/dev/null; done) |\
    awk '/^\//{FS=":";p=$1;}/^[ \t].*libncurses/{print p;}'

There are some obvious things in here like htop/top, alsamixer, dialog, vim, irssi, links, mc, etc; I assume most of what I'm seeeing is depending on ncurses as a result of pulling in another library.

Your package manager's "list packages that depend on this package" may produce faster results.

Hi i336_, I believe the problem lies not in the fact that there are only 256 values available in an "unsigned int" (an int is at least 16 bits wide). The point is that ncurses requires to work with color pairs, which encode both the foreground and the background color in an index. This was made in order to preserve compatibility with some old HP terminals that were not able to set the two colors separately (using "setaf" and "setbf", like Tektronix terminals used to do).

The problem with direct color is, if you have N colors, the number of pairs is N^2. When N=2^24, indexing pairs would require a variable of size 48 bits at last, but the current ncurses' ABI doesn't allow this.

I believe that almost every modern terminal doesn't use paired colors anymore, but ncurses has always tried to provide compatibility with a wide set of terminals, and I understand their lack of enthusiasm in supporting this.

It would be great if ncurses supported more colors, but doesn't exist any other library except ncurses that one could use for cli programs?

This sounds like an opportunity for somebody write a backwards-compatible ncurses replacement in a modern systems language. I promise to try using whichever language manages to do this first.

Or even in C! The (n)curses API is ancient -- it dates back to the early 1980s, and has not changed much since then. We've learned a lot about API design over the last 30+ years; we could do much better today.

Fair point, and today's C is far from what it was in the '80s! Regardless, this is an excellent project for a programming language shootout --- ncurses is well overdue for a rewrite, and I'd love to see the C++17 partisans take on the Rust fans, the Go-ers, and the C diehards.

Serious question: will terminals ever support P3 color? Maybe that's too much to ask for?

https://webkit.org/blog/6682/improving-color-on-the-web/ https://webkit.org/blog-files/color-gamut/comparison.html

Why P3? It is better to go directly to Rec. 2100 with HDR which is more future-proof.

It only improves images, no?

It's a wider color space. It improves the color range of everything, assuming the source hasn't been flattened to SRGB.

Well, yes, but what are the benefits for text over 24-bit colour? To have two even less different shades of a colour adjacent to each other? For images there's a very clear benefit, but for a text frame buffer I'd say not so much.

What you are talking about is bit-depth which is not related to color spaces (although for wider color spaces you need higher bit-depth to avoid banding).

See my comment above: it would allow more precise specification of a color by avoiding the need of indexing an implementation-dependent palette.

citrusui originally asked about DCI-P3 color space, ygra then confused this wider color space with higher bit-depth. Now you want to further confuse everyone by referring to your comment about advantages of 24-bit RGB over paletted colors which is completely irrelevant to the original comment about DCI-P3 color space. DCI-P3 contains colors which are outside of the gamut representable by the 24-bit RGB (which uses sRGB color space).

Sorry, it wasn't my intention to hurt you with my comment.

No problem :)

What I think is really interesting in this is not the possibility to use 16 millions colors per se, but rather the ability to directly specify the RGB components of the color to use.

The current approach used by terminfo and ncurses is to refer to an index within a palette, which has the drawback of giving little guarantees about the final shade of the color. (Look at the color table here [1] for a comparison among different terminals; here it is stated that xterm changed the RGB components of two shades of blue from one version to another!)

Allowing true colors in terminals would make curses programming very similar to what you do in HTML and CSS, where you directly specify the color you want. It would surely be a nice improvement.

[1] https://en.m.wikipedia.org/wiki/ANSI_escape_code

Except you can already query/set the color palette to arbitrary RGB values with control sequences (at least with xterm, rxvt-unicode, and I imagine others).

As far as I've been able to tell, "truecolor" terminal support is about simultaneous display of a range of colors beyond what fits in a palette. Which, eh...seems a bit much for a terminal.

You're right, but I think that truecolor support is easier to program.

I vividly remember when I wrote DOS programs which used VGA/SVGA graphics in the early 90s; switching from palette-based colors to RGB colors was something I really enjoyed! Back in the 90s, loading a 256 color image that used its own palette into DeluxePaint would change the colors of DP's interface too.

Something similar would happen today under tmux, I think: if two side-by-side panels contain two different programs, and one of them reprograms the palette, the other program would be affected too.

Here is an image printing function for bash/zsh:

  function img { for image in "$@"; do convert -thumbnail $(tput cols) "$image" txt:- | awk -F '[)(,]' '!/^#/{gsub(/ /,"");printf"\033[48;2;"$8";"$9";"$10"m "}'; echo -e "\e[0;0m"; done ;}

Example: http://i.imgur.com/d5RzWAC.png

I think yours has stretched the image vertically, though it is simpler than the one linked in the article [1]. That uses "Upper Half Block" characters to give square pixels, and sets a foreground and background colour -- doing two rows at once.

With that, I see this: http://i.imgur.com/uqCHAmF.png

[1] https://git.gnome.org/browse/vte/tree/perf/img.sh?h=vte-0-36

Just a note that this requires imagemagick.

Reply on news.ycombinator.com