Hacker News Reader

Comment by skerit | Hacker News Reader

I see Ghostty does not support (and does not plan on adding support for) Sixels, instead preferring the Kitty image protocol.

Now if the Kitty image protocol is so great and the Sixel stuff is so bad, ~~why is it only used in Kitty and Ghostty?~~

*Edit: it's also supported in Konsole, WezTerm, ... but still I'm interested in why we have 2 competing protocols right now.

View on HN

Sixel came earlier, and already fulfilled the basic requirement of "put pixels on screen in a single well-defined format" (something not even iTerm2's protocol does.)

Kitty is a lot more complex: it accepts five different encodings, has three different ways to load the data, supports animations, etc. So it's no wonder only a few terminal developers had time to implement it.

See also: https://github.com/veltza/st-sx/issues/1#issuecomment-190272... 5000 lines (Kitty) vs 1000 lines (Sixel) even though the Kitty patch is just a "subset".

> Now if the Kitty image protocol is so great and the Sixel stuff is so bad, why is it only used in Kitty and Ghostty?

Images as in "pictures" or is that something else? I'm using Alacritty, and I don't think I've once thought "I need to see this image inside the terminal" and I do deal with images and frames from videos a lot. Probably if I saw it being added to Alacritty I'd think it was adding unnecessary bloat, so I wouldn't be surprised not every terminal is rushing to implement it.

Or I completely misunderstand what you're talking about.

I run Kitty and use this feature regularly. Most of the time, I rely on it within Yazi [1], a TUI file manager, but I can also display plots within the Julia REPL, thanks to the KittyTerminalImages.jl package [2]. It's even more crucial when I'm navigating a remote directory and need to check an image file, as I usually have timg [3] installed on those servers. Once you discover how valuable this is, it becomes a permanent part of your workflow.

[1] https://yazi-rs.github.io/

[2] https://github.com/simonschoelly/KittyTerminalImages.jl

[3] https://github.com/hzeller/timg

Definitely. I use KittyTerminalImages.jl often, and also the image.nvim plugin for embedding images into a Markdown or other buffer in Neovim: https://github.com/3rd/image.nvim

I have to say, (caveat, I have not tried any of these yet) that I am intrigued by all these features like graphics being added to terminals. It feel like exploring an alternate timeline 1990s where the GUI “lost” — of course in the absence of a successful Windows and Macintosh, terminals would have naturally gained these graphical abilities 30 years ago.

That alternate timeline started in 1982 with the Blit terminal. [0] It was a GUI made of overlapping terminals that had a serial graphics protocol.

[0] https://www.osnews.com/story/26315/blit-a-multitasking-windo...

The horrible kludge in Kitty of embedding pathnames inside VT-100 codes and making the terminal emulator delete the file when done is really, really far from the alternate timelines we once had.

Plan 9 says you're welcome to explore more. Here's a taste: Starting a "GUI app" in a "terminal" puts the GUI "window" in the pane where the terminal was.

It’s pretty nice to ssh into a remote host and plot some data there without needing either X forwarding, or dumping to files and rsync’ing, or similar workarounds.

Yes, pictures. It's quite useful. Opening images on remotes for one. Viewing plots arbitrary python scripts create for another.

Off the top of my head.

The alacritty maintainers reject any image protocols as unnecessary. I am fond of images in terminals, but I gotta say that I respect their decision, very good call. Not every terminal emulator should do the same.

I didn't know, but I'm happy to hear we seemingly are aligned regardless :) Thanks for the additional context!

Viewing an image in a terminal can be really handy for debugging ML systems that use images or bitmaps. You can also paste images directly into claude code as context.

Once while working on a daemon that did both ML and DSP on live audio I added the ability to play sounds and display spectrographs of in-memory audio data at various points of the internal pipeline to debug an issue that would have been difficult otherwise. Way quicker than dumping WAV files to view externally.

I'm not particularly fond of displaying images on the terminal in ways that would resemble a GUI.

That said, augmenting a shell-based workflow with tidbits such as this had me sold:

https://xcancel.com/thingskatedid/status/1316074032379248640...

https://xcancel.com/thingskatedid/status/1316075850580652032...

To achieve that, either Sixel or Kitty protocol is fine. IIRC Sixel works over SSH without any fuss, dunno about Kitty.

I can confirm that I just do it over ssh fine all the time -- gnuplot, img2sixel as an "image-cat", etc. (`st` with patches to add sixel support as discussed in various places in these comments.)

I’ve found it useful, when paired with a terminal file manager, to preview graphics in the terminal.

It would be nice if matplotlib or Octave could display pretty plots and figures on a remote server, in the terminal.

Images in a terminal emulator is neat for stuff like `ranger`

I think it's because Kitty and Ghostty are the newest terminals, so they came up with new modern options and solutions.

More than two, e.g. there’s also the Inline Images Protocol supported by iTerm2 and WezTerm.

Kovid documented his rationale at some length here: https://github.com/kovidgoyal/kitty/issues/33

FWIW The definitive thread on "images in terminals" is probably found in these threads:

https://gitlab.freedesktop.org/terminal-wg/specifications/-/...

there's lengthy discussion from just about everyone at this point in those threads, about why images in terminals is Hard

IMO none of them are particularly useful. Sixels is hilariously inefficient. Kitty is slightly better because you can send data as PNG, but ... you have to send image data as PNG!

I wish there was a high performance way of remoting graphics over SSH. How cool would it be if you could SSH to a remote machine and it just showed you the remote desktop in the terminal itself? No messing around with port forwarding, weird X servers, etc.

I think probably that requires a full fat video codec like H.264 to work well though. Or maybe RDP?

Probably too many GUI naysayers and "What's wrong with remote X?" for this to ever happen though.

One of my pet proof of concept projects is figuring out how to ergonomically tunnel web apps over ssh without needing to fiddle with listen ports and port forwards. First attempt was to push http2 over stdio which actually worked, but it didn't really integrate well with terminal use. Currently I think similar approach to X forwarding makes sense, where SSH forwards one unix socket over ssh connection and then the applications can connect to that socket and put http2 traffic over that connection. Basically the idea is to make webapp tunneling as easy as X tunneling, so you can just type command in shell and (browser) window would pop open without any extra hassle. The neat thing is that because http2 has persistent connections with multiplexing etc built in, it works really well for this sort of hack; plain http 1.0 would be far more annoying.

Check x2go.

> I wish there was a high performance way of remoting graphics over SSH. How cool would it be if you could SSH to a remote machine and it just showed you the remote desktop in the terminal itself? No messing around with port forwarding, weird X servers, etc

I think there's at least three different experiences here, and they're all valid, but I don't know what you really want.

A) remote desktop --- connect to a fully formed desktop environment (with SSH to authenticate, I guess?), possibly persisted and/or shared so you can connect back and get into the same place or share with another user?

B) run a program remotely and display it on your local terminal; essentially remote X, but I gather you're looking for more performance and maybe some other nice to haves? Maybe you want to transport audio too... Maybe you don't want the crap experience remote X has become since app developers don't spend any effort on it and you kind of get what you get, which is a lot of jank.

C) images in the terminal, with high performance. PNG should be ok for that, right? Maybe an extension for lossy compression might be nice depending.

Yeah I want all of those. Only the third one really works at the moment but it's also the least useful. The second one I think would be the most transformative because then all the TUI apps people use like htop, ncdu and so on can have decent graphics.

At that point you're really better off using some other remoting protocol instead of trying to tunnel it all over a terminal session. There's nothing left of the original terminal.

There is though - the ssh authentication and connection is already handled, and I'm already in a terminal. When I quit the app or session I'm back in the terminal.

If it worked it would greatly reduce the hassle.

Think about all the TUI apps that exist. They're useful because they're convenient when working in a terminal, not because they look like shit.

If I want to view an image file on a remote machine, and all I have is ssh... I just connect to that machine with filezilla and click on whatever files I want. I can even open files that aren't PNG! Even files that aren't even images at all. Mindblowing.

A terminal with in-band graphics primitives is called an RDP client.

We've had graphics terminals since RIP BBS's and even before that. If they were actually useful enough to be worth the bother, then we'd all have been using them all along and there wouldn't be posts like this.

It's not a case of there's this awesome idea that just for some reason no one knows about. No, it's just not that awesome of an idea. It's not harmful so it doesn't bother me that most xterms support tektronix graphics, it's just a gimmick of no real value. It's a solution to no problem.

Don't believe me? When was the last time you used passthrough printing? Or saw it being used even in some place where they do actually need to print? The terminals all still support it. It's just a thing that you don't need to do in-band in a tty, and today there is no reason to bother doing it that way even though you could. It's not better and does not solve a problem.

With sshfs and 'rclone mount' you forget the shell and everything it's a filesystem.

I don't think viewing files is the most compelling use case for this.

> I just connect to that machine with filezilla

"Just" carries a lot of hassle, and this only applies to viewing files.

What you are looking for is forwarding an X session via SSH, and that has been supported since the dawn of time.

> Probably too many GUI naysayers and "What's wrong with remote X?" for this to ever happen though.

Is there a wayland equivalent?

Closest is wprs

https://github.com/wayland-transpositor/wprs

I have yet to use it though because Wayland still doesn't work properly for me (it doesn't restore the desktop properly after sleep) so I'm still on X11... without compositing... because KWin's compositor causes random freezes.

Yeay, Linux on the Desktop.

Yes, waypipe

drawterm under Unix clients and 9front cpu connections; but that's Unix Philosphy 2.0.

>why is it only used

Sixel has existed for 40y, Kitty protocol originated and initially made for a single project (Kitty) few years ago.

>why we have 2 competing protocols

Well, iTerm2 also got its own image protocol (predating Kitty's but basic, only meant to show images rather graphics in general) that has been adopted by few other projects. Terminology also got something on its own, and urxvt can show images by setting the background image.

Curious, what do you do with this?

mitchellh says no plan to support Sixel:

https://xcancel.com/mitchellh/status/1985432954089455856#m

to be fair, pretty much anything would be better than sixels

Reply on news.ycombinator.com