Can't we define what behavior should a modern terminal have with these already existing tools instead of inventing a brand new wheel?
It reads like what you get when someone has a poor idea of what 2D UI requirements are (alignment, relative positioning, layout, out-of-band image delivery, updates, animation, ...) and then has to retroactively fit them in one by one.
And then the mechanisms it has for doing so read like they are a poor fit for how you want to use them, requiring weird global bookkeeping/state, and sound like a nightmare without a proper API around it, abstracting it away.
Aka reinventing all the lessons of 30 years of UI development from scratch.
From what I remember sixel is _worse_ but that doesn't really change things.
Your only actual complaints were "weird global state" and "no API". I look forward to seeing you design a protocol that involves two way communication between two unrelated entities that avoids some shared state between them. As for no API, this is a protocol specification. if you need an API so badly go browse NPM.
Pixel aspect ratios are not always 1:1. Fonts have character height to width ratios which are all over the place and a terminal which must adapt any character cell to any region of an image pixel perfectly is way more difficult than it needs to be. Unless you're Casey Muratori, probably. then there are things like display scaling, which, ideally, the terminal application itself shouldn't need to know anything about in order to render correctly. What good is a terminal that renders pixels exactly if the OS can then magnify the window so that you lose the pixel "perfection"?
I don't like the Kitty protocol described in this post, either. The Windows Terminal people are thinking about this, believe it or not, and they are the ones that convinced me that existing methods are insufficient. We need yet another standard for doing this. One that can adapt to video display technology of the 1990s, at least.
> I don't like the Kitty protocol described in this post, either. The Windows Terminal people are thinking about this
I've been part of these discussions (mostly from the VTE side). I agree sixel, ReGIS, and Tek are "legacy" and have been implemented in such ways it's a miracle that they ever worked, and what you see on one brand of terminal is entirely dependent on the positions of the planets when the code was written and when the image is rendered and might not resemble at all what you'd see on another brand unless by coincidence ;-P.
Which is kind of why I brought up DPS - it's the only one that takes pure geometry and generates pixels (Tektronix does so as well, but it's very limited, and doesn't really like to be anchored to the text screen).
Very different from VT100-compatible communication, though, but maybe it can be encoded to fit that mold.
Nothing is wrong with them: Sixels for example work perfectly for all the tasks I throw at them in my terminals, from showing pixel perfect images to playing videos with mpv.
> Can't we define what behavior should a modern terminal have with these already existing tools instead of inventing a brand new wheel?
Actually, something is wrong with these already existing tool: they are preexisting, and therefore someone else idea. If you don't invent a new wheel (on which you can take credit for), you have to recognize that 40 year old technology is now perfectly adequate.
I think it wasn't ideal in the 1980s due to the bandwith requirements, but in 2023, if I can play videos with mpv, I think that's enough for most use.
I think the tech has been adequate for some time (at least about 15 years), and I can freely recognize that both because I have no horse in the game, and I care more about actual use than having/creating a shiny new tech.
Inventing a better wheel is the perfect way to avoid confronting such issues: you can argue your new wheel is better, and technically it will be true, even if in practice it's no different that the old wheel, in the sense it allows your vehicle to go forward for all the practical measures you care about.
There may also be a loss-aversion effect: if what you invented that was great is no longer needed, it may be painful to recognize it. You may not want to go for what's technologically not as nice as your invention, but that can now do the job just as well as your invention did, thanks to Moore's law.
Some people also just want to see their favored pet idea to win - unfortunately, they pet idea is only theoretical, or if it exists, it has a very small install base: since sixel is "adequate" or "sufficient", blocking/sabotaging sixel support is the logical action to prevent it from becoming established through network effects.
Put all these people together, add a little handwaving about how old formats can't work, and you get the current situation.
BTW here's a project I'll try to resurect: rending X applications ... in sixels: https://github.com/csdvrx/xorg-sixel
So you could have graphical applications in your terminal! I've just up. I've only uploaded the picture as I'm still fighting with the code but what you see is Xeyes show inside wezterm thanks to the sixel output of Xorg.
For terminals that don't support sixels (yet), sixel-tmux will convert them "in flight" into derasterized graphics: see https://github.com/csdvrx/sixel-tmux/
I'm sure this tech stack (X apps -> Xsixel -> sixel-tmux -> text) will make many people cringe, but in terms of functionality, it works (latency, bandwidth etc are sufficient for my uses) and if you have sixel support in your terminal, you get pixel perfect results.
FWIW I use and love Hyprland, but I want to use portable graphics apps more than I care about the underlying tech being better.