Ledit: Simple, GPU-rendered, no bullshit text editor
github.com
github.com
I'm editing a file on an embedded system in a PuTTY serial window (115.2 kbps). I'm in a remote desktop connection over a VPN to get to that PuTTY window. (1920x1200 resolution). Everything is responsive; no complaints.
(I'm using BusyBox vi. OMG, it has poor optimization of refresh. I just switched to 9600 bps for fun and opened a small file in the editor. So most of the lines are filled with the tilde character, characteristic of Vi. When I type Ctrl-R to refresh the screen, BusyBox is repainting those ~ lines in their entirety. It's printing all the spaces! It has no knowledge of the "clear to end of line" or "clear from here to bottom of screen" escape sequences. It also scrolls the buffer by repainting, even single line scrolls when you move the cursor down from the bottom line or up from the top line. At 115200 bps, you don't notice any of this.)
Some web pages can't keep up with a moderately fast typist, even though it's all rendered locally.
It seems obvious that how quickly a web page can refresh is entirely down to the skill of the developers who created it. There's nothing inherently slow about a web page - plenty of pages update quickly enough. Browsers are fast. Some userspace code is not. If you're using a page that doesn't work fast enough that's down to the developer, not the web itself.
Of course back then you could also use emacs on a printing terminal, which surely won’t work any more.
Printing terminal mode with similar in that it showed only the line you were on, and, IIRC, optionally the line above or below. This worked pretty well in Emacs where search is extremely lightweight and is (for most people at least) the primary cursor movement tool. Of course deletion couldn't display properly so you would get a marker on the screen (#?) to indicate, and then ^L would draw the then current copy of the line.
You could write this support today, and perhaps someone has, but I wonder if it could possibly be worth it.
This same reason is why the unix terminal driver by default starts up with # as the delete character command and @ deleted the whole line. That line discipline dates back to Multics, which had more efficient I/O as the Multics hardware had channel controllers (all our I/O is done with channel controllers these days, though we don't call them that).
Actually, speaking of channel controllers: Emacs had a third mode: if you used the SUPDUP protocol (stood for super duper: roc 739, 746, 749) a bunch of the IO and screen update was done by your local host, not the remote machine on which you were running Emacs (or whatever). It implemented an abstract display and could even interpret a few commands to display the local buffer properly like a souped-up channel controller! This all predated Unix on the arpanet so later approaches went in a different direction.
But remember this was back when you could have dozens of people using networked computers with clock speeds in the hundreds of kilohertz or less, back before the cross continent backbone was upgraded to ~56 Kbps.
Instead, buy a high-refresh-rate monitor with good pixel response time and input latency. Probably 90% of the people reading this post are on a 60hz display, and many of those monitors may have noticeable input delay (Microsoft found that people can notice even just a few ms input/display delay.) Even just 75 hz or 100hz will feel noticeably nicer.
There are a couple of pretty inexpensive ~95hz monitors aimed at office users, but 2.7k ~150hz gaming monitors aren't that expensive if you shop carefully. Even nice ones with IPS and good color gamut/calibration.
Everything is nicer with higher refresh rates. Scrolling through web pages, file listings, editing text files, terminal sessions. Same is true of gaming mice; if you do a lot of mousework, a quality "gaming" wireless mouse is durable, has virtually zero input lag, great tracking, very accurate response to quick "flicks" letting you move windows and such around, and high polling rates. Default on most mice is around 120hz. Yes, you can tell the difference - though I have my mouse set to either 250hz or 500hz when not gaming, I forget which.
Edit: just to note that someone pointed out keyboards matter, too, and they're right Some keyboards can have over 50ms input delay, which when you think about it, is fucking nuts in the year 2022.
RTINGs tests for keyboard latency and their idea of "high latency" for a gaming keyboard can be 10ms. The difference between that and the fastest (1-2ms) keyboards is probably not worth selecting for over features like ergonomics and daily use factors.
RTINGs tests for latency, but I would caution folks to note that "high latency" to them is ~8 or 9ms, which as Dan's tests show, can be almost 10x better than some common keyboards out there.
However, the high-refresh-rate monitor also needs to be supported. I have a 144 hz monitor which works fine with my Windows PC, but when I connect it to my work Macbook, it goes back to 60 hz. In the MacOS display settings, there is a dropdown for the refresh rate, but it only lets me choose between 50 and 60 hz... :(
Currently I'm using mlterm which is slower than Kitty again by a significant margin, and I still don't notice the difference in day to day use.
(xterm, st and gnome-terminal are all slower than Kitty by huge margins; of the terminals on my system (I have a lot because I tested a bunch of them a while back...) there's a performance difference in terms of rendering throughput of at least a factor of 10, and yet even the slowest once are perfectly fine for most use other than the cat large file test)
Ever since the Amiga days text rendering at sufficient speed to bitmap displays has been a solved problem, with only scrolling accelerated by the blitter, and resolution and number of bits per pixel has increased at a much slower pace than CPU performance since then.
Most X terminals use one of ~3 different basic approaches to glyph rendering anyway (as shown by redirecting them through xtruss to log the X requests they make), and their throughput still varies from far faster than Kitty (as per rxvt) to far slower, so apart from possibly the fastest ones, most of the x terminals spends most of their time outside of their rendering code.
That's a tad too simple
I think the current implementation might have some bugs, but why not create an atlas? Especially for code, most of the characters will be in ASCII, and if they're not, you can always push what isn't dynamically as needed. Most of the rendering still occurs GPU side.
I suspect the post's code is also cheating on shaping, but even that seems like something you should be able to at least cache on the GPU?
Like the 98% of cases, the GPU, and if for the 2% oddball stuff, the CPU needs to step in, so what?
(Edit: perhaps I'm projecting the "wrong" part onto your comment; there's a lot of "why?" in these comments and the more I think about it, it honestly seems pretty sensible to push graphics to the *G*PU.)
That's not to say it's not impressive considering how simple it is.
I would also say, personally, there is something aesthetically pleasing about rendering vector graphics (fonts, SVG, etc.) on the GPU. “It’s a graphics processing unit, shouldn’t it be handling vector graphics as well?!” :)
People are definitely doing tons of vector rendering on the GPU. And the GPU is fantastic for vector graphics generally, mainly still because of the conversion to pixels. I posted links to Hughes Hoppe’s paper and the NV path rendering library, for example. Those can do font rendering as well, but I still wouldn’t use those for rendering fonts directly or for rendering text atlases because it would be difficult to implement.
I suspect the underlying question is “why a supposedly-3D-optimized GPU for a 2D task?” While it’s true that modern GPUs are 3D-optimized, that’s AFAICS because the decade-long lull of 1MP-or-so screens and ever-more-powerful desktop CPUs ca. 2000–2010(?) made CPU 2D rendering mostly “fast enough” so as the programmable 3D pipeline emerged the 2D accelerator from the workstation era died out, leaving perhaps only an optimized blit behind.
Compute- and power-constrained handheld devices and higher-resolution screens made developers (software and hardware) wake up in a hurry, but now, a redraw-everything-every frame, 3D-adapted graphics facility is what you have, so a redraw-everything-every-frame, 3D-adapted graphics facility is what you shall use. The much more parallel and power-efficient processor and wider bus is still easily worth it, if you spend the effort to wrangle it.
(It’s interesting to think what 2D-optimized hardware would look like. Do people know how to do analytic or even just AGG-quality 2D rasterization on a GPU? Or anything but simple oversampling.)
Not a graphics programmer, treat with a measure of skepticism.
Also, Pathfinder [2] has the ability to do most of its rasterization on the GPU.
I read somewhere that an experimental renderer for Google’s Fuchsia (Spinel?) [3] also does this.
1: https://github.com/linebender/piet-gpu
2: https://github.com/servo/pathfinder
3: https://www.tdcommons.org/cgi/viewcontent.cgi?article=1580&c...
None of them were text editors or terminal emulators.
Very few non-gpu accelerated terminals get anywhere near maximising performance and they're still fast enough that most people don't even notice because most applications, including most editors, once even remotely optimized themselves, don't tend to push even a non-accelerated terminal very hard.
Put another way: a typical test case of the performance of a terminal tends to be spewing log output or similar to it at a rate where the obvious fix is to simply decouple the text buffer from the render loop because nobody can read it all anyway (but really, a lot of the developers of these terminals should start by looking at rxvt and at least get to a decent approximation first before they start doing those kinds of shortcuts)
Fast enough terminal rendering was solved by the mid 1980s, and performance increases have outpaced resolution increases by a substantial factor.
Modern GPUs & so-called 3D APIs don’t really do all that much that is 3D specific. @greggman likes to talk about WebGL as a fundamentally 2D API https://webglfundamentals.org/webgl/lessons/webgl-2d-vs-3d-l...
The bulk of the 3D work is still 2D, filling all the pixels, rasterizing 2D triangles after they’ve been transformed, projected to 2D, and clipped. Shaders might have lots of 3D math, but that’s really just pure math and no different from 2D math as far as the GPU is concerned.
> Do people know how to do analytic or even just AGG-quality 2D rasterization on a GPU? Or anything but simple oversampling.
Yes, I think so - if you mean quads & triangles.
The problem, of course, is analytic filtering (even 2D) usually isn’t worth the cost, and that oversampling is cheap and effective and high enough quality for most tasks. (Still, adaptive oversampling and things like DLSS are popular ways to reduce the costs of oversampling.) The path rendering people think carefully about 2d path rendering though:
https://developer.nvidia.com/gpu-accelerated-path-rendering
And don’t forget mipmapped texture sampling, even in 2D, is better than oversampled.
A very common shader trick for 2d antialiasing that’s better than oversampling (not analytic, but sometimes damn close) is to use the pixel derivatives to compute the edges of a mask that blends from opaque to transparent in ~1-pixel. https://www.shadertoy.com/view/4ssSRl
I suppose the sensible answer is "do it in a compute shader if you care about pixel-perfect accuracy". Which you arguably should for 2D stuff, given that the overhead is low enough.
There were other hardware accelerations my memory is somewhat more vague on. I remember there were some color palate translation hardware, and hardware dithering.
There wasn’t any standard cross platform graphics api back then, but that’s more a statement about the era. Everyone wrote directly to the metal.
You can do all these things with compositing, though. A "sprite" is just a very limited hardware surface where compositing happens at scanout, and some modern GPU's have those too.
https://web.archive.org/web/20081209121702/http://www.vesa.o...
Also it looks like XV did video resizing and some color mapping (https://en.wikipedia.org/wiki/X_video_extension). Today’s GPUs are doing the rendering. Filling the display buffer using primitives and textures, and outputting the display buffer are different activities that are both done by the GPU today, but it sound like XV didn’t do the first part at all, and that first part is what we’re talking about here.
X without 2D acceleration was crawlish even under my AMD Athlon.
There are certain kinds of rendering and corner cases where CPUs can have an edge, but those happen in areas like scientific visualization or high end VFX, they don’t come up often in text editor rendering engines.
You can’t use your anecdote of 1 app that might have a broken implementation to question GPUs categorically. I’ve never noticed iTerm2 using significant CPU. My iTerm2 sits at 0.0% CPU while idle. Maybe your install is busted?
"The base motivation was just simply that i wanted to have a
look into OpenGL and doing GPU accelerated things, i did not
plan to create a text editor from the get go.
After starting to experiment a bit i decided to call this a
small side Project and implement a proper editor."On the other hand, the CPU is often busy. Particularly for a text editor, theres a good chance the user is running all sorts of CPU heavy stuff like code analysis tools, databases, compile loops etc.
For most text-heavy apps there's little reason for the text-rendering to be a bottleneck either way.
What does CPU rendering even mean? Is it sending per-pixel instructions to the GPU, or is it rendering a framebuffer in CPU ram, and transferring that to the GPU or something else? Pure software would be the latter (save pixels to a block of RAM, then transfer to the GPU), but the line today isn’t very clear because most CPU renderers aren’t transferring pixels, they’re calling OS level drawing routines that are turned into GPU commands. Most of the time, CPU rendering is mostly GPU rendering anyway.
> there’s little reason for the text-rendering to be a bottleneck either way.
Think about 4K screens, which is up to 8M pixels to render. Suppose you’re scrolling and want 60fps. If you wanted to rendering this to your own framebuffer in RAM, and you render in 24 bit color, then your bandwidth requirement is 8M * 3bytes * 60frames = 1.4GB/s bandwidth. It’s also 8M * 60 pixel rendering operations. Even if the per-pixel rendering was 1 single CPU instruction (it’s more, probably a lot more, but suppose anyway) then the CPU load is 0.5 billion instructions per second, which is a heavy load even without any cache misses. Most likely, this amount of rendering would consume 100% of a CPU core at 4K.
Anyway, there’s no reason to have the CPU do all this work and to fill the bus with pixels when we have hardware for it.
Text rendering is pretty much the best case for the CPU in this respect, because rendering inline to a buffer while processing the text tends to be pretty efficient.
> they’re calling OS level drawing routines that are turned into GPU commands. Most of the time, CPU rendering is mostly GPU rendering anyway.
This is true, and is another reason why it's largely pointless for text-heavy applications to actually use OpenGL etc. directly. But what I've said also applies in purely software rendered system. I've terminal code with control of the rendering path for systems ranging from 1980's hardware via late 90's embedded platforms to modern hardware - there's little practical difference; from the mid 1980's onwards, CPUs have been fast enough to software render terminals to raw pixels just fine, and typical CPU performance has increased faster than the amount of pixels a typical terminal emulator needs to push.
>> > there’s little reason for the text-rendering to be a bottleneck either way.
> Think about 4K screens, which is up to 8M pixels to render.
I stand by what I said. I'd recommend trying to benchmark some terminal applications - only a handful of the very fastest ones spend anywhere near a majority of their time on rendering even when doing very suboptimal rendering via X11 calls rather than rendering client side to a buffer. Of the ones that do the rendering efficiently, the only case where this becomes an issue is if you accidentally cat a massive file, and only then if the terminal doesn't allow.
> Most likely, this amount of rendering would consume 100% of a CPU core at 4K.
So let it. You're describing an extreme fringe case of shuffling near-full lines as fast as you can where you can still trivially save CPU if you care about it by throttling the rendering and allowing it to scroll more than one line at a time, which is worth doing anyway to perform well on systems under load or without a decent GPU, or just slow systems (doing it's trivial: decouple the text buffer update from the render loop, and run then async - this is a ~30-40 year old optimisation).
Yes, it looks ugly when you happen to spit hundreds of MB of near-full lines of text to a terminal when the system is under load. If you care about that, by all means care about GPU rendering, but in those situations about the only thing people tend to care about is how quickly ctrl-c makes it stop.
Try tracing what some terminal emulators or editors are actually doing during typical use. It's quite illuminating both about how poorly optimised most terminals (and text editors) are, and how little of their time they tend to spend rendering.
> Anyway, there’s no reason to have the CPU do all this work and to fill the bus with pixels when we have hardware for it.
A reason is simplicity, and performing well on systems with slow GPUs without spinning up fans all over the place, and that the the rendering is rarely the bottleneck so spending effort accelerating the rendering vs. spending effort speeding up other aspects is often wasted. Especially because as you point out many systems optimise the OS provided rendering options anyway. E.g. case in point: On my system rxvt is twice as fast as Kitty. The latter is "GPU accelerated" and spins up my laptop fans with anything resembling high throughput. Rxvt might well trigger GPU use in Xorg - I haven't checked - but what I do know is it doesn't cause my fans to spin up and it's far faster despite repeated X11 requests. A whole lot of the obsession with GPU acceleration in terminal emulators and text editors is not driven by benchmarking against the fastest alternatives, but cargo-culting.
It seems strange to demand evidence and benchmarking but end with pure unsupported hyperbolic opinion. If you can’t explain why a huge number of major text editors are now starting to support GPU rendering directly, nor why the OS has already for a long time been using the GPU instead of the CPU for rendering, then it seems like you might simply be ignorant of their reasons, unaware of the measurements that demonstrate the benefits.
I really don’t even know what were talking about anymore exactly, since you moved the goal posts. I don’t disagree that most of the time text rendering isn’t consuming a ton of resources, until it does. I don’t disagree that many editors aren’t exactly optimized. None of that changes the benefits of offloading framebuffer work to the GPU. The ship sailed already, it’s too late to quibble. I can’t speak to rxvt vs Kitty (you really need to do a comparison of features and their design decisions and team sizes and budgets before assuming your cherry picked apps reflect on the hardware in any way), but by and large everyone already moved to the GPU and justifications are pretty solidly explained and understood, evidence for these decisions abounds if you care to look for it.
> If you can’t explain why a huge number of major text editors are now starting to support GPU rendering directly
I did give a reason: It's largely cargo-culting. Some have other reasons. E.g. using shaders to offer fancy effects is a perfectly valid reason if you want those effects. Most of these applications do not take advantage of that. Some, like the linked one, are learning experiences, and that's also a perfectly fine justification. Very few have put any effort into actually benchmarking their code to a terminal which is actually fast - most of the performance claims I've seen comes from measuring against slow alternatives.
Hence why "cherry picking" rxvt: The point being to illustrate that when someone can't even match rxvt, which is open source and so easy to copy and at least match in performance, maybe they should actually figure out why rxvt is fast first before worrying about the specific low level rendering choice.
One of the things people would learn from that is that the rendering choice is rarely where the bottlenecks arise.
> then it seems like you might simply be ignorant of their reasons, unaware of the measurements that demonstrate the benefits.
I've done many enough code reviews and enough measurements and benchmarking of terminals and text rendering code over the years that I'm quite content in my knowledge that I know the performance characteristics of this better than most of those who have implemented the large number of poorly performing terminals over the years. That's not a criticism of most of them - performance on text rendering is simply such a minor aspect of a terminal because most of them are fast enough that it's rarely priority. What this has taught me is that the rendering itself isn't typically the bottleneck. It hasn't been the bottleneck for most such projects since the 1980's. You can continue to disbelieve that all you want. Or you can try actually looking at some of this code yourself and try profiling it, and you'll see the same thing. Your choice.
> I really don’t even know what were talking about anymore exactly, since you moved the goal posts.
I did no such thing. In the comment I made that you replied to first I made two claims:
1. that it is not a given GPU use will be faster.
2. that for most text-heavy apps there's little reason for the text-rendering to be a bottleneck either way.
I stand by those.
> I don’t disagree that most of the time text rendering isn’t consuming a ton of resources, until it does.
My point is that the "until it does" is largely irrelevant, as even the most pathological case for a terminal is easily accommodated and in practice almost never occurs. And as you pointed out a lot of OS text rendering support is accelerated anyway (or will get there), which makes this largely moot anyway by making it a system concern rather than something an application ought to put effort into dealing with.
> I don’t disagree that many editors aren’t exactly optimized. None of that changes the benefits of offloading framebuffer work to the GPU.
The point is that the failure of prominent "accelerated" terminals to even match rxvt is a demonstration that spending the effort to write code that is far less portable is a poor tradeoff and tend to show that people writing these terminals rarely understand where the actual bottlenecks are.
> but by and large everyone already moved to the GPU
This is only true to the extent you're calling running on top of OS API's that have been accelerated as "moving to the GPU". The vast majority of terminal emulators and editors are still relying on generic APIs that may or may not be accelerated depending on where they're running. Only a small minority are targeting GPU specific APIs. In this context when people are talking about GPU accelerated editors or terminals, they're talking about the latter.
renderType: Text.NativeRendering
QQuickWindow::setTextRenderTypeThough, the biggest limitation is that the framerate is only 30Hz since the HDMI version is older on that laptop, so maybe that has something to do with it as well.
Try again, with --disable-gpu.
Anyway, just curious: if I use vim in a GPU-rendered shell like kitty, is it pretty much the same as this in terms of GPU-related benefits? Or is this supposed to be better?