Hacker News Reader

Comment by remexre | Hacker News Reader

I bet it'd be more work on the side of the game developer and streaming platform, but IMO Zoom, Google Meet, Jitsi, etc. aren't a ton better at streaming text than it looks like Twitch is from TFA. Even in the absence of effects (just small text on a monitor of differing resolution), things can be plenty hard to read.

I suspect that it should be possible for at least the streaming platforms that have native applications to tie into the OS's screen-reading APIs to grab text and provide it out-of-band, e.g. by streaming rectangles of the background color and render text over it on the viewer's client. Then it'd be copy-pastable by the viewer too, and their own assistive technologies could work on it.

View on HN

> but IMO Zoom, Google Meet, Jitsi, etc. aren't a ton better at streaming text than it looks like Twitch is from TFA

Come to think of it, it's a little bonkers that sharing a desktop over a meeting doesn't leverage any of the stuff that is in remote desktop and similar protocols - why send screenshots of text when viewer's PCs are capable of rendering the same text? I suppose the status quo is easier to implement but there'd be a measurable improvement in text quality and bandwidth usage if a different approach were taken.

Video codecs can be extremely good at encoding static content, with the right tuning, and trying to capture the text of any arbitrary app on the screen would be quite a task.

One of the bigger issues for text is 4:2:0 subsampling... not really sure how to work around that, as higher levels of subsampling are extremely expensive.

Is it really chroma? I always thought poor text rendering was an artifact of dct+quantization

Depends. The chroma subsampling still hurts text even at really high bitrates or in static scenes, and you can see a big difference in 4:4:4 or RGB videos.

That's actually something I never really thought about but makes total sense.

I think it's because applications are rendering their own text directly to the GPU these days. Back in the day, you used to use standard OS controls for everything which allow you to do exactly this without any direct attachment to the underlying rendering layers, but these days most applications are either Chromium with an icon or something rendered to a GPU canvas (Flutter style) to get easy animations for cheap, each with their own GPU acceleration system.

RDP has several modes (block updates, video streams, you name it) and switches between them on the fly. Windows especially uses the win32 render hooks to significantly reduce bandwidth necessary to run native applications, integrating in the window event loop mechanism.

I don't think browsers or streaming tools can leverage that tech as easily and even if they could I expect the benefit to be diminishing over the coming years. This understated use case for natice applications is often forgotten and at this point I don't expect the industry to move back to the old way of writing applications.

  Because in general, how do you verify that the two PCs or Mac or Linux or iPhone or Android have the same fonts installed?

Reply on news.ycombinator.com