Hacker News Reader

Comment by pjc50 | Hacker News Reader

> Instead of sending a frame to the system, send a tree of layers.

Which API does one use to do this?

I'm slightly surprised that hardware overlays aren't already a feature, especially given that they're handled by the graphics card. I know there's a special API for video overlay, especially DRM video (where part of the requirement is that the system doesn't allow it near the compositor where it could be screenshotted). Can you do video "genlock" on Windows? (edit: https://msdn.microsoft.com/en-us/library/windows/desktop/dd7... )

I'm also wondering how VR/AR stuff handles this on Windows.

View on HN

DirectComposition. It's been there since Windows 8, and is used by Chrome among other apps (see https://bugs.chromium.org/p/chromium/issues/detail?id=524838).

It's possible some hardware already does overlays, given that the talk I linked above was 2 years ago. I haven't researched this carefully.

There's a bunch of stuff in the interface to support video and also integrated 3D content ("create swapchain for composition"), but I don't know how well it works. In my experiments and reading through the Chromium bug tracker, Microsoft's implementation of all this stuff is far from perfect, and it's hard, for example, to completely avoid artifacts on resizing.

> It's possible some hardware already does overlays, given that the talk I linked above was 2 years ago. I haven't researched this carefully.

All hardware does some overlays, though not a large number. (You can see Intel GPU overlay setup at [1].) One scanout overlay is already in use on all major OS's, to draw the mouse cursor.

[1]: https://github.com/torvalds/linux/blob/e60e1ee60630cafef5e43...

Typically hardware will have the base layer, a cursor layer, and possibly a video overlay. But in many cases, the video "overlay" is actually emulated by the driver using a shader to do the colorspace conversion and the texture hardware to do scaling.

People generally aren't composing other effects on top of a video overlay. Making it work in the general case requires hardware that can do all the things the compositing engine is doing in software.

The usecase mentioned in Jesse's talk (linked in my root comment) is displaying a notification from some other app while playing a game. They added the "flip_discard" swapchain effect so that the OS can paint the notification on top of the game content before flipping it in hardware. This is something of a hack; I think you're right that the endgame for this is that the hardware can indeed implement the full compositing stack. I'm not sure how far away we are from this.

Reply on news.ycombinator.com