Comment by DonHopkins | Hacker News Reader

Comment by DonHopkins | Hacker News Reader

Parent Full thread

Sounds like a modern more general version of ActiveMovie aka DirectShow Filter Graphs, from 1996.

https://learn.microsoft.com/en-us/windows/win32/directshow/t...

The whole point of filter graphs was that it would cleverly try to choose filter implementations based on their wishy-washily defined "merits", depending on what filters and codecs were currently installed, in a way that minimized copying data between the CPU and the video card (pre-modern-programmable-GPUs, mainly video cards with multimedia stream splitting/combining/compressing/decompressing/filtering/rendering and other forms of acceleration).

It introduced the term "Codec Hell", an even deeper layer of "DLL Hell":

https://en.wikipedia.org/wiki/DirectShow#Codec_hell

>Codec hell (a term derived from DLL hell) is when multiple DirectShow filters conflict for performing the same task. A large number of companies now develop codecs in the form of DirectShow filters, resulting in the presence of several filters that can decode the same media type. This issue is further exacerbated by DirectShow's merit system, where filter implementations end up competing with one another by registering themselves with increasingly elevated priority.

>Microsoft's Ted Youmans explained that "DirectShow was based on the merit system, with the idea being that, using a combination of the filter’s merit and how specific the media type/sub type is, one could reasonably pick the right codec every time. It wasn't really designed for a competing merit nuclear arms race."

>A tool to help in the troubleshooting of "codec hell" issues usually referenced is the GSpot Codec Information Appliance, which can be useful in determining what codec is used to render video files in AVI and other containers. GraphEdit can also help understanding the sequence of filters that DirectShow is using to render the media file. Codec hell can be resolved by manually building filter graphs, using a media player that supports ignoring or overriding filter merits, or by using a filter manager that changes filter merits in the Windows Registry.

It is a lot more constrained than DirectShow graph building.

DirectShow has many characteristics that make its graph building fragile:

- It allows for fuzzy rules to choose filters, which often overlap and require prioritization.

- It runs code in the filters themselves as part of the decision process. This leads to apps breaking when filters are installed that have broken input logic and inject themselves everywhere or reject connections for obscure reasons (e.g. allocator requirements).

- It often needs to build a telescope of multiple filters to complete a requested connection, such as a decompression codec followed by a Color Space Converter. This makes it vulnerable to filter loops and horribly inefficient conversion paths (like converting color spaces by compression followed by decompression).

- Filters are provided by multiple third parties and installed system-wide.

DirectX work graphs work with a private node/shader collection provided by the application and connect inputs to outputs by matching node IDs, so it's more like serialization/deserialization of a predefined graph than constructing one by depth-first search, and it's not pulling unknown external nodes of ambiguous quality.

Something I want to point out is _why_ you would want a graph for GPU work.

The existing programming model is based on passes. You have one dispatch spin up thousands of workgroups, each of which do some computation. Then you issue a barrier, change your shader, and issue another dispatch doing more work, often reading the previous pass's result.

At first glance this seems fine, but it has some inefficiencies.

1. Often the size of the second pass depends on the output of the first. The first pass might determine that some work is unneeded, and therefore the second pass would have less to do. But from the CPU side, you need still need to allocate memory for the worst-case amount of output from the first pass. 2. Towards the end of the first pass, you'll have only a couple of workgroups left running. But you can't start running the second pass yet - you need to wait for _all_ workgroups in the first pass to finish, and then issue a barrier for memory synchronization.

Workgraphs let you fix these problems. You define two nodes A->B that would correspond to your old passes. Then, you only need to allocate a reasonable amount of scratch memory for the first pass's output - you don't need to allocate for the worst case. When A is running, B can immediately run and pickup the results from A via the scratch buffer as A's workgroups complete.

Workgraphs also give you a lot more flexibility with branching based on work type. A common technique is to split the screen into fixed-size tiles, and classify each tile as "cheap" or "expensive" to shade. Then you can run a cheap shading pass only on the cheap tiles, and an expensive shading pass only on the expensive tiles. The problem here is the same as we saw earlier - you need a full barrier and wait for the previous pass to complete before the next can be issued, even though they're independent. And that's with only 2 possible shaders. With more passes, or different length chains of work (e.g. this tile just needs some quick shadows, this other needs shadows and then 3 passes of denoising), it gets more and more inefficient. Workgraphs let you turn all that into nodes, and schedule them much more efficiently.

To me the downside is that it's basically building an AST with a cumbersome 'AST builder API' instead of designing a new specialized programming language where the dependency tree and 'node payload' is all integrated into a single language (instead of having the traditional split between CPU- and GPU-side languages, and a 3D API as glue sitting inbetween - as more and more work moves to the GPU that traditional split makes less and less sense IMHO).

The idea to build an AST with an API instead of a dedicated language feels a lot like GLTF's new interactivity extension, which describes an AST in JSON (blech): https://github.com/KhronosGroup/glTF/blob/interactivity/exte...

PS: also, it would be nice to get an idea what the debugging situation is for workgraphs.

Reply on news.ycombinator.com