Comment by ack_complete | Hacker News Reader

Comment by ack_complete | Hacker News Reader

Parent Full thread

It is a lot more constrained than DirectShow graph building.

DirectShow has many characteristics that make its graph building fragile:

- It allows for fuzzy rules to choose filters, which often overlap and require prioritization.

- It runs code in the filters themselves as part of the decision process. This leads to apps breaking when filters are installed that have broken input logic and inject themselves everywhere or reject connections for obscure reasons (e.g. allocator requirements).

- It often needs to build a telescope of multiple filters to complete a requested connection, such as a decompression codec followed by a Color Space Converter. This makes it vulnerable to filter loops and horribly inefficient conversion paths (like converting color spaces by compression followed by decompression).

- Filters are provided by multiple third parties and installed system-wide.

DirectX work graphs work with a private node/shader collection provided by the application and connect inputs to outputs by matching node IDs, so it's more like serialization/deserialization of a predefined graph than constructing one by depth-first search, and it's not pulling unknown external nodes of ambiguous quality.

Something I want to point out is _why_ you would want a graph for GPU work.

The existing programming model is based on passes. You have one dispatch spin up thousands of workgroups, each of which do some computation. Then you issue a barrier, change your shader, and issue another dispatch doing more work, often reading the previous pass's result.

At first glance this seems fine, but it has some inefficiencies.

1. Often the size of the second pass depends on the output of the first. The first pass might determine that some work is unneeded, and therefore the second pass would have less to do. But from the CPU side, you need still need to allocate memory for the worst-case amount of output from the first pass. 2. Towards the end of the first pass, you'll have only a couple of workgroups left running. But you can't start running the second pass yet - you need to wait for _all_ workgroups in the first pass to finish, and then issue a barrier for memory synchronization.

Workgraphs let you fix these problems. You define two nodes A->B that would correspond to your old passes. Then, you only need to allocate a reasonable amount of scratch memory for the first pass's output - you don't need to allocate for the worst case. When A is running, B can immediately run and pickup the results from A via the scratch buffer as A's workgroups complete.

Workgraphs also give you a lot more flexibility with branching based on work type. A common technique is to split the screen into fixed-size tiles, and classify each tile as "cheap" or "expensive" to shade. Then you can run a cheap shading pass only on the cheap tiles, and an expensive shading pass only on the expensive tiles. The problem here is the same as we saw earlier - you need a full barrier and wait for the previous pass to complete before the next can be issued, even though they're independent. And that's with only 2 possible shaders. With more passes, or different length chains of work (e.g. this tile just needs some quick shadows, this other needs shadows and then 3 passes of denoising), it gets more and more inefficient. Workgraphs let you turn all that into nodes, and schedule them much more efficiently.

To me the downside is that it's basically building an AST with a cumbersome 'AST builder API' instead of designing a new specialized programming language where the dependency tree and 'node payload' is all integrated into a single language (instead of having the traditional split between CPU- and GPU-side languages, and a 3D API as glue sitting inbetween - as more and more work moves to the GPU that traditional split makes less and less sense IMHO).

The idea to build an AST with an API instead of a dedicated language feels a lot like GLTF's new interactivity extension, which describes an AST in JSON (blech): https://github.com/KhronosGroup/glTF/blob/interactivity/exte...

PS: also, it would be nice to get an idea what the debugging situation is for workgraphs.

Reply on news.ycombinator.com