Comment by kvark | Hacker News Reader

Comment by kvark | Hacker News Reader

The article is full of inaccuracies, unfortunately. It mixes up the objective criteria W3C CG agreed upon with subjective claims of Apple, leading to WHLSL as a collaborative product of the group... it is not.

> Last year, Apple established the WebGPU Community Group inside the W3C to standardize a new 3D graphics API which provides the benefits of these native APIs, but is also suitable for the Web environment. <...> All of the major browser vendors are participating and contributing to the standardization effort.

The browser vendors negotiated and agreed to proceed with W3C group (as opposed to Khronos/WebGLNext). It's a collaborative effort from day one (where all of Google, Mozilla, and Apple had prototypes), not exactly Apple-led like the article attempts to present it.

> Forking the SPIR-V language means developers would have to recompile their shaders, possibly being forced to rewrite their source code anyway.

This (and the whole section about SPIR-V) reads like FUD to me.

> Weirdly, this would mean on those two platforms, the starting point and the ending point are both human readable, but the bit in between is obfuscated with no benefit.

Inaccurate. The end point is always GPU machine code, and on the way there all the APIs introduced their own intermediate binary formats (SPIR-V, DXBC, DXIL, AIR). It's just that not all of them are clearly specified. We should be able to convert SPIR-V to DXBC/DXIL without going through HLSL, and the AIR limitations are up to Apple to unblock.

> This (and the whole section about SPIR-V) reads like FUD to me.

It's really not. These are legit arguments:

- SPIR-V does not yet have a documented security story that is suitable for the web. Maybe it could have one eventually, but this does not exist right now.

- There is not yet a story for what web-SPIR-V would do about all those optional capabilities and other issues of SPIR-V that are a bad fit for the web. That includes, for example, SPIR-V's underspecification of most operations to allow for UB-style optimizations.

- It's easier to write a code generator that targets a text-based language than a binary one. If I want to write a JS program, or even an offline program, that emits shader code that a browser can run, then: (A) to emit WHLSL all I would need is strings and appending; (B) to emit SPIR-V, realistically I would need some SPIR-V library. Sure there are many SPIR-V libraries, but string appending is easier every time.

It's not FUD. It's just arguments that you don't agree with.

> Inaccurate. The end point is always GPU machine code, and on the way there all the APIs introduced their own intermediate binary formats (SPIR-V, DXBC, DXIL, AIR). It's just that not all of them are clearly specified. We should be able to convert SPIR-V to DXBC/DXIL without going through HLSL, and the AIR limitations are up to Apple to unblock.

Binary formats that are underspecified. What a shocker. Maybe it's because text-based languages are easier to clearly specify.

Creating a clear specification of a text-based language is easier for lots of reasons. Probably the biggest is that the specification has an easy way of citing code examples. Binary specs always have a hard time with that. No question in my mind that trying to glean the C/JS/Java syntaxes from those specs is easier than trying to do the same for x86/arm/wasm/spir-v "syntax".

> It's really not.

Let's take the biggest offender (that I quoted):

> developers would have to recompile their shaders, possibly being forced to rewrite their source code anyway

Those are likely to hit WHLSL harder than SPIR-V, given that WHLSL is attempting to be a whole new specification of the format and execution model, while SPIR-V is only missing the execution model for the Web. The compatibility story of WHLSL (towards HLSL) is still to be figured out, yet the blog focuses on the issue only in the context of SPIR-V (compatibility of existing SPIR-V versus WebSPIR-V)...

I agree there is work to be done for SPIR-V to be adopted on the Web. From the last call, I thought there is also work to be done on WHLSL. This blog post tries to show it as the latter is ready, and SPIR-V is likely to fail, hence I called it FUD.

> It's easier to write a code generator that targets a text-based language than a binary one.

It's easier to s/one/two/, but if you are doing anything more complex, you'd still build a logical representation in memory that is non-textual (hello, AST). From there, emitting binary output isn't more complex. Needless to say, there is already a large ecosystem around SPIR-V, which can be targeted from other SLs like HLSL and GLSL, and even output directly from Rust.

> Binary formats that are underspecified. What a shocker. Maybe it's because text-based languages are easier to clearly specify.

Is HLSL specified? Is MSL? They are documented, but not strictly specified. I only see an actual specification for SPIR-V (among the current-gen graphics shading languages), which is binary. So I don't buy the argument that text-based languages are easier to specify. Let's take a floating point number for example. In binary, you'd just say here come 32bits denoting IEEE 754 float (basically, piggy back on the existing spec). In text you need to say what symbols are accepted (exponent form, dot-based form, sign, how many digits are allowed, etc), which doesn't appear simpler to me.

> Probably the biggest is that the specification has an easy way of citing code examples.

Examples are not required to specify a format, they are basically documentation. In that sense, I agree with you - text formats are easier to document.

To elaborate on the float example, see JSON, where text-based representations of numbers has led to massive incompatibilities among different parsers and generators. The original "spec" didn't even define if numbers were doubles or infinite precision or what. This is much harder to mess up with a binary format.

http://seriot.ch/parsing_json.php

Programming languages and intermediate representations are about a lot more than representing floats. I think that it’s true that the best specification for how to store floats is a binary one. But programs have a lot more than just float literals and text wins at most of the other ones.

> It's easier to s/one/two/, but if you are doing anything more complex, you'd still build a logical representation in memory that is non-textual (hello, AST). From there, emitting binary output isn't more complex. Needless to say, there is already a large ecosystem around SPIR-V, which can be targeted from other SLs like HLSL and GLSL, and even output directly from Rust.

The in-memory representation in any interesting compiler for some source language that is neither HLSL nor SPIR-V is unlikely to be either HLSL or SPIR-V. It’s going to be something that requires some effort to convert to whatever shader format the browser injests.

I’m saying that it’s easier to write a converter to text because you don’t need any special tooling to get started.

> In binary, you'd just say here come 32bits denoting IEEE 754 float (basically, piggy back on the existing spec). In text you need to say what symbols are accepted (exponent form, dot-based form, sign, how many digits are allowed, etc), which doesn't appear simpler to me.

Programs are about a lot more than floating point literals. The parts that cause friction are control flow structure, variable scoping rules, etc. WHLSL goes with conservative choices that are easy to describe to anyone familiar with C. This makes it easy to understand both for folks who just want to use the thing without other tooling and for folks who want to build tooling.

If it was the case that the dominant part (in terms of execution time, memory, security, or any other metric) of a shader compiler was parsing the text then it would be interesting to look at binary as an alternative. I just don’t think that’s the case:

- most of the time and memory footprint of compilation is the backend so binary input is not better.

- binary parsers have no security advantages. I would guess that they tend to have more bugs because they tend to have to use integers in the input as indices in lookup tables.

- I haven’t seen data to suggest that binary parsers are less complicated, though I’ve heard folks express that as a feeling. For sure, parsing of full C is harder than parsing most other formats regardless of whether they are text or binary. But WHLSL is not full C. I don’t think it’s any harder to parse than SPIR-V.

Why not raise and discuss these points on the mailing list instead of a (much more public) WebKit blog post?

Unfortunately it seems many developers have interpreted the blog post to mean that WHLSL has been selected as the ingestion format, which is incorrect.

Reply on news.ycombinator.com