Comment by rektide | Hacker News Reader

Comment by rektide | Hacker News Reader

I'd really really like to see 3d content start making it's way into the DOM. (Ideally in manners far far far far beyond showing one lone hovering item, as per Apple's <model> proposal.)

Seeing Steven use his Live library to provide such an implementation should be utterly unsurprising. His past works have been so exemplary & leading-edge: mathbox, TermKit, Live itself, & others. Live was an interesting exploration of what grounds really lay underfoot the latest emerged React patterns (hooks), with lots of new capabilities, and seeing it applied to such a holy grail of an idea (3d via dom) is thrilling to see.

> Live goes far beyond the usual React semantics, introducing continuations, tree reductions, captures, and more

There's a ton of incredible wonkery here. As usual for Steven, he's chased incredibly far reaching frontiers, applying this hyper-advanced computing toolkit to 3d, then hunting down a series of hard problems/demos to make expident work out of. The article talks to a lot of specific frontiers, but the "general" work of the underlying Live library (and it's tour-de-force proof-of-value by creating Use.GPU out of it) is quite the distillation of computing into a consolidated, clear form:

> The result is a tree of functions which is simultaneously: 1. an execution trace 2. the application state 3. a dependency graph of that state

Hardly the point, but one thing I'd like a little more affirmative on, to see more clearly- where my desires & Steven's intents might be mis-aligned is- what would it look like rendering environments & spaces here? Can this "dsl for dsls" help us craft landscapes, houses, & actors inside? What do situated spaces look like in this environment, if we can pull that off? Can we represent space in the DOM?

Have you seen what react-three-fiber is doing?

https://docs.pmnd.rs/react-three-fiber/getting-started/examp...

https://twitter.com/0xca0a (the creator, who has some examples as well)

Good example!

I was trying to use A-frame a bit back, another contender. https://aframe.io/ Somewhat ashamed to admit, I fell on my face a couple times trying to get the legacy cjs modules building with a couple toolchains & put my 3d web projects back in deep-freeze. https://github.com/aframevr/aframe/issues/3732

Briefly, it does make me a bit sad that React isn't the real dom. I doubt very much any of these solutions make the content actually available in view-source. Which, like, 10 years ago would get you 20 lashings for bad behavior. Now-a-days? Normal, react & elsewhere. Un-page content is anti-user.

The problem imho is bandwidth. Unless you are iq or similar, creating geometry and textures from thin-air with pure math is just not possible, so you need to load 3d models, which are much heavier than their html/png counterparts.

For individual models, like if you browse sketchfab, it's not such a big deal if the artist optimized things to be "game-ready", but for full environments, it's a problem, especially as audience expectations are influenced by what they see with multi-gigabyte assets on standalone games and simulations.

I don't think this is a complete showstopper, like 10-20 years ago we had the same problem with 2d games and Flash, but as long as the experience was compelling enough and the preloader wasn't completely boring, people waited, but there has been a shift towards wanting things to be instantaneous on the web (e.g. video with adaptive bitrate switching, blurry but instant > sharp but buffered)

Anyway, I guess my point is I don't think it's exactly a technical limitation of programming, as much as a conflict between audience expectations and wait time

Signed distance fields _are not that bad_ (thank you Inigo Quilez)

Once you have your primitives, it becomes basic math and union, subtraction, intersection for the most part.

I made this shader a few days after learning the basics -> https://twitter.com/LeapJosh/status/1494417750395179012

If glsl ever got function pointers or similar, you'd see simple SDF DSLs and frameworks bloom overnight and many of them would be easy and awesome.

Raymarching ever-more-complex mathematical functions is neat if you're creating a demo out of curiosity or for art's sake.

But it's a very expensive and not-very-scalable method of rendering things, certainly not a replacement for meshes of triangles.

Shameless plug - I wrote sdf-csg[1] as an attempt to have that cake and eat it too. Basically, build things out of SDFs, then create meshes out of them for efficient rendering.

[1] https://github.com/wwwtyro/sdf-csg

This is very cool. I wish I could use it in unity!

Going to take a few minutes to scope out what a port would take...

Check out Fuse[1][2], it also has a SDF to mesh feature and since it is based on Stride[3] (C#) it might be easier to port.

[1] https://github.com/TheFuseLab/VL.Fuse

[2] https://www.thefuselab.io

[3] https://www.stride3d.net

Fair point.. and the response is also right on. I'd edit my original comment to say "...creating full interactive scenes of geometry and textures..." if I could :)

"Unless you are iq or similar" => it's actually quite accessible if you dive into it. This Pouët thread is a nice starting point: https://www.pouet.net/topic.php?which=7920&page=1. Also: https://www.pouet.net/topic.php?which=7931&page=1

Browsers are resource hogs, I'm not sure I'd want any 3D content more than is necessary. CSS alone is capable of hogging my CPU at 100%

Not just that: few people are Steve Wittens or Farbrausch vetted, most are bewilderingly contented with the ugly and deformed. Even among professionals, and with decades long evidence. See for example the adoption of 3d in online maps, that brought from "good as expected" to "utterly unwatchable, unusable" (and it is fortunate that you can still access the past versions). And that is the big 'G', with nobody able to note, "look - and do look -, this is an unacceptable result".

Normally you need good illustrators and artists: with some approaches you need much better illustrators, artists and taste.

Related: a few weeks ago StackOverflow came out with a visual filter that made the page psychedelic, in the wrongest way. ...As a feature.

I didn't notice any psychedelic filter? Where can I see it?

Most consumer machines have a literal supercomputer sitting mostly idle as you browse the web. Css is slow because it was designed for cpus and css doesn't map to gpus very cleanly. But Webgl/webgpu enable some incredible things.

> Can we represent space in the DOM?

Why would you want to represent this in DOM, of all places? It's uniquely unsuited for anything that's not a few paragraphs of text with a few images.

But representing the DOM etc inside a 3D space is useful so you don't loose access to the world of 2D information. However there are no good solutions to this at present the Immersive Web Workgroup in W3C are looking at Layers or DOM Overlay which is for AR modes but might support 2D navigation but it seems low down on the priority list due to CORS and other security challenges.

https://github.com/immersive-web/navigation

hoisting divs as hud works nicely already, both in ios (webxr viewer) and android (chrome + google play services for ar)

you just need a div directly below body, then you can actually enable it as a hud in augmented reality.

at the moment we only use it to show announcements to the visitors, but i also have tested buttons with event listeners attached and you can even console.log from there.

html

  <html>
    <body>
      <canvas id="three-canvas"></canvas>
      <div id="hud">
        <button>Will be visible in ar</button>
      </div>
    </body>
  </html>

js

  const xrSession = await window.navigator.xr.requestSession('immersive-ar', {
    // ...requiredFeatures,
    domOverlay: {
      root: $('#hud'),
    },
  })

css

  #three-canvas {
    z-index: 999;
  }
  #hud {
    z-index: 1000;
  }

edit: use window.navigator instead of W.NAV, my custom "window" global.

> Why would you want to represent this in DOM, of all places? It's uniquely unsuited for anything that's not a few paragraphs of text with a few images.

Your negative bias is showing. That's, just, like, your opinion man. One built around disdain & dislike, and not generally shared. There's no books that publish that opinion. You don't find that limited view abundant in trade publications.

Use.GPU is here to enable 3D via the DOM, and is proof positive that this declarative setup is highly powerful. The heirarchical/acyclic-graph structure more than passingly resembles a scene-graph, which is ultra-common in 3d. Since the hierarchy is apparent, bindings all just work:

> The shader snippets will end up inlined in the right places with all the right bindings, so you can just go nuts.

This is just Use.GPU. It's a huge step up the vast amounts of rote work required to set up bindings in modern vulkan & 3d in general. Barely the tip of the ice-berg. As I said, Use.GPU is a tour-de-force demonstration of the value of Live, which is a much more general & competent way of representing compute on the DOM. Steven calls Live a DSL for DSLs; a small language which can keep representing more, all with common structure & form of the DOM language underneath, a much higher level base for computing that escapes much reinvention along the way. Steven touches numerous times across Live write-ups & this article about how the shape of the DOM is ultra-close to that of Lisp, another blissfully homoiconic computing representation known for it's flexible means of assembling compute.

> Along the way Live has gained actual bona-fide <Quote> and <Unquote> operators, to drive this recursive <Reconcile>. This means Use.GPU now neatly sidesteps Greenspun's law by containing a complete and well-specified version of a Lisp. Score

I think it's under-ambitious to stop at just Space, and that the DOM's S-exp like nature is a powerful & advanced tool for representing all manners of things. Outside of Steven's work, there's plenty of other material. Work like React-Router are extremely clear & direct means of expressing routing/addressability concerns, far cleaner than any other toolkit I've seen. I invite everyone to keep an open mind, and try to use their brain to escape the limited preconceptions they already know, & to explore openly & with active regard, looking for potential. I think the potential for the DOM is high.

> I'd really really like to see 3d content start making it's way into the DOM.

There's a lot of overlap w/ HTML5-native game development. If you're interested in that, pay https://github.com/hmans a visit and take a look at his projects (RenderComposer, ShaderComposer, VFXComposer etc.). Nothing short of amazing!

Reply on news.ycombinator.com