Hacker News Reader

Comment by juliusbk | Hacker News Reader

This looks super cool! Looking forward to trying it.

I think a killer feature of these gpu-plotting libraries would be if they could take torch/jax cuda arrays directly and not require a (slow) transfer over cpu.

View on HN

Thanks! That is a great question and one that I've we've been battling with as well. As far as we know, this is not possible due to the way different contexts are set up on the GPU https://github.com/pygfx/pygfx/issues/510

tinygrad which I haven't used seems torch-like and has a WGPU backend: https://github.com/tinygrad/tinygrad

Yeah, I remember looking into it myself as well, and not finding any easy path. A shame.... Maybe there's a hard way to do it though :)

I've been looking into this issue with Datoviz [1] following a user request. It turns out there may be a way to achieve it using Vulkan [2] (which Datoviz is based on) and CuPy's UnownedMemory [3]. I wrote a simple proof of concept using only Vulkan and CuPy.

I'm now working on a way for users to wrap a Datoviz GPU buffer as a CuPy array that directly references the Datoviz-managed GPU memory. This should, in principle, enable efficient GPU-based array operations on GPU data without any transfers.

[1] https://datoviz.org/

[2] https://registry.khronos.org/vulkan/specs/latest/man/html/VK...

[3] https://docs.cupy.dev/en/latest/reference/generated/cupy.cud...

This looks cools thanks! Makes me wonder if there's any way to do that with WGPU if WGPU is interfacing with Vulkan, probably not easy if possible I"m guessing.

WGPU has security protections since it's designed for the browser so I'm guessing it's impossible.

Indeed, it doesn't seem to be possible at the moment, see e.g. https://github.com/gfx-rs/wgpu/issues/4067

Wow. So are you saying that you can have some array on the GPU that you setup with python via CuPy, then you call to the webbrowser and give it the pointer address for that GPU array, and the browser through WASM/WebGPU can access that same array? That sounds like a huge browser security hole.

Yea the security issue is why I'm pretty sure you can't do it on WGPU, but Vulkan and cupy can fully run locally so it doesn't have the same security concern.

Exactly, this is the sort of thing you can more easily do on desktop than in a web browser.

I published my proof-of-concept here: https://gist.github.com/rossant/517806ea551f4038fd412c23c3d6...

Would it be possible to leverage the python array api standard? Or is that more suited for just computations?

Not sure about the memory transfer bottleneck and potential mitigations. But out of interest, how insurmountable would it be to 'retool' fastplotlib to use JAX acceleration instead of wgpu?

Reply on news.ycombinator.com