Hacker News Reader

Comment by nathan-barry | Hacker News Reader

A few weeks back, I implemented GPT-2 using WebGL and shaders. Here's a write-up over how I made it, covering how I used textures and frame buffer objects to store and move around weights and outputs from calculations while using WebGL.

View on HN

Request -- big bold link to a working web page right at the top? I read the page, I read your github, and I saw instructions to clone and run a node site, and was like "..nah" I think github pages will serve this up for free if you like.

p.s. Cool!

Yeah, that would have been a good thing to set up. Main thing to add would be loading the weights into the browser

Here's a link to the github repo. At the top of the README it has a demo of GPT-2 running and the visualizations of the attention matrices and transformer block outputs

Repo: https://github.com/nathan-barry/gpt2-webgl

I also implemented Perlin noise using glsl shaders. It was a bit tedious to output a texture as float, but I did it.

Although obviously it's not deterministic depending on the GPU.

Kudos for going down the WebGL route and not the Chrome only WebGPU approach that most likely some people expect.

It is going to take at least yet another year, for WebGPU 1.0 to be available on stable versions on other browsers, Chrome still hasn't stable WebGPU on GNU/Linux, and it is already much far ahead with extensions that most likely won't be on the 1.0 MVP of the other browsers.

Interesting article.

Member of Firefox's WebGPU team here. I'm curious what extensions you're referring to!

We're hoping to ship on Windows stable in the next couple of Firefox releases. Other platforms should follow in the following year. If you want, I'd encourage you to try out WebGPU on Nightly on Linux and see if you run into any issues!

Reply on news.ycombinator.com