nathan-barry

LLMs are very good at lossless compression via arithmetic coding. But I didn't know that it was possible to go the reverse direction (do language modeling via a compressor). It's not super great quality, but I'm surprised it worked! Other compression algorithms (like PPMd) use variable n-grams under the hood, and should be much better (although less interesting due to already containing basic language models internally).

Actually NVIDIA made one earlier this year, check out their Fast-dLLM paper

Hey, I’ll add the MIT licenses later today!

Yes, this is the case. During training, the model will get a sequence of text (ex, 512 tokens long) with a percentage of them masked out (with a special <MASK> token). It learns how to unmask those tokens to construct the original text.

In the case that you mentioned, if we had 4 <MASK> tokens in a row, all we are doing for decoding is predicting what those 4 tokens should be.

Generally, this does not seem to be a significant problem, as there are usually multiple ways to express an idea in varying lengths. Also, with confidence-aware parallel decoding, it can usually avoid the scenario you mentioned, as focusing on decoding the highest confident tokens will generally avoid such scenarios with a well trained model.

HTML version: https://www.cs.utexas.edu/~EWD/transcriptions/EWD13xx/EWD130...

Thanks for your input. Lots of good points on the technical side. Will go through and make some edits later tonight or tomorrow.

> You're not hijacking anything, you are using the old APIs as intended and using the right terms in the description that follows.

When it comes to the use of the word "hijacking", I use it in to refer to the fact that using graphic shaders for general computation wasn't initially intended. When NVIDIA allowed programmable vertex and pixel shaders, they had no idea that it would be used for anything else other than graphics rendering. So when I say I "hijack" a fragment shader to compute layers of a neural network instead of as a part of a rendering pipeline, this is what I mean. I don't see a problem with this use of language.

Dr. Vouga's Graphics: Honors class at UT Austin, Undergrad CS

That's a great insight, you're right.

If you couldn't tell, the post was about using shader programming for general-purpose computation, specifically. Yes, WebGL adds compute shaders, but the point of the article was to use the graphics pipeline specifically. If you say there are statements that are incorrect or inaccurate, pointing them out would be very much appreciated :)

Here's a link to the github repo. At the top of the README it has a demo of GPT-2 running and the visualizations of the attention matrices and transformer block outputs

Repo: https://github.com/nathan-barry/gpt2-webgl

Yeah, that would have been a good thing to set up. Main thing to add would be loading the weights into the browser

Thanks for the comment! I did this as a final project in a graphics class where we mainly used WebGL for all the assignments. Seeing the improvements a WebGPU port would bring would be cool to see!

A few weeks back, I implemented GPT-2 using WebGL and shaders. Here's a write-up over how I made it, covering how I used textures and frame buffer objects to store and move around weights and outputs from calculations while using WebGL.

Not my repo, but I think there are others that allow this functionality via the same method the “iMessage Wrapped” project accessed your message history. Since they’re also MCP servers, they should work seamlessly with Claude https://github.com/willccbb/imessage-mcp/tree/main

This is a post covering the basics of machine learning, along with a Go implementation of Karpathy's micrograd

Yeah, right now the weights aren't even pushed to the repo so there's no way for the github page to load them in. I've looked a tiny bit into it and might end up making a fix for it, having it load the weights by fetching them from somewhere else on load, but probably not today.

There is a link to the zipped up folder with the weights here: https://github.com/nathan-barry/gpt2-webgl/releases/download...

If anyone wants to make a PR which fetches the file from there, unzips it, then loads in the weights from there that would be greatly appreciated.

This was a final project for a graphics class where we used WebGL a lot. Also I was just more familiar with OpenGL and haven't looked that much into webGPU

Working on getting it up on Github pages, will let you know. Not sure if there will be an issue with hosting the weights, I'll look into it

Yeah it'd be great if you could file an issue and show what you see! I was using Safari but I'm gonna check other browsers to test if it works on them