Using the Infinite Bookspace to Reason About Language Models

I think Hacker News might appreciate some of the behind the scenes of this post.

Getting this page to load quickly was not trivial. The initial dataset of books starting sentences was over 20 megabytes. By only sending the unique prefix of each book, I was able to get that to be much smaller. Using a custom format, sorting the prefixes, and gzipping got the size down to 114kb. About 3 bytes per book. The full first sentences are downloaded on demand as the books are filtered down.

Rendering the books requires 5 million triangles. I used WebGL 2's drawArraysInstanced method. This allows me to define the book geometry only once, and each book is just defined by it's rotation/position/color. Then it's just a matter of keeping the fragment shader simple.

Going into this project, I wasn't sure if it was possible. But I have left feeling really impressed with how capable the web is these days if you are willing to push a bit.