yu3zhou4

Much thanks for the explanation and for your help Daniel!

It’s not a dupe, it’s a full training and infra report

As far as I know we don't know much about metacognition in LLMs, though? Not sure

Same! I believe that you could actually train a LoRA on top of a model to get results close to that

The strange thing is that the base models (before RLHF) use the "experiential" voice, even though they are not incentivized to do that.

Thanks for pointing out, maybe I should be more explicit in the wording - I mean we don't fully know what drives the voice in LLMs. Models that are post trained as instruct models are expected to have the disclaimers, but what about base models (those that are trained on just a lot of text)? How do they talk about themselves? What happens when you strip off the chat template from instruct model's prompt? I hope the rest of the paper makes the questions clearer, but I will try to do better in the abstract next time, as you point out this sentence is kind ambiguous. Thank you!

I’d love this but for every hardware

Gratulacje Jan! Looks like something critical to gain adoption these days, security-wise. For others who also wonder how it works, I find this docs page a bit more informative than the landing page https://droprun.sh/docs/sandbox-overview/

Michał Zalewski (the author of this book) is goated. His book Silence on the Wire was very inspiring to me and the techniques he described often come to my mind years after reading the book

I study stats and probability

I found that in LLMs the self-reference defined as "referring in generated text to itself" (so a bit different than in Scott's blog) comes largely from the chat template

https://openreview.net/pdf?id=3O2A23MhNi

Also PageRank as a Markov chain

[0] https://math.libretexts.org/Bookshelves/Linear_Algebra/Under...

I’m happy it helps you!

If you prefer C++ and CUDA, then there's also tiny-vllm of mine [0] - recently we broke 1k gh stars

[0] https://github.com/jmaczan/tiny-vllm

My fav zine

It's a very useful guide, I used it when working on torch-webgpu some time ago. Elie also published few packages that help with using some WebGPU related things, I don't remember exactly what was that but I recall it saved me lots of time

https://github.com/jmaczan/torch-webgpu

Print ondemand can be ordered for less than $5 per zine in Lulu, since 2022 or so. I guess Zach doesn’t make a cut from it since this $5 probably barely covers the cost of print and Lulu fee :(

PyTorch?

README is in my opinion (author here) the most interesting - I wrote it to help others build useful mental model to be able to recreate the project yourself, without need to even read my code

There was onivim that was a bit hyped a few years ago but unfortunately it died

Just learning math and trying myself in ml research

Laughed hard about Collegium Tumanum

Poland was sort of occupied until 1989

Good idea, too! Why do you see my explanation as cynical, though?

Maybe it’s in order to have an external provider to blame for failures and shift the blame/responsibility?

Adding a support for new hardware to PyTorch is actually quite convenient. I did that with WebGPU using the same PrivateUse1 mechanism TorchTPU used. Every hardware has its own slot and identifier, and when you want to add a support for a new one without merging it into PyTorch, PrivateUse1 works essentially like plug-in slot

https://github.com/jmaczan/torch-webgpu

They write that they use PrivateUse1, so it’s a custom out-of-tree backend

The system like iOS is still closed. Once Apple ends support for a device, being able to swap a battery won’t help much

Thx, great read about Yemen from Maciej Cegłowski https://idlewords.com/2014/07/sana_a.htm

An open course on building high performance LLM inference engine! Hope to finish by the end of April

https://github.com/jmaczan/tiny-vllm