Hi internet friends, I recorded a workshop about building your own LLM without any math / ML prerequisites. It covers everything from machine learning fundamentals, deep neural networks, transformer architecture, and pre/post-training.

The only prerequisite is being comfortable with learning through code & excel examples.

1. Sampling Large Language Models https://go.justinangel.ai/video-1

2. Reverse Engineering Large Language Model https://go.justinangel.ai/video-2

3. Perceptrons: wx+b https://go.justinangel.ai/video-3

4. Activation Functions: ReLU, GELU, SwiGLU https://go.justinangel.ai/video-4

5. GPU Coding: PyTorch, torch.compile(), fused kernels, CUDA, Triton https://go.justinangel.ai/video-5

6. MLPs/FFNs: Multi-input, Multi-Layer Perceptrons, Feed-Forward Networks https://go.justinangel.ai/video-6

7. Loss Functions: Residual errors, RMSE, Cross Entropy, Loss Landscapes https://go.justinangel.ai/video-7

8. Backpropagation: Training loops, Optimizers, Learning Rate, Batch Size https://go.justinangel.ai/video-8

9. Saving & Loading Models https://go.justinangel.ai/video-9

10. Initialization: Kaiming, Glorot https://go.justinangel.ai/video-10

11. Residuals: Addition, Scaling, Gated, Concatenation https://go.justinangel.ai/video-11

12. Normalization: Pre-norm vs. Post-norm, RMSNorm, BatchNorm, LayerNorm https://go.justinangel.ai/video-12

13. Regularization: Dropout, Gradient Clipping, Weight Decay https://go.justinangel.ai/video-13

14. SoftMax https://go.justinangel.ai/video-14

15. Tokenizers: By Character, By Word, BPE, SentencePiece https://go.justinangel.ai/video-15

16. Embeddings: Absolute vs. Learned, Sinusoidal vs. RoPE https://go.justinangel.ai/video-16

17. Attention: MHA, GQA, MQA, MLA https://go.justinangel.ai/video-17

18. Transformers https://go.justinangel.ai/video-18

19. Pre-training: Data Sources, Datasets, HTML Cleaning, Quality Filtering, Sharding https://go.justinangel.ai/video-19

20. Evaluation: Leaderboards, Benchmarks, Verifiers vs LLM-as-Judge https://go.justinangel.ai/video-20

21. Instruction Tuning: Alpaca & Other Formats, Self Instruct, Capabilities https://go.justinangel.ai/video-21

22. Reinforcement Learning: Policy Optimization, SimPO https://go.justinangel.ai/video-22

23. What We Didn't Cover: Scaling https://go.justinangel.ai/video-23

Each section has slides teaching the concepts, followed by excel-by-hand developing intuition for the math, and then coding examples. The goal is able to grok all parts of modern LLM development.

We did this workshop in-person in San Francisco last month and hopefully the spaciousness of watching online works for everyone. https://emilyhk.com/llm-workshop/

If don't like watching videos, you can get the slides and exercises and work self-paced. https://go.justinangel.ai/deck