Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration

I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.

If you ever feel like improving your matmul kernel you can check this blog https://siboehm.com/articles/22/CUDA-MMM

Thanks! I will probably do that this weekend

Lovely project, what did you find as the beat place to learn about cuda?

Thanks! I highly recommend the book Programming Massively Parallel Processors

This is the second time this week that people have recommended this boot while I was searching for information about CUDA. Thanks!

Awesome, thank you for publishing it. I was planning the same, but didn't find the time yet.

Is your work publicly available anywhere? I'd love to have a look if possible.

Hi :) Yes it's in the post url or here: https://github.com/mlecauchois/micrograd-cuda

Oh sorry, I didn't realise you were OP :) Thanks!

Awesome. Thanks

Very cool!!

Cool