Here is a C implementation of GPT-2. https://bellard.org/nncp/gpt2tc.html
I don't disagree that hardware acceleration is key in enabling these models, but I still find it interesting how simple the core techniques are.
I don't disagree that hardware acceleration is key in enabling these models, but I still find it interesting how simple the core techniques are.