This is pretty cool. I had the same idea but in zig: https://github.com/EugenHotaj/zig_gpt2

Not fully finished yet, haven't gotten around to implementing bpe encoding/decoding and only some ops use BLAS.