It's kinda funny how Lucidrains came back to make numerous commits to this repo following Q*.
Apparently this guy like a bunch of others like https://github.com/ggerganov/ggml are implementing transformers from papers for people that want them. Pretty cool.
I’m glad to see people in the open source community taking the initiative to implement these tools for the general public. It’s one thing to understand the high-level concepts in a paper, but another entirely to implement them efficiently. Some people just really have a knack for reading academic papers and implementing those ideas in software.
The additional link is indeed very useful, however, with one exception, it is limited to inference, ie code for evaluating a model using somebody else’s trained model weights. The original link has code for training (which can also be used for inference, of course.) Inference is super important, of course, so it makes a lot of sense to put effort to make it work well on all kinds of hardware; the papers typically focus on training and only then evaluating/validating a new model.
Lucidrains also created the much-missed EpicMafia, which still doesn’t have a good replacement after shutting down. Exceptionally skilled person!