The GPU version of libNC is available as a free binary, and you can find MIT-licensed source code implementing (training and inference of) transformers using libNC at https://bellard.org/nncp. (nncp was meant to be a submission to the Hutter Prize, which would have required open-sourcing libNC too, but it didn't qualify due to using AVX2 and too much RAM. At least the CPU version binary is MIT licensed since ts_server is.) I think it wouldn't be that big a project to support LLaMa starting from that code, although it is dense code.

Edit: licensing

I think it's fine if a software legend (or anyone) wants to make some income.

"Shame. I cannot take the code / binary and run it as a SaaS efficiently." - HNers

Good for him to commercialize it, and at least he is not pretending to be a non-profit accepting VC money.

Shame as in "it's a shame" or as in "shame on them"?

I'm pretty sure a dev skilled in ML and GPUs has no trouble modifying the MIT code (which runs on the CPU) to something that runs on GPUs...