At first glance i would be surprised if this works at all.
The readme says it loads a significant amount of data the first time, 250mb. None of the LLM weights I know of are less than several gigabytes in size.
It says it only requires 512mb of RAM. None of the interesting LLMs I know of run in less than 6gb of VRAM.
It says it uses no API keys, which is great, but that means inference is local, which I can't imagine works with the above constraints.