> Have you considered a limited LLM that could run locally?

I think there are two main issues here. LLM are large (the name even hints at it ;) ) and the smaller ones (still, multiple GB) are really, really bad.

Edit: and uses a ton of memory, either RAM if CPU or VRAM if GPU.