I have no experience doing this, and I dont mean this to be snarky: what is the power (and thermal) usage of running your CPU only LLM?
When I do something "heavy", my 9800x3D will kick on the fans and start making lot of heat. This is fine when I'm intentionally doing say a transcode, but if I'm just querying syntax it could get pretty annoying. Do you "feel" it? I know that will be pretty machine dependent.