How much cost would it take to run a LLMs locally
If I want to run LLMs model like Qwen or gpt locally how much would it cost me and I want to connect it to my main website creating an API link, also which models would be best
If you want to use the model through an API for your website, a good setup would be a dedicated machine/server to host it, since running LLMs locally can consume a lot of memory.
Depending on your hardware, you can look at smaller variants of Gemma or Qwen Coder.
Another option is to use a hosted inference provider. It’ll cost a little, but you’ll usually get much faster inference compared to running locally especially if your system isn’t high-end.