I’ve personally run the Llama 3 models locally on a Mac Studio with 128GB of ram, and found it (and other open source models) to perform highly erratically locally.

And while I’ve also encountered quality control issues with the open source models in “professionally hosted” settings as well (e.g., Groq, OpenRouter), it was much worse locally.

Sentences ending abruptly and non-sensical/unrelated words were a common problem.