What kind of hardware are you using and what is your token generation speed (tok/s)?
Every time I've tried to run local models, even on my beefy mac book (128Gb Ram), I've been very disappointed at tok/s speed.
But I should mention: I am trying to implement workflows and processes akin to CI/CD that run 24/7 in the background. These are not interactive use cases, so I don't care so much about tokens per second.