> I have been testing
With local models which are often benchmaxxed, testing unfortunately isn’t as predictive as you’d like.
With local models which are often benchmaxxed, testing unfortunately isn’t as predictive as you’d like.
That is precisely why my testing has been daily driving the model for everything + 8 tasks in a domain I care about. Could there be something very similar in their datasets? Of course, at least for most of the tasks, but if that lead to the good performance experience and results I'm getting, I am personally ok with that. I don't care how high the numbers are on the common benchmarks, only if it works well enough for me.
And if this model doesn't work for you, that's perfectly ok. Everyone has different needs from models. I was just impressed that it did for me, as it was a first from a local model.