Depends on how many times you need to iterate to get the result you want. If you need to run the fast model 5 times to get the results you need, compared to 1-2 times for a smarter but slower model, you've just eroded any advantage that the speed gave you.
So, as always: it depends on the use case.
fair enough, but in my experience fast models seem to be 5-10X faster than slow models.