How do you go about comparing models? Personally, I've found that the quality of output is overwhelmingly dictated by the quality of the input I give it - and that the quality of input differs massively depending on the task at hand, how much documentation exists, what kind of mood I'm in, etc.

Hence, I'm beginning to believe that unless you're doing very specialized work - the model itself has become a commodity, incremental quality is irrelevant and that economics and privacy are what actually matter now.