However, in many applications there is a limit on how intelligent you need the LLM to be. I have found I am able to fall back to the cheaper and faster GPT-3.5 to do the grunt work of forming text blobs into structured json within a chain involving GPT-4 for higher-level functions.
* Text summary/compression
* Creative writing (fiction/lyrics/stylization)
* Text comparison
* Question-answering
* Logical reasoning/sequencing ("given these tools and this scenario, how would you perform this task")
IMO, for stuff like text comparison and question-answering, some combo of speed/cost/context-size could make up for a lot, even if they do "worse" versions of stuff just that's too slow or expensive or context-limited in a different model.
Intelligence is the most important dimension by far, perhaps an order of magnitude or more above the second item on the list.