LLMs are only going to improve by fragmenting them into specialized systems for low parameter high performance results. We’ve reached the point where models will get smaller and more compact
But it's informative for the engineers that need something right now, because it means taking the best general purpose tool and specializing it will outperform the general tool, and you can sustain that if you are willing to always hop tools and respecialize. As we may.
Sure you can throw more compute at it. But it cost a lot of money and you hit resource limits.
We have been doing an end run around the bitter lesson with prompt engineering. Also by using different models for vision vs. text. By getting (human coding) agents to "think" and run code.
The bitter lesson might be that you cant predict what the thing is that will be most optimal tomorrow and any player in the AI game can be innovated out of existence at any time.
Maybe anyone except TMSC.
Like I wanted this from a year or two ago to just lets say have a model which lets say is genuinely really really good at sveltekit as an example instead of a model which is good at a lot of different things of sorts yknow
A model for sveltekit, A model for react and for coding general purpose too and preferably we can have a website which can make it easy to find these models/run them, ollama comes to my mind right now but it has really enshittened a little bit from the time when I was thinking about this but so maybe now a little competition on that side wouldn't hurt I suppose.