If current LLMs hit a scaling wall and the game becomes about efficiency, I wonder if there's going to be space in the market for small models focussed on specific use cases.

I use Gemini to extract structured data from images and the flash model is great at this. I wonder how much effort it would be to create a smaller model that would run on something like a NUC with an AMD APU that is good enough for that one use case.

Or perhaps you end up with mini external GPU sticks that run use case specific models on them. Might not be much of a market for that, but could be pretty cool.