Some of this is insanely impressive. I wonder how big the OS ROM (or whatever) is with all these models. For context, even if the entire OS is about 15GB, in order to get some of these features locally just for an LLM on its own, its about 60GB or more, for something ChatGPT esque. Which requires me to spend thousands on a GPU.
Apologies for the many thoughts, I'm quite excited by all these advancements. I always say I want AI to work offline and people tell me I'm moving the goalpost, but it is truly the only way it will become mainstream.