AI tools are here to stay. They will start to creep into everything, everywhere, all the time. Either you recognize the moment at which it becomes a significant disadvantage not to use them (I agree that moment is not now), or get left behind.
Without integrating metaverse and blockchain features into Firefox, Mozilla is at a significant disadvantage compared to other browsers. Don't get left behind!
Edit: so I mean I agree here in case that wasn't clear
> or get left behind.
Last time I heard this phrase it was about VR, and before that it was NFTs. I wished the tech community wasn't so susceptible to FOMO sentiments.
There was a real, genuine product in the Oculus Rift. It did something that was an incremental improvement over the previous state of the art which enabled new consumer experiences for low cost.
The Metaverse was laughable, and VR got glued to a lot of things where it added zero value, or worse negative value, for example my attempts to watch pre-recorded 3D video gave me nausea because the camera can only rotate, not displace, with my head movements.
Compare and contrast with AI:
LLMs and Diffusion models are also real, genuine products, that are incremental improvement over the previous state of the art which enabled new consumer experiences for low cost.
A lot of the attempts to integrate these AI have been laughable, and have added zero-to-negative value.
Another way to see this: Hammers can be useful, the Internet can be useful, but this doesn't mean that as a hammer manufacturer you should make your next hammer an IoT product ASAP or you will be left behind.
Just wanted to note that even after the bad publicity that companies like Meta (ugly avatars, unusable bland virtual spaces) or Apple (overpriced device with no software or content) have given to VR, some people tend to regard it as dead even though there is quite a vibrant user and creator community doing some incredible things (even just what people do with VRChat is amazing!). And there are even companies that seem to get it (Valve).
Does it make the world a better place, and more prosperous? Does it just move economic activity around a bit in regards to who is doing what? We'll find out in ten years when the retrospective economic studies are done.
I can spend hours learning photoshop and then trying out color schemes for my new intricately detailed historic house or removing a car from the driveway or I can use Nano Banana and be done in a prompt. There is dignity in learning all that minutiae but I don’t care, I’m not a Photoshop artist, I just want the result and to just move on with my life and get the house painted.
While I expect the improvements to slow down and stop, due to the money running out, there's definitely evidence that the models can keep improving until that point.
"Sooner or later", given the trend lines, is sill enough to do to SWEng what Wikipedia did to Encyclopædia Britannica: Still exists, but utterly changed.
This will certainly happen with the models that use weird proprietary licenses, which people only contribute to if they're being paid, but open ones can continue beyond that point.
Open models, even if 100% of internet users joined an AI@HOME kind of project, don't have enough resources to do that.
However, this technology is not magic, it is still just statistics, and during inference it looks very much like your run of the mill curve fitting (just in a billion parameter space). An inherent problem in regression analysis is that at some point you have too many parameters, and you are actually fitting the random errors in your sample (called overfitting). I think this puts an upper limit on the capabilities of LLMs, just like it does for the rest of our known tools of statistics.
There is a way to prevent that though, you can reduce the number of parameters and train a specialized model. I would actually argue that this is the future of AI algorithms and LLMs are kind of a dead end with usefulness limited to entertainment (as a very expensive toy). And that current applications of LLMs will be replaced by specialized models with fewer parameters, and hence much much much cheaper to train and run. Specialized models predate LLMs and we have a good idea of how an open source model fares in that market.
And it turns out, open source specialized models have proven them selves quite nicely actually. In go we have KataGo which is one of the best models on the market. Similarly in chess we have Stockfish.