There is a complete trifercation of ML: 1. ML Engineers: the high priests, with access to 10K GPU hours, designing novel Transformer architectures using Tensorflow / PyTorch / JAX. 2. Data Scientists: conducting SFT on pre-trained models via the HuggingFace APIs + MLOps & model optimization (eg via TensorRT). 3. GenAI devs: building LangChain orchestrations and RAG prompt flows using off the shelf LLMs commoditized behind APIs - no stats or linear algebra required.
Too many are jumping on this GenAI bandwagon, which will result in a massive hype-cycle trough of dissalusionment and potential VC AI winter.
Furthermore, GenAI is a local maxima on the path to true AGI. COT / REACT heuristics lack the integrated differential aproach of Hybrid AI, ignoring everything previous generations of researchers focused on: problem solving, planning, probabilistic logic, reasoning etc. For true AGI, we need some focus on: 1. concept representation. 2. goal formation. 3. code introspection and self-modification. GenAI is a big distraction from that kind of R&D.