I answered this last night, though, I wanted to share a lot more with them, but I don't like giving ideas for free. There's so many things these companies can be doing that I keep to myself but I'm mostly convinced a lot of them need to take a step back and rethink LLMs, LLM design, etc because it looks to me like we're all stuck forcing ourselves to use an insane amount of compute, with an insane amount of data, but the small models have shown to be capable of reasonable reasoning on way less resources and data. Heck, the distilled or quantized version of the only open model OpenAI ever released is insanely good.

I don't know all the answers, but I think we're seeing the "ceiling" of where LLMs can go, and we're trying to force them to go further in the hops of achieving AGI somehow, but AGI in my eyes is not going to work with the current design for LLMs unless they're actually building AGI completely differently similar to what I assume it could / should look like, then I'll give them that, but none of them have shown any signs of true AGI. Sure an LLM can be impressive in terms of speed of outright (and correctness now!) but its not AGI. AGI isn't a buzzword you change, it was defined before Sam Altman and Dario tried to redefine and water it down for IPO gout.