Sure LLMs are getting better and better, and at least for me more and more useful, and more and more correct. Arguably better than humans at many tasks yet terribly lacking behind in some others.
Coding wise, one of the things it does “best”, it still has many issues: For me still some of the biggest issues are still lack of initiative and lack of reliable memory. When I do use it to write code the first manifests for me by often sticking to a suboptimal yet overly complex approach quite often. And lack of memory in that I have to keep reminding it of edge cases (else it often breaks functionality), or to stop reinventing the wheel instead of using functions/classes already implemented in the project.
All that can be mitigated by careful prompting, but no matter the claim about information recall accuracy I still find that even with that information in the prompt it is quite unreliable.
And more generally the simple fact that when you talk to one the only way to “store” these memories is externally (ie not by updating the weights), is kinda like dealing with someone that can’t retain memories and has to keep writing things down to even get a small chance to cope. I get that updating the weights is possible in theory but just not practical, still.
What's still missing is the general reasoning ability to plan what to build or how to attack novel problems - how to assess the consequences of deciding to build something a given way, and I doubt that auto-regressively trained LLMs is the way to get there, but there is a huge swathe of apps that are so boilerplate in nature that this isn't the limitation.
I think that LeCun is on the right track to AGI with JEPA - hardly a unique insight, but significant to now have a well funded lab pursuing this approach. Whether they are successful, or timely, will depend if this startup executes as a blue skies research lab, or in more of an urgent engineering mode. I think at this point most of the things needed for AGI are more engineering challenges rather than what I'd consider as research problems.
You see this in construction - the capital is used for certain things and is operated by labour.
Eventually (maybe taking a lot longer than a lot of people expect and/or are hoping for) we'll achieve full human-equivalent AI, at which point you won't NEED a centaur approach - the mechanical horse will be capable of doing ALL non-physical work by itself, but that doesn't mean this is how this will actually play out. If we do end up heading for some dystopian "Soylent Green" type future where most humans are unemployed, surviving poorly on government handouts, then I expect there would eventually be riots and uprising that would push back against it. It also just doesn't work - you can't create profits without customers, and customers need money to buy what you're selling.
Part of why we may (and hopefully will) continue to see humans, from CEO on down, still working when they could be replaced with AI, is that even "AGI", which we've yet to achieve, doesn't mean human-like - it's really just focusing on intelligence. Creating an actual remote-worker replacement requires more than just automating the intelligent decision-making part of a human (the "AGI" part) - it also requires the human/social/emotional part, which will take longer, and there may not even be any desire to push for that. I think people maybe discount how much of being a successful member of a team is based around human soft skills, our ability to understand and interact with each other, not just raw intellectual capacity, and certainly at this point in time corporate success is still very much "who you know, not what you know".