Not for RAG and Automation applications
But for Fixing Hallucinated code that written by Non Software Engineers.
AI a threat to humanity, my ass.
Perhaps that is what he is trying to get at? I am sure my communication skills are poor, but I found utility in using these tools a year or so ago. Whatever gibberish I was able to give it often produced a good result. These days I can't seem to get anything usable out of them. It does seem, like the parent suggests, that they have gotten worse.
It may very well be that said tools are no worse, if even better, where communication ability is stronger. However, if we have chosen to optimize these systems for those who are great at communicating, at the cost of those who are not, that doesn't help with the topic at hand. The non-software engineers – implying Average Joe – are almost certainly not going to be great technical writers.
All the solutions I've heard up until now is "we just need more data" which isn't particularly sustainable or something a SWE would be in charge of doing
AI is exponentially improving and there is no reason to believe that this won’t continue.
Not for the last two years at least. Over the last year it's been obvious that there are diminishing returns already...
We increase the computational power by a factor of ten, and see a factor of 1.5 improvement.
The theoretical research may have improved exponentially. Implementations available for use seem to be on the decline – at best, stagnant. I find less and less utility as time goes on. Things LLMs shined at a couple of years ago now produce garbage. And for creative work, the output is much too formulaic, which was all well and good initially while still novel, but one has to keep pushing new boundaries and the current crop of tools really seems to struggle with that.
I would also disagree with the idea that LLM tech is "exponentially improving" now. There was the initial release of ChatGPT which was an enormous step forward, but since then its been small iterations on the same fundamental technology. We are already seeing significantly diminishing returns in our ability to improve these models. Most available training data has already has already been fed in, 10xing the compute results in only marginal improvements in performance, etc.
And with that experience I am advising my grandkids to look at trades for their future careers. Because there is zero doubt in my mind that nearly every job that requires a college education today is going to be gone in 20 years.
I am not one to talk in absolutes but the basic circumstances of current LLMs have hallucination at their core and can likely not be guaranteed fact based. It's just an expression of likelihood where you rely on your training data to contain truths.
This is why OpenAI gives more weight to Wikipedia than Reddit comments - the model contains no reasoning mechanism.
It ought to be treated more as a fallible human rather than a superintelligent lookup resource based on fact checked data.
And for testing to find those inconsistencies.