This is such a trope at this point.
We asked ChatGPT a question about a system the other day. Its response was,
> That number is equivalent to [another number]. That number controls $thing, and that number is the recommended value for thing according to $standard. It relates to $unrelated_thing.
It did the math wrong when converting the number from one unit to another. (From days to seconds.) While it got the gist of the number right in the first half of its response, and it even cites essentially the right source, the claim it says the citation backs … is not backed by the citation. It then goes on to equate it to something that it isn't related to.
It's words, strung together: i.e., bullshit. The higher-level reasoning (like that the math needs to work out, or the citation should actually have the cited claim) are utterly absent, and will never be present in these models no matter how much data you train them on.
The response is at best wrong, and at worst, fools someone into believing that it knows what it is talking about.
But yeah, this is going to replace me.
The actual article is … pretty long-winded, and really lacks specific, concrete examples as to how AI will actually move a needle somewhere. Take this chunk,
> Computers haven’t had the effect on education that many of us in the industry have hoped. There have been some good developments, including educational games and online sources of information like Wikipedia, but they haven’t had a meaningful effect on any of the measures of students’ achievement.
(Wikipedia hasn't had a meaningful effect?! [citation needed] there. Wikipedia affected my own achievements, and it has been a while since I was in school. I can only think that such a high quality encyclopedia is of immense value to students, particularly if you're not lucky enough … old enough? … to have an actual encyclopedia — even the use of the word actual here exposes how much effect Wikipedia has had!)
> But I think in the next five to 10 years, AI-driven software will finally deliver on the promise of revolutionizing the way people teach and learn. It will know your interests and your learning style so it can tailor content that will keep you engaged. It will measure your understanding, notice when you’re losing interest, and understand what kind of motivation you respond to. It will give immediate feedback.
We've been saying this about ads for decades. The last ad I remember seeing was "The Mandela Effect: In Queen's 'We Are The Champtions (sic)' He Doesn't Say '…Of The World'" ((sic: the ad title is factually inaccurate) ; … this is no more related to my interests than any of the many Taboola and Taboola-like junk ads at the bottom of half the Internet now. I have no idea where that ad ponzi scheme even ends.)
> There are many ways that AIs can assist teachers and administrators, including assessing a student’s understanding of a subject and giving advice on career planning. Teachers are already using tools like ChatGPT to provide comments on their students’ writing assignments.
And we're 3 paras in with very little meat here: the only example is "comments on student papers". Given the inability to reason … no, I don't think AI will be evaluating papers anytime soon. (The whole passage … if not the article … feels like a human vaguely touched up the output from GPT.)
And the entire idea of trying to help poor unfortunate souls in desperate places with AI feels very much like God vomit from a Cory Doctorow tale.