I want to believe that this will actually happen, but my experience with people blindly trusting what a tool (not even an AI-based one) says, including but not limited to students, suggests that's not going to happen.
https://en.wikipedia.org/wiki/Alcohol_advertising#Drink_resp...
They also generally won’t help you evaluate scientific claims unless you also have enough scientific background. If someone says “humans will set foot on Mars within 30 years”, you’re going to have a hell of a time determining whether that’s reasonable unless you know a lot about: rocketry, physics, space travel, radiation, astronomy, human anatomy, nutrition, engineering, economics, and many other subjects!
If you’re just a math/logic wiz and not an expert in those other areas, you may be inclined to say “they’ve already put rovers on Mars, should be easy to get humans there!” because there’s nothing impossible or illogical about the question.
How is that not an application of domain knowledge (know your neighbours) with a bit of intelligence? How do you teach people to do that in general? How do you teach people to deal with questions when they lack domain knowledge?
You can even get it to break its reasoning down into little convincing but, none the less, incorrect steps.
If I was running this course I would make that "artefact" part of the process.
i.e. I would tell students that the chatbot will occasionally very convincing lead them in the wrong direction but that they can fact check its answers in the course material.
I would set traps to ensure that they are doing that. ;)
Use LLMs but never trust them, they're the Cliff Clavin of information tools.
You won‘t have to. Students are extraordinarily skilled at fining endless pitfalls on their own.
Setting the stage where they either land in the pit, or start from within, ensures they're interacting with the material as intended.
This is true of humans just as well
LLMs as of now simply appear not reliable enough for an institution like this one to attach their name (which brings credibility in the eyes of the public), especially as their comments on the need for users to be vigilant ("they should always think critically when taking in information as input") make it sound like they did not in fact find some new way of reliably grounding output in accurate information.
Mind you, this is coming from someone who has great hopes for the future of LLMs in education as they can more effectively deal with students on an individual level. I just feel that currently LLMs are still too error-prone for an institution such as Harvard to incorporate them into an introductory course to assist. Until these insitution stop putting the majority of responsibility on students finding erroneous output, I feel this is not ready for wider educational use.
What's a reputable institution? These days most "experts from institutions" are proven to be wrong. The COVID-19 pandemic was a clear eye-opener on how little you could just trust anyone.
Even this, I wouldn't bet money on what I just said.
_"Point out incorrect assumptions or statements in the above answer."_
I got the idea from Khan Academy's [implementation](https://youtu.be/A7REVn9gzgs?t=1208) of GPT-4, where they allow it to 'think' by generating an internal response first, before generating a final response. This apparently improved the veracity of its output by a significant margin.
If ChatGPT doesn't double down on its original answer, then I know definitely not to trust it.
This doesn't solve the underlying problem, because it doesn't ultimately correct the error in a reliable way, but it does help me avoid some of the more obviously garbage output.
"Simply read the textbook" would be a more practical instruction, and also what I did in university anyway.
Critical thinking is a useful description but not a good prescription. It's like telling someone to always be orange. If you want that to happen you have to eat carrots. It won't just happen by fiat or willpower.
I think AI trained on text books should be able to achieve the same thing relatively easily.
It’s just an interactive textbook, seems like a pretty good idea to me.
If they think ChatGPT is ready to teach, they should offer an electrical course using it and force the administration to take it.
Y'know, the thing we /human/ were supposed to be teaching you.
Being efficient and being lazy are rubbing up against each other with LLMs. A lot of blurred cognitive lines with this budding technology.
Maybe I'm imputing too much intelligence to Harvard's administration.
What they really mean is "think critically when others tell you something, but accept as gospel everything I am telling you".
They most certainly don't welcome challenges to the basic assumptions on which their teachings rest.
Not that professors are in any way unique in that regard; it is a trait shared by all people with power and authority.
Because this doesn’t match my experience, at all.
If you had good professors in college you should consider yourself lucky.
There is a lot of absolutely terrible professors.
A professor teaching about evolution will answer students' questions within reason - and students with a religious background might have heard some anti-evolution gotchas the professor will be happy to explain - like how something as complex as the eye could evolve.
But that doesn't extend to debating bible verses, allowing so many challenges that it disrupts the class, or changing the exam so you can pass it while denying evolution exists.
The idea that academics are on the right path has turned from something that must be continuously demonstrated to something that is assumed by default, as part of a new orthodoxy that ironically is almost indistinguishable from the religious and ideological orthodoxies that science once sought to replace.
You're talking about a Ph.D. level investigation, not something that is usually considered at the level of an undergraduate in the US.
Most beginners lack the contextual knowledge to recognize even glaring errors. For example, I once saw a distributed application that occasionally updated a central database without any locking. Occasionally the data would get corrupted by simultaneous writes, and the original dev had no idea why.
Consider adding more context if you want to continue to write about dogma in academia. Share your own experence. At the level of generality you are writing it's impossible to engage more because we don't share the same context.
I chose evolution merely because a substantial and politically important group of Americans earnestly disagree with the academic consensus; it's an area where there has been genuine public debate; it's foundational to some academic fields; and it would believably come up in first year compulsory classes.
I might disagree with academics about whether mailing a survey to mentally competent adults counts as human experimentation that needs ethics board approval, as outside of academia people use surveys all the time. But that's hardly something an academic would refuse to discuss.
I might disagree with academics about feminism or marxism or underwater basket weaving - but that stuff's all elective, why would I have taken an elective module from a teacher I thought was full of shit?
I might disagree with the high tuition costs of universities, and the money wasted on sports, overpaid administrators, and overpriced journal subscriptions. But most academics would actively agree with me, they just can't change it.
I might disagree with details of how a course is taught, like whether Java is a good language for an introductory CS class. Or whether we really need so much math in the CS curriculum. But that's not really a fundamental belief.
I might disagree with academics because I think the moon is made of cheese, but that would be a straw man argument.
Professors are trying to teach a class, not engage in a debate with a student. Challenging basic principles that everyone in the class are already assumed to have accepted is usually seen as an attempt to derail the lecture.
Derailing the lecture and drawing the professor into a debate with you effectively denies the other students access to education. It’s roughly equivalent to heckling a comedian.
If you want to debate your professor, do it on your own time, in office hours. If the professor is still offended and unwilling to debate then you have reasonable grounds to complain. Most professors I’ve met absolutely love to debate outside of class.