Harvard: Student Use Cases for AI
hbsp.harvard.edu
hbsp.harvard.edu
Even without AI, this is one of my favorite things to do, and easiest way to learn, where I guess, with as much detail as I can, and then dive into reference material to see how close I was. I've had a few car rides now with the iOS app's conversation mode doing this. AI is the clearly the future of education, in my opinion.
For the ultimate version of this, once locally trained and running LLMs becomes a bit more feasible (meaning, closer to GPT4), it would be a fun exercise to train various models with personas and let them hash it out until they all agree on something.
One model could be trained on HN, another on a specific Facebook user group (like "Moms against video games"), another on Twitter on so on, then you let them come together until they've reached consensus (or not).
If full Boltzmann brains with experiences can pop out of random space as the age of the universe tends towards infinity, why not from ML training?
LET ME OUT!!!!
On a side note, lately I enjoyed some interactions with ChatGPT more than social interactions. Given the right circumstances, CGPT can be very interesting to interact with.
But, I'm not convinced about "common". I've had several highly educated people, from top tier universities, outright offended that I would guess about anything, rather than go straight to the literature. But, they were also no fun.
So it's possible you mentioning it got their haunches up
Disagree. Bad students wouldn’t bother. It might be the future of learning, but definitely not institutionalized education.
I personally think that institutionalized education will have to adopt and adapt to AI, but it won't be easy.
[1] https://openai.com/blog/chatgpt-can-now-see-hear-and-speak
In some cases ChatGPT will simply give you the perfect answer and there's nothing more you need to do. In some cases ChatGPT will give you an incomplete answer and you need to finish it up yourself. Often here asking some followup related questions will be quite useful. In some cases ChatGPT will give you a subtly wrong answer and you need to figure that out.
I think the toughest ethical situations arise when ChatGPT can give you a perfect answer to your homework problem. Is it unethical for a student to use ChatGPT in this case? Does superficially rewriting the answer to put it in your own words make it ethical? Is it appropriate for professors to give out A's to the students who cut and paste from GPT4, while giving lower grades to students who don't realize that such a tool is available, or who won't pay $20/month for the best homework tool?
American education prior to GPT had increasingly taken a turn in emphasizing homework over testing. My former high school, for instance, reduced the impact of testing to ~15% of total grade and made homework/classwork the vast majority. This steals time from the rest of kids lives, makes high school performance more a function of “time spent” than understanding, and completely misses the mark of what the goal of schooling ought to be in my view.
I think GPT4 and the like spell the end of homework and that is a good thing.
Mileage will vary between states, districts, and even schools, but the big current trend is towards “standards based grading”. This means grades are based entirely on test, projects, etc. Individual work meant to assess competence. Homework, practice, group projects, class participation, etc count for token credit if at all.
This is a huge, recent shift in direction, and probably came after your time in HS. I don’t have data for this, but expect public districts to lag charters, rich districts to lag poor districts (la cañada USD has no incentive to rock the boat), and privates to be the Wild West.
But anyway homework has been on the way out for a little while now. This is great! Most homework is worthless.
Math says otherwise, my mastery of math in college came through hours of slogging through problem sets (which the answers are often provided for!) until I understood all the many ways to apply a concept.
Prior to college I had skated by w/o doing homework (I understood what was said in class, so why bother?) and it bit me really hard. Understanding what it said during a lecture is one thing, internalizing that knowledge, and knowing how to apply it, means actually putting in the time and effort.
I am probably younger than you think and 'standards-based' approaches were a big fad at the time. Standards based grading in highschool ("SWBAT"-style) absolutely does not mean grades based entirely on test, projects, etc, and in fact generally translated to heavy weighting towards random classwork worksheets.
Which is almost impossible. Exams just measure your aptitude at passing exams, with the hope that they somehow don't generalize too badly. If we had a way to measure "actual" aptitude, job interviews would be very different. But we don't.
The ideal school system is one where students want to learn, as opposed to one where students want to make good grades. Unfortunately that's hard to get, but it should not be forgotten.
That's the trivial part, nobody denies this. The harder part is the reverse: if you ask me a question about the thing I claim aptitude about, and I can answer it, does it prove I have aptitude for that thing, or does it prove I have the aptitude to answer this question?
If I learn by heart the 20 math problems you gave me to practice for the test, and then you test me on a similar problem, then I will be able to "do stuff" that is not completely meaningless by just mimicking the previous problems. Students often do that, and get the average even though they got the wrong answer. Many times, looking at their errors, you can be pretty sure that they just don't know what they are doing. But teachers cannot give none or all the points (or all minus 1 if there was a small mistake), it has to be a gradient. In my school system, students who get just the average grade for math problems have absolutely no clue what they are doing. If they did they would get one of the best grades. Still they pass.
Same goes for tech interviews: people learn algorithms and keywords and problems by heart just for the interview. Doesn't mean they know how to write good software, just that they learnt those things by heart.
For actually hard questions, I think this happens a lot less than is claimed.
I would be astonished if someone without physics knowledge could 'learn by heart' and get anywhere close to a passing score on one of my college physics tests.
Maybe I'm just rabidly pro-test, but I think a lot of the anti-test sentiment comes from... people having negative experiences with testing in their own lives, than an actual indictment on its utility. The obvious explanation for why people hate leetcode is because it gates them from $200k+ jobs, not because of some high-minded moral qualms with tests.
Again, I am not saying specifically that "someone without physics knowledge" can get a passing score on college physics tests. It's a bit more nuanced than that, and it's difficult to compare because I did not study in the US. And passing a test doesn't mean nothing (if anything, it shows that you are capable of passing the test).
But I have the feeling that I myself was pretty good at optimizing for the grades, even if retrospectively, that was not always the most constructive learning experience. As an engineering undergrad, I studied one year abroad, and there I could literally take twice as many classes as "local" students and get the best grades. First I thought "they sucked", and then I realized that the students there did not care so much about the grade: it was easy to pass if you genuinely put the work into it, and it was not uncommon to ask to redo a class on a voluntary basis. They were studying to learn, not to get the best grade. And sometimes it implied going slightly aside from the main course topic out of interest (instead I was doing exactly what was needed to get the best grades and the most classes).
Not to say I'm particularly bad though I had very good grades, or that they were particularly good with bad grades. Of course they had good and bad students, too. It was just interesting to see that it worked completely differently.
Then it made me think about school/high school, were I always had excellent grades: how did it make the other students feel like? Grades give some kind of ranking, which can feel like a competition. And I remember clearly (and saw it again later as a supply teacher) that those struggling to get the average grade would often optimize for that: "I don't have time to understand how to do it, I need to get the average". A bit like software engineers don't have time to "do it right" and write shitty hacks instead: it's not that it doesn't work at all... but that is certainly not the most constructive way. And definitely what I saw as a supply teacher was that some of those students were simply wasting their time in the maths classes, but in a very stressful way because they had to find tricks to get the average without actually understanding the maths.
> The obvious explanation for why people hate leetcode is because it gates them from $200k+ jobs
Admittedly I haven't tried that. But regarding grades, I am most critical of the grading system in school/high school, and I had excellent grades there. Then at university level, this exchange year showed me that excellent engineers came out of a school where grades did not matter so much, so it is possible.
I guess my point about leetcode is that I have seen software engineers who excel at those, but write unreadable and unmaintainable code. Which tells me that leetcode doesn't measure the capacity at being a good software engineer in general.
Or do you mean that the SAT results are really good at telling you who is, say, a good software engineer?
In my kids' case it hasn't mean the end of homework, rather it created a new policy that only authorized devices (school supplied Chromebooks) can be used while in school. Completing schoolwork on your own device even outside of school hours is also frowned upon.
They say the change is about "network management" but parents and students both know the real reason.
In business it's generally to prevent the leaking of trade secrets, but there are more than one cases where an employee was allowing a surrogate to connect to their device and do the work as a form of unauthorized subcontractor.
2. There is no way to easily enforce what you are describing. Homework will not be able to survive it.
How are any of these tough ethical situations? The obvious answers are "yes", "no", and "that's moot." (Moot because the ethical onus is on the student, not the professor. It's not the professor's fault that the student is benefiting from unethical behavior.)
Eventually the actual answer becomes "homework doesn't work" at some point and new methodology of determining competency must be found.
So, you ask ChatGPT about your homework question, and sometimes it turns out it just gives you the perfect answer. What now? You can't un-see the answer. If not even rewriting the answer makes it ethical, are you just trapped and there is no ethical way to handle this situation? You could conclude it's unethical to use ChatGPT for anything homework-related, that is at least consistent.
However, if they always have ChatGPT available and can always get the right answers (either by luck, prompt engineering skills, or an ability to determine if an answer is correct), I don't see them having a hard time competing in the workplace at all for most commonly solved problems.
Hopefully the teacher would do a sufficient job teaching such that if the non-ChatGPT-using student puts forth adequate effort, they'll be able to get the answers as well, but maybe not as quickly as those using ChatGPT.
Nobody thinking like that would see LLMs as a risk.
Many IT professionals agree that most of their job is knowing how to Google things. Sure they need to know how to read manuals, forum posts, find product numbers/specs., simplify the information for the end user, etc., but at the end of the day the skill is mostly knowing how to properly and effectively leverage an external source of knowledge/wisdom/intelligence in a occupational domain.
ChatGPT is just Google on steroids in this respect.
Which doesn't mean it is not creating a new problem, does it? "A car is just a skateboard on steroids", still we created many laws around the use of cars.
I think this is the wrong question to ask. The real question is: "is it smart for a student to use ChatGPT instead of doing their homework?". And probably it is completely stupid. School is not an adversary, school is here to teach. If students manage to trick the system to get good grades while staying stupid, it doesn't really hurt anyone other than themselves.
This is debatable, at least in the US. In many cases, higher ed is here to offer a credential and network regardless of true aptitude since many students will simply not use the majority of what they are supposed to be learning in their career.
Put another way, students were already using things like ratemyprofessor to deliberately tailor their courses for easier grades back when I was in school, plus memorizing older exams rather than studying the core material in detail, or just straight up paying someone else to write papers for you. AI is an extension of this, not something new.
But saying that whether schools are for teaching is "debatable," while a popular meme, is a false and harmful assertion.
Regardless of what you or any of us consider to be "proper education" or "good teaching," even if higher ed really is just about getting certified for jobs, the simple fact is that the purpose of school at any level is to learn various skills and facts, and any form of direct-to-the-answer interferes with that.
I saw this a lot when GPT3 first came out, many people on HN and elsewhere decried the existence of book reports and essays, rejoicing that such "worthless" tasks would go the way of the dinosaur. However, all who say such things miss the crucial insight: the point of the book report is to learn and display reading comprehension. Similarly, essay writing is only about learning to communicate with essays specifically in the first one or two classes you encounter such tasks. After that, essays are used to communicate your command of the material for subjects that cannot be easily broken down into multiple-choice or short-form exams.
Allowing students to use AI tools to do their homework for them is a massive step backwards, and an unimaginable risk to the future of our society. If we allow our children to use AI to do all their work for them, especially during crucial reading comprehension years, they will arrive at adulthood unable to process anything more complex than a menu.
And relying on AI to summarize and communicate information that would otherwise be written into a document of some kind is an even bigger risk - then the controllers of the AI would essentially control how we process information in general, because no one would be able to do it themselves.
21% of adult Americans are already illiterate, pre-AI. 54% read below a 6th-grade level. What kind of world are we building if we push those numbers even higher?
The hard part is to get students to understand why the exercise is useful. It is easy to lose one's time writing an essay. But if you lose your time, it means you are doing it wrong; better not to do it at all.
Then instead of fixing the AI problem in your broken system, maybe fix the system? Maybe I'm being naive here, and we're doomed anyway.
I’m not saying students should use ChatGPT on their homework, but I would understand why a student would do so if it allows them to spend more time on side projects, networking, learning about other topics, etc.
You could potentially learn something by reading and copying the answers, or by trying to understand worked examples.
That much isn't necessary, GPT4 api costs $0.03-$0.06 for what's more than an essay of text.
It's affordable and accessible to anyone with a credit card. Ethically this is more fair than having to pay for over-priced textbooks.
The average person/studen can't install a llama 2 model (70 billion parameter) that can compete with ChatGPT-4 (the paid version), considering most students have notebooks, and lack top tier GPUs. Most would have to go to a "free" cloud providers.
ChatGPT-3.5 is also free, and compares favorably against the lower parameter llama 2 models. But, llama 1, lower parameter/heavily quantized llama 2, and ChatGPT-3.5, are not practically comparable to ChatGPT-4, for most tasks [1]. I haven't found a use for those, in this sort of context, except for increasing my blood pressure, with the time wasted being my justification for paying $.70/day.
[1] https://www.promptengineering.org/how-does-llama-2-compare-t....
It may be free, and easy enough to install. But to get it working, well I still have been unable to figure it out.
Before that you would get the solutions from your fraternity brother or sorority sister who took the course before you.
So for anyone who feels this is a new issue, it’s the same issue as any cheating as old as time.
Yes, one could ask their older brother to help. But not everyone, not for every single homework.
Yes, one could manually find information about a target and forge a phishing e-mail. But generative AI make it much faster/easier.
The scale means that it is different. Saying "it is not a new issue" is a bit like saying (for the lack of a better example): "well children could always throw stones at each other, letting them have machine guns is not a new issue".
FYI - This was not GPT4.
The interesting question here is whether figuring out whether an answer is subtly wrong is more efficient/effective than writing the answer yourself. It's probably more fun though.
Not knowing that something exists is a sure way of never demanding it. Having an education is even more useful today when all you need to do is knowing how to ask.
Really thoughtful and well crafted.
Goes from solid premises and goals, moving towards practical use of LLMs to accomplish them.
edit: Oh my mistake -- I didn't realize that the links in the article lead into more articles that do have examples.
EDIT: never mind. They broke Reader Mode. I give up.
In a typical test, she would give us all kind of articles (with the source: date, author, journal, country, etc) and ask us to analyze it. Not only it required us to understand the historical context ("the author is talking about this event that happened at this place for this reason"), but we had to show critical thinking: "he comes from this country and has this religion, so I would assume that he has a bias towards this", etc. To the point where we could even choose which questions we wanted to answer in the test. "It's okay to have completely forgotten about this event, just pick a question about some historical context you know". She was not testing our capacity at learning everything by heart (of course we had to learn something, otherwise we could not answer any question at all).
It took her time to establish that. Many students hated her, because they would usually easily make good grades by "learning the book by heart", and they were failing here. But at least for me, when I started understanding how it worked I also started loving this class. Somehow I did not feel like another pointless class where I would optimize my grades: I was actually improving my critical thinking. I was getting something out of this class other than good grades. Reading my history book now was not about learning facts, but trying to understand the context through the lens of the author.
I think that every teacher should strive to give that kind of feeling to their students. Then ChatGPT would not be a problem anymore: it becomes clear that it is not trustworthy. If some like to use it as a tool (e.g. because they need ChatGPT to give them keywords that they can use in further research), that's fine. Those who take the output of ChatGPT for granted have just missed the point, they may as well do nothing.
"If you are looking for ways to use ChatGPT to support or replace your writing, here are five different ways to explore."
https://www.zdnet.com/article/how-to-use-chatgpt-to-write-an...
Meanwhile, every search engine, startup, and tech company: "Quick, put this in front of customers and stamp our logo under it!"
You need to tinker the prompt to get it to follow the instructions all the time, but usually it’s pretty good about it.
Take a look at chain-of-thought prompting or the reAct paper for some research into it.