Measuring actual learning versus feeling of learning
pnas.org
pnas.org
>We find that students in the active classroom learn more, but they feel like they learn less.
>We show that this negative correlation is caused in part by the increased cognitive effort required during active learning.
In other word, "real" learning is hard, it takes work, it's not fun. The problem is that "the algorithms" (whatever recommendation ML algorithm YouTube has) works so damned well we are sucked into this endless shallow junkfood dopamine loop of videos that give us just the right amount of dopamine to click on the next one.
If you're struggling, you're learning.
For me it's the same reason I like video games. It gives me the same feeling learning something new, especially when I can connect it somehow to previous knowledge, that completing an especially frustrating section of a game I have to do over and over before I get it does.
I agree with you, enjoyment shouldn't be evidence of an education result.
Playing sports for example is a struggle, most people think it's fun, and you get physical benefits afterwards.
I don't teach all my classes in 100% active ways, but it has seemed to me that in the classes that are more active I get significantly lower student evaluations. People often rate my knowledge of the subject as less, for instance. That is, when I ask in class, "What do you think?" at least some of them perceive, at least a bit, that I am not sure.
I'm sure it's also true that forced participation in classes is doubtless tough for a lot of people. Again, not really an issue in business grad school, but I can see it being something of a problem in other settings.
I think this is correct, but the curve can't be too steep.
Active learning is about promoting things like talking to other students face to face or on classroom-based chat systems about assignments.
It's more of a social media flavored experience compared to watching a lecture. And it feels a lot less like work.
I don't think I've seen active learning implemented very well, but in my experience the more actively students work in groups and discuss their opinions the less actual material you cover.
What does happen in group settings is that the best students re-explain the material to students not doing so well. This might lead to increased average performance, but I'm sure they've accounted for that.
If you've never felt utterly defeated practicing a difficult passage in music or pulling an all-nighter on a problem set, then I question the extent of learning.
I've learned more from the chess and go games I've lost (leaving me in tears) than the ones I won.
Bandying opinions with one's peers is a form of learning, yes, but it's a nonrigorous one that I wouldn't pay tuition for.
Mainly calculus because that is be there level at which pedagogy theories are typically experimented with most. In upper level classes are typically straight lectures.
Edit: I meant "opinion" in the sense that you don't really have much to say to your partner about a technical topic you don't understand yet. And in much of math you don't have intuition you can verbalize until you've worked many problems, which means the time to have conversations with your peers about it is probably not the same time you're learning it for the first time. That may explain why in practice the students who read the book before hand are often really just instructing the other students when they are having discussions.
I think this is close, but reversed. Learning requires a struggle (eg effort), but you can struggle without learning (eg you aren't getting good feedback).
Struggling is feeling "under", less than, overwhelmed, and it brings fear or anger as primary emotions — red signs that this isn't a good path. By that point, you've switched a lot of attention away from the problem to focus on saving yourself, because genetic legacy is funny like that.
If you're enjoying however, you're learning to the best of your abilities, cognitively.
This isn't to say that the greatest games aren't hard at times. Difficulty can and should be enjoyable (it means you're in the sweet spot for improvement).
This isn't to say that enjoyment and learning are related in general, beyond the former enabling the latter in specific contexts. But when you're struggling, you're indeed not learning, you're stuck; what you need is a new teacher, a new way to understand the thing, which may be yourself — at which point you begin to "play" with it as we say it (words carry all the answers we need, don't they?)
However it's done, learning feels very good, like this inner "win" or "eureka" feeling. If you're struggling, you've failed. Already. You're in above your head. Even understanding would be partial and based on weak prerequisites, it's doesn't bode well for the next chapter, or the next task. Difficulty and pain are hard, but the good kind of hard, the one that teaches you, when pain becomes gain; it's a well known "truth" of life and the body by athletes, and the mind works just the same.
In fact, the learning itself happens afterwards, when you sleep: maximum learning is enabled by sleep in most mammals apparently. Who would have thought the most active learner in class is the one dozing near the heater, haha.
I do not buy the distrust of negative emotion. There are keys to the kingdom hidden inside fear and anger. Talking those feelings out and getting to the driving principle underneath them has been worth it's weight in gold.
The relevance realization pathway you describe is important, the successive eurakas are valuable. But when you go off into the unknown that is not structured like a language or a school topic, the Eurekas are several or more steps apart. You need to cope with the overload to grow your output. You are learning to deal with a character challenge.
I think the balance between being vulnerable with your limitations and fears whilst stimulated enough to push to the limit of your competence and minimize flaw in your output and the sword and shield to performance.
High speed learning is different to performance. It is to live on the boundary between being a fool and wise. Struggle is no issue, only credibility matters for truth seeking.
On the "distrust of negative emotion", that's a very good point. I very much believe that emotions are but the body's language, neither good nor bad, just informational. Fear, anger, sadness are powerful drives ("energy" in some circles), warnings too, and one should certainly listen to — but not be driven by — one's emotions.
I essentially mean to warn against two weaknesses that bit me back years down the line:
- partial learning, more mechanical than understood, typically because of weak foundations (and "struggling" is often a sign of that for me). I usually need to go one step back and "clean it" closer to 100% to unlock the previous struggle — I'll confess I still apply my good old math method to do dozens if not hundreds of exercises until something becomes automatic). For instance if past 1 chapter I still don't understand a paragraph of what I'm reading, I'll humbly admit it's too high level and take another book more suited to my level — and try to really ace it, knowing the steepness of following learning curve.
- (in the case of real difficulties to learn, prolonged, spanning multiple topics, etc.) failing to see the elephant in the room, typically because it's unrelated to learning, and that's pretty strong an obstacle in social settings like school, teams, etc. Imposter syndrome, health issues, and all that. Essentially, the idea that however difficult and overwhelming, learning should never feel "bad", like nauseating or abnormally tiring or depressing. Trying to switch on some "playful" mindset and seeing how that goes is a good indicator for me: if I can't even play, the issue is probably not the topic, but myself — cue break, food, meditation, coffee, talking, philosophy, whatever but fix the machine (me) before attempting to work it out more.
This is really about academic style or skill learning, classic lectures + exercises with teachers, online trainings, reading a book, practicing, etc. Life, more generally, is very much like you wonderfully describe.
Yeah the mysterious room elephant. Similar to the nuggets of gold underneath fear or anger, but not the same. I don't have a solution to that either. It seems we can only work so much and getting the preparation but also the supply to meet our need demands lined up so we can fulfill them consistently and keep work performance up, is a big life problem.
Using play as an indicator for unmet needs is interesting. I find myself alternating between work and play, but they are clearly defined as two separate tasks. And yes when I play it becomes much easier to prioritize getting my needs met than under the gun of work.
On the topic of positive emotion. Supposedly most of the positive emotion we get in life is in the pursuit of a goal. That may explain why we do best at the things we enjoy and the necessity of positive emotion for work. Maybe we have to balance the driving power of positive emotion with the crushing weight lifting negative emotion that comes from the challenge of working (and life in general). Too enjoyable and it's too easy, you're not working at your limit. Too negative and you're dying, you're likely to under perform but also not hit your target. Hmm interesting. I appreciate your post it put me in a mindset I had not explored in a while.
About positive emotions in work, my personal take-away — took me two decades to figure this out, but I think it's really true:
The goal is not to try really hard to always do what you love. It's about trying really hard to always love what you do. (Now or ever.)
I could write a book about why and how but this is probably not the place. One key though is exactly this playful mindset, it lets us redefine, in thought only, how we feel while doing something. We can always at least choose to love our effort, our doing it, try to mini-game quality or production — whatever makes it easier and ourselves better in one fell swoop.
She described her most productive work in life as being a form of "serious play". That has stuck with me and now you're saying something similar too.
Just for the benefit of all the people who will be struggling with burnout at some point in their lives:
If you are struggling and trying new approaches then you are learning.
Struggling and doing the same thing over and over won't end well, and isn't helpful to or expected by anyone.
The article compares active learning to passive learning like listening to lectures, and sees evidence that active learning is better. I don't think many people consider academic lectures more "fun" than active learning. The article also suggests that the reason why increased cognitive effort makes people feel like their learning less isn't because it isn't fun, but rather because students associate increased cognitive effort with poor learning:
> Most importantly, these results suggest that when students experience the increased cognitive effort associated with active learning, they initially take that effort to signify poorer learning.
-- The quote is about active learning, learning that involves the student in the education process. I don't think the subjective rating of that as "not fun" is a given. Rather, I think this subjective impression from the entire context of a student's activities.
It's pretty standard to expect "engaged" students to be happier than disengaged students. But students who are pressured to be engaged but don't engage are most unhappy of all. And today, students are confronted with a lot of easy, disengaged "junkfood" entertainment and are often working many hours a week along with school (or simply work endless hours of homework, etc). Under those circumstances, sure, learning that pushes the student to participate might get negative reviews. But I don't that justifies defining participatory learning "not fun" ipso-facto.
The more useful thing I discovered was getting the students to read the lecture notes before a lecture boosted the class average by 10% points. 10 minutes spent reading the lecture notes before the lecture and answering a few simple online multiple choice questions about the lecture was surprisingly effectively at improving learning. My assumption is it cut down on the novelty overload effect where the student’s brains shut down mid-lecture and so they were able to get more out of the lectures.
https://news.ycombinator.com/item?id=20980025&ref=hvper.com&...
> AWS purportedly puts design documents forward in the form of six-pagers. They start meetings with a 20 minute silent reading session. It's like the book club from hell.
>> Not just AWS- that's an Amazon-wide technique. And it's freaking amazing. You should try it.
In one of the classes I’ve taught we had a similar approach where we had students do a homework assignment beforehand where they would encounter the very problem that he material in the lecture would address. It did seem to make the students who actually did their homework more attentive.
Rather pathetically admin made me change this approach after a couple of years to two large quiz’s because I was testing the students “too much”. 3 quick questions every week was too much, but 30 long questions every 6 weeks was OK even if it didn’t get the students to read the notes.
It's important to distinguish between "performance" (how well you're doing right now) vs "learning" (how well you do after some time delay).
Compare blocked practice with interleaved practice. Suppose you're practicing calculating the area of different geometric figures of types A, B, C, etc.
Blocked practice means you do problems in the order of: AAAABBBBCCCC, etc.
Interleaved practice means you mix them up: ACCBAABACA, etc.
Blocked practice increases your performance (how well you're doing right now) because problems of the same type cluster together, i.e. you're able to "cache" the right formula and just plug in the numbers. But this doesn't help learning, because you're not practicing recognizing what features of the figure should prompt you to retrieve which formula from memory.
Interleaved practice reduces your performance, because more cognitive effort is required to retrieve the right formula, you might get it wrong, etc. But it improves learning, because you're training yourself to recognize which figure requires which formula.
So "desirable difficulties" can be introduced (of which interleaving is one) to increase learning at the cost of reducing performance.
[0] https://bjorklab.psych.ucla.edu/research/ [1] https://youtu.be/gtmMMR7SJKw [2] https://www.youtube.com/watch?v=oxZzoVp5jmI
One big issue I think is that the big lecture hall environment is something of an anachronism. There is something of a forcing function but, for the most part, with a modicum of production work, the video version can actually be more useful than the live one. (At conferences these days, which I mostly attend for the "hallway track" anyway, I find I usually prefer to watch the streaming feed than to be crammed in a room with 10,000 of my closest friends.)
The lecture format is just plain wrong. I can only assume it gained popularity in european academia because printed publication was too costly, and because the royal society liked to dress up and feel self-important?
I can't agree. I've been to many many non-interactive lectures and talks that were quite wonderful. It's a very different experience than just reading something. Of course it's not always better. But it's a wholly different experience.
And I'm pretty sure it pre-dates European court life.
The big problem with video, that makes me still value text based tutorials from the yesteryear (2000s), is that it's very hard to search for a particular sentence or string of information in a video. The varied pacing and delivery of information between speakers can get vary frustrating when you have to break your work flow to get to that exact moment in a video where the guy repeats what you want to hear inbetween ums and ahhs, and his 3rd time explaining a very basic concept.
Text still allows the reader to be in control of the pace & is the most easily searchable for the right information, which is vital for maintaining focus & flow at work. Video needs to catch up to that, hopefully automatic transcription can save the day.
With active learning, it's clear that early description of the approach, its effectiveness, and that it often doesn't feel like that, improves student approval, and outcomes(IIRC).
With 'belonging interventions', priming at-risk freshman with a narrative of "everyone here struggles - you get help, work hard, and succeed" inoculates against a narrative of "I'm struggling - they made a mistake admitting me - I clearly don't belong here", flipping a feedback loop of (not)forming academic and emotional support networks and (not)seeking help, and (not)succeeding.
One fun corner of medical reform involves medical interventions that are cheap, easy, with little downside, large upside, and no disagreement that they should happen. And still often don't. Promptly getting aspirin when presenting to the ER with heart pain, I recall as an example. It's been a long-term process improvement struggle that the profession has been pursuing for decades now.
One reflection of the state of education process improvement, is that this is barely even a conversation yet. And perhaps something personalized semi-automated teaching can help with. Everyone gets an intervention 'punch list'.
For example, checking and rechecking that everyone, but especially students from no-previous-college families, has an "education is something I create for myself with help" model, instead of a "education is something the teacher does to me" model, seems something we might actually do, to great effect.
one immediate question I had, after read the abstract, is how are they going to measure learning result. without looking into the testing material, I'd argue if the tests were not carefully designed, it could produce whatever result anyone likes, by adjust depth/width ratio of the given subject. maybe the testing materials are so well regarded in the field so there would be no justification necessary, but by ignoring the possibility to readers, it feels not trustworthy to me.
back to the explanation of the result, it's entirely possible that learners (and the authors) not able to draw distinctions between information and performance. I'd assume the traditional teaching would expose much more information to students (anecdotes, connections to other fields), but 'active' learning could give better result in performance building in a narrowed domain.
I'll read carefully into the paper sometime later, but for now, it's not a paper I'd recommend. (feel free to downvote if I get anything wrong)
“The instructors did not see the TOLs [post-class test of learning], which were prepared independently by another author. 9) The author of the TOLs did not have access to the course materials or lecture slides and wrote the tests based only on a list of detailed learning objectives for each topic.”
I’m sorry for the self-promotion, but it is extremely relevant in this case.