Why do we have such terrible evidence on homework effectiveness? (2022)
astralcodexten.com
astralcodexten.com
I'd love to see results more like these N methods will cover most people, and here's what you can do to identify which method is best for a person, and how to switch. But also, in education, it's not easy to provide multiple approaches for learning to different students in the same classroom, and it's not easy to provide very small classsizes so that many approaches are available.
Conceptually the problem is solved, it's one-on-one tutoring that was widely practiced in the ancient world for the aristocracy. The question is how to effectively implement that for the masses. Khan academy has attempted to use GPT 4 as a teacher bot in much this manner (to some success?), but the current gen of language models isn't smart enough nor designed to tackle this problem in a holistic manner. In a decade or two though? Easily.
Obviously there are numerous problems when it comes to deploying this sort of thing to the public school system. I mean, can you really trust a private company do handle it? Eventually though, something will have to be implemented when the ever dwindling number of underpaid teachers forces our hand.
Secondly, we can demonstrate that this method is some AI hype nonsense because there exist hundreds of books on basic topics like mathematics all written for different learning styles. The LLM is not bringing anything to the table to make up for it's major shortcomings.
Most importantly is that the human learning interface for children is far too vast for a simple model to attempt to interact with. Smells, hunger levels, nap phase, whatever. An LLM won't interface with it anymore than a pigeon can interface with a 24fps film.
https://www.psychologytoday.com/gb/blog/what-shapes-film/201...
And while I'd be the first to argue against blindly following what some dude says on youtube, he does have two degrees, has worked as an actual teacher and has been a lifelong educator. Plus has a team to fact check anything he makes. Kinda gives him a moderate amount of credibility on this specific topic in my book.
Yeah you probably can't teach first grade kids who don't know what a tablet is anything, even with a fully multimodal model. I'd imagine if there is any kind of transition towards this sort of approach it makes sense to start in university first, where the teachers are objectively the worst at teaching of all levels of education, and students the smartest. Then a level lower in high school once the systems work better, etc.
Hell, one of the docents at my university lab would give anything for an automation system that would replace the lectures and practice classes he grudgingly has to do, lmao, I think the entry hurdle for adoption there is rather low.
Covered well by Malcolm Gladwell's spaghetti story.
What is the purpose of the homework? What is homework? Is it literally just work? Does it take into account an individual student's needs? If it doesn't, then it's pretty obvious why that would be ineffective. Not all people are the same and, as a result, not all work has the same effect. I'm sure somebody has thought about this before.
But honestly, this line of research kinda reminds me of the conversation over weight loss and calories. Have you tried eating less? Have you tried studying more? Are we waiting for formal research to tell us that putting in work might lead to better results? Yes, there are individual differences. But unless you know you're an exception, you're probably not. And it not being perfect doesn't mean it's not a good policy.
When I was doing my math homework in high school, I really wasn't concerned about edge cases. That's work at a higher level than most K-12 students aim for.
It all seems quite haphazard, to the point that even in the same school the same subject can be taught very differently depending on the teacher. Anecdotally, teachers seem thrown to the wolves regarding materials or what works best.
I'm assuming none of those have held up?
Huh, interesting I actually did my graduation thesis on this when I finished my education degree. There is actually some well established literature about how effective homework can be. I wrote about it years ago on reddit: https://old.reddit.com/r/history/comments/jkt6l3/no_we_reall...
Cliffs notes and sources:
- For homework to be effective teachers need to give it with a clear communicated goal and make sure to also discuss the work done is discussed afterwards to reinforce meaningfulness.
- There is a clear limit on the amount of time students can spend on homework before it starts to have diminishing and even negative returns.
- Ideally there is a school wide policy where teachers are aware of the amount of work other teachers give and make sure the combined work doesn't exceed the threshold.
- Home situation matters a lot. tl;dr a clean quiet place to work is best.
- Parents role is important but not in the way they often think. The role of parents lies in "facilitating" the homework by, for example, offering the child a good place to study, helping to monitor time. However, parents have to be careful not to solve problems for their child.
All of these points need to be met in order for homework to have a reasonable chance of being effective.
* Cooper, H. a. V., J. C. (2001). Using Research to Answer Practical Questions About Homework. Education psychologist 36(3), 11.
* Donk van der, C., Lanen van, B. (2009). Praktijkonderzoek in de school. Bussum: Coutinho.
* Geerlings, T., Veen van der, T. (2008). Lesgeven en zelfstandig leren. (12e druk). Assen: Van Gorcum.
* Marzano, R. J. a. P., D.J. (2007). The Case For and Against Homework. Educational leadership 64(6), 6.
* Marzano, R. J., Pickering D. J., Pollock J. E.(2001). Classroom instruction that works: research-based strategies for increasing student achievement. Alexandria, Virgina: ASCD.
* Neuwahl, N.M.E., Bogaart, P.H.M. de. (1989). Huiswerk. Leiden: Martinus Nijhoff.
* Rosmalen, M. van, Dam, J van, Struijk, A., Vermeer, E. (2009). Wat maakt huiswerk nuttig?. Master thesis. Utrecht: Instituut voor Lerarenopleiding, Onderwijsontwikkeling en Studievaardigheden (IVLOS).
* Trautwein, U. (2007). The homework-achievement relation reconsidered: Differentiating homework time, homework frequency, and homework effort. Learning and instruction 17(3), 372-388.
* Trautwein, U. a. K., Olaf (2003). The relationship Between Homework an Achievement - Still Much of a Mystery. Educational psychology review 15(2), 115-145.