Comment by mlyle | Hacker News Reader

Comment by mlyle | Hacker News Reader

Parent Full thread

Can you please provide some evidence that this kind of scripted and recitation-heavy instruction is beneficial compared to other approaches?

I've only seen pretty limited, pretty confounded evidence for it. A lot of studies I've seen are studies of students in charter programs, but these studies tend to ignore pretty big selection effects (e.g. comparing students to the general student population, when studies have found that students entered into charter lotteries who are not selected do about as well as those who get to go to the charter school).

I definitely use recitation in my classroom where there's a body of knowledge, but I typically reserve it for situations where it's clear that there's less need for deeper critical thinking or application of concepts.

As we look forward, it seems like there's a lot less value in having a broad body of knowledge and much more usefulness in being able to fluidly apply concepts in comparison to 19th century practice. Further, blab schools were really pretty demanding of attention span and cooperation and relied pretty heavily on corporal punishment to make them work.

I have pretty limited, indirect tools to get students to put in high effort. There's the gradebook and their general desire to do well, which isn't a terribly effective mechanism even though I am teaching an affluent, motivated group... and there's whatever social pressures I can foster in the classroom to encourage students to value performance.

> deeper critical thinking or application of concepts.

These things come after one has the basics down pat. Modern "Progressive" education rejects this point altogether. It's whole approach is entirely founded on putting the cart before the horse.

> Further, blab schools were really pretty demanding of attention span

Attention span is a function of engagement. As it turns out, hearing the lesson and blabbing it back until one has memorized it fully is a pretty engaging and even "gamified" activity, especially wrt. the most marginalized and disadvantaged students for whom other drivers of high effort mighy be not nearly as effective, as you hint at.

I asked for sources, not a quibble on a sub-point.

I disagree. I like rote and rigor, but I think it's a mistake to ignore developing problem solving and intuition early. A lot of programs overshoot, but figuring out how to make decent guesses and test them is important (as is getting lots of practice on well-defined problems).

edit: it looks like you're editing your comment. You added:

> As it turns out, hearing the lesson and blabbing it back until one has memorized it fully is a pretty engaging and even "gamified" activity

I disagree here, too. ;) I mean, yes, it can be, but we have other tools in our toolbox. The hammer is useful but has diminishing returns as we try and apply it more and more.

> developing problem solving and intuition early.

There's no reason why these things couldn't be developed in a more "structured" approach than the default (avoiding the overshooting you mention). The quick feedback cycle for every answer is really the most critical point.

> There's no reason why these things couldn't be developed in a more "structured" approach than the default (avoiding the overshooting you mention). The quick feedback cycle for every answer is really the most critical point.

Again, citations for the efficacy of scripting and recitation would be appreciated.

> The quick feedback cycle for every answer is really the most critical point.

I agree that quick feedback improves performance and morale. We close that loop pretty quickly in my classroom most of the time.

https://en.wikipedia.org/wiki/Follow_Through_(project) for starters, one of the largest educational studies ever conducted: "The results of Follow Through did not show how models that showed little or no effects could be improved. But they did show which models—as suggested by the less than ideal conditions of the experiment—had some indications of success. Of these models, Siegfried Engelmann's Direct Instruction method demonstrated the highest gains in the comparative study. [T]he models which showed positive effects were largely basic skills models. ..."

Ugh. I was wondering whether it was going to be Follow Through. You understand it was a terribly conducted study, and analyses of the data by other parties have drawn the exact opposite conclusion?

There's a reason why I'm particularly skeptical to what you're saying, btw: we know from pretty high quality research lasting decades that the combination of tutorial instruction plus mastery methods are supremely effective. The big problem is, these approaches don't scale.

Structured recitation in a classroom is basically the opposite of this approach.

On the other hand: Direct Instruction could be a way to hit a minimum quality level in schools which have suffered from instructional quality problems. It's also worth noting that modern Direct Instruction is much, much less recitation-based than you imply.

Just one piece of anecdata: the private school I'm at was much more scripted and regimented around this type of philosophy 15 years ago. The private school down the road is still there. We've really pulled away in performance since broadening methods and doing a lot more of the open-ended inquiry that you look down your nose at. Indeed, the engineering programs that I teach share very few features with DI, and have gotten nationally recognized results.

> Structured recitation in a classroom is basically the opposite of [tutorial instruction plus mastery methods].

Reference for this statement? From a rather abstract POV, it seems to be the closest thing to "mastery methods and tutorial instruction" that can actually scale to a large class size and engage all students by default.

Also, what method can be most effective in a "nationally recognized" engineering program (most likely with highly recognized students to match) has very little bearing on what's most effective for marginalized and disadvantaged students who may have significant challenges with basic skills.

> Also, what method can be most effective in a "nationally recognized" engineering program (most likely with highly recognized students to match)

Part of what got my attention and asking you for citations:

> > > As an important point, the merit of this kind of education is by no means exclusive to disadvantaged students!

For more anecdata: I coach a MS competitive math team as an elective. Anyone can join. It's not selective. Our school is roughly 3% of the middle school students in our chapter, but we routinely take >50% of the top 12 spots. We don't do rote but just explore ideas and compare approaches. Also, students in the program on average pick up about .7 years of math skills beyond what similar control students at our school do during that time.

I came into this job thinking I was going to be all teaching and demonstrating methods, and doing a bunch of drill-and-kill, etc. In practice that's where I've been least effective personally and where the classroom has been a sad place to be.

During COVID, I was a substitute teacher in an 8th grade science classroom and I was teaching physics. We recited definitions and drilled and killed. The students did well in the material, but developed no great love for me or physics, and didn't do better in their next year of science classes than students who had been taught in a more informal, exploratory way. The students that I taught physics still actively avoid my programs.

> Reference for this statement?

In tutorial instruction, students study materials beforehand and come ready to debate, critique, and defend ideas in "class." It's about engaging with concepts actively.

On the flip side, Direct Instruction is more structured and teacher-led, focusing on clear, step-by-step teaching with a ton of overlap between sessions and very clear structured measurement of basic tasks.

> In tutorial instruction, students study materials beforehand and come ready to debate, critique, and defend ideas in "class."

This seems to be what's often called the "flipped classroom" approach. You're right that this is not directly "teacher led" in a material sense, but only inasmuch as the equivalent effort happens outside the classroom. The "debate, critique, and defend" approach shares both the "immediately applicable practice" and the "quick feedback" features of DI - there's clearly "a ton of overlap" between studying a lesson on one's own and later debating, critiquing or practically applying the same content in class. Just because it might not be literal "blabbing back" doesn't mean that much of the same underlying dynamics is not involved.

> This seems to be what's often called the "flipped classroom" approach.

A flipped classroom falls short of tutorial instruction. Often, it's just recorded lessons presented outside school time to leave more time for using the classroom in other ways.

Homework in conventional primary classrooms is not helpful and may be harmful, and as best as I can tell, this includes flipped lessons and textbook readings. We've got a pretty big mountain of data accumulating on this topic.

I'm trying to flip some topics in AP Microeconomics and it's really hard to do in an effective way.

Actually: I find in AP Micro I am doing things much closer to how you describe, because I'm micro-optimizing for (students doing well on the AP Micro exam) instead of (producing students that I feel understand economics in a generally useful way).

> "quick feedback" features of DI

I think pretty much everyone agrees quick feedback and measurement is valuable. Having one clean expected answer that the class says together is one way to get this, and I guess it's perhaps the one that requires the lowest instructor skill and thus is most repeatable.

But I would say, for example, that Khan Academy does it much better-- pacing things to each student, providing individualized feedback, supporting spacing effects through mastery challenges, and allowing questions whose responses would not be said the same by all students. It's trickier, though: you need to stand in the back of class to tamp down on device misuse. And in the end students do work in exchange for recognition from their instructor in a well-functioning classroom, which is hard to get if they're spending most of their time trying to please a computer.

Somewhat offtopic, but I'm surprised that "device misuse" is that big of an issue. I suppose that these are most likely school-provided and school-administered devices, so they should arguably allow for some sort of time-based kiosk mode where the student is restricted in what they can do on the device. Aside from that, I do in fact agree wrt. on the potential of Khan Academy and similar systems - they seem to have the potential get closest to the "Bloom's two sigma" result of fully individualized instruction.

> I suppose that these are most likely school-provided and school-administered devices, so they should arguably allow for some sort of time-based kiosk mode where the student is restricted in what they can do on the device.

In our case, schools purchase and bring their own devices past elementary school. But even with technical measures, there are a nearly infinite number of ways to screw around.

> they seem to have the potential get closest to the "Bloom's two sigma" result of fully individualized instruction.

They don't, though. In my experience, there's 3 reasons why a student will devote effort to improvement in a classroom. In order of their efficacy and difficulty to instill:

1. Pressure from grades/the gradebook. In my experience, this is only weakly effective. Even in my environment where families are really achievement focused. There is too much of a delay; even if things are updated in Khan or the gradebook nigh-immediately, the measure doesn't become consequential for a long time.

2. Social pressures in the classroom: desire to not look foolish; relationship with an adviser; desire to please the teacher; effects of appropriate praise; desire to do fun things that other students are doing.

3. True interest and independent engagement in the subject.

You could alternatively view this also as a scale of how quick and effective feedback is. By #3, the student starts to measure themselves.

Khan or DI will have a hard time taking a student to #3. Khan's a super-strong, super immediate version of #1; DI is a very weak version of #2 (but possibly the easiest to implement).

> But even with technical measures, there are a nearly infinite number of ways to screw around.

Indeed there are! I was in high school around 20 years ago, and our computers were “locked down”. You couldn’t install anything (or really run any unapproved binary), access to system tools was disabled, and every time a computer rebooted, it would wipe a clean image to the HD so no changes would be saved to disk.

My friend and I figured out how to bypass the lockdown before boot, and made a floppy that did so. Removing evidence of tampering was as simple as shutting down the PC. From there, we added a menu to the bootloader, put it on a CD, wrote a simple Visual Basic app launcher on the CD, and included Quake 2/3, GTA, Worms, and a handful of other games that could all install or run from the launcher, and then sold the CDs for idk $10 a pop?

We also learned that every computer in our district (that we had access to test on) used the same admin password for all the remote VNC access, which was fun. And the same password (different from the VNC password at least?) for all of the networked printers in the district - on football game day, we printed hundreds of our school’s logo on a rival school’s printers, with the logo being just barely off-white.

There's definitely technical/fancy tomfoolery. But I'm also just surprised in very low-tech ways.

I once had someone draw a very elaborate stick figure fight in CAD for an entire 74 minute period instead of the high priority project his teammates were waiting on.

It's tricky for me in the engineering project classes, because I'll often be very deeply involved with one group teaching a new task or skill. 90% of the time the other groups will make good progress, but the other 10% yields interesting surprises. I definitely am more careful to show up unexpectedly more and to carefully look at screens, not just to shrug when they look OK-ish from 15' away.

Ha I had a classmate in one of my design courses who would make stunning art in Paint while we were all diving into Blender xD

> hearing the lesson and blabbing it back until one has memorized it fully is a pretty engaging and even "gamified" activity, especially wrt. the most marginalized and disadvantaged students

There are many kinds of marginalized and disadvantaged people and many require the opposite approach. I was very smart but had severe ADHD, was noticeably autistic, and my parents were poor at the time. Most of my normal public school classes were nothing more than repetition, rote memorization, and parroting back answers, no critical thought or deeper understanding of the concepts was expected. That was not engaging. That style of "education" had me failing classes and hating every waking moment of school. It was only the last year of HS that I started to shine after hitting AP classes with more interesting topics that required some deeper understanding and mastery. If I hadn't experienced non-rote classes my last year I might be a janitor now.

> Most of my normal public school classes were nothing more than repetition, rote memorization, and parroting back answers

Doesn't that directly support my point? The school system ends up relying on rote memorization even when it pretends to be all about having the students learn by themselves and exert critical thinking and open inquiry, as advocated for by the most "Progressive" educators! Isn't it then worth it to just get the rote learning part done with in the easiest, quickest and most effective way, by employing the structured approaches that are ignored by most teachers today?

I actually can't learn from rote memorization, as I cannot commit something to memory that I don't sufficiently understand. I just can't get those things to stick. I need to comprehend it and be able to employ its utility.

I also have a very poor working memory that hinders my ability to solve certain problems that most would find to be trivial. I think I would have done quite badly in the environment you're describing.

> I actually can't learn from rote memorization, as I cannot commit something to memory that I don't sufficiently understand.

I apologize but I don't believe these things to be literally true. You can clearly speak at least one natural language with native-speaker fluency, and that inherently involves committing a large amount of raw data to memory that may never be "fully understood" in a complete sense. More generally, memorizing stuff effectively that one does not clearly understand happens all the time in all sorts of education, and my view is that genuinely effective methods can indeed make this roughly as easy as memorizing the lexicon of one's native language.

(The success of spaced-repetition software is at least an existence proof that such methods are in fact possible, though the context is not exactly the same as classroom instruction.)

Of course whenever "sufficient understanding" happens to be feasible it should be included; I have never said otherwise. What "Progressive" education means when it pushes "open inquiry" and "students learning by themselves" is something radically different, that in practice amounts to letting the students 'sink or swim' with no effective support whatsoever.

> Can you please provide some evidence that this kind of scripted and recitation-heavy instruction is beneficial compared to other approaches?

Singapore/Hong Kong/Japan/Taiwan/Macau dominating the PISA

Singapore's math program in elementary is actually much less recitation and rote based than we are used to in Western mathematics education.

Indeed, it's very much pictorial and intuition-building in ways that fans of DI tend to look down on. It's concept and problem solving before rote.

I don't know so much about these countries in primary education, but I do have a few Japanese textbooks from secondary school translated into English and published by the AMS. This material also seems less rote-heavy than I am used to.

E.g. I'm looking at an on-level grade 7 mathematics textbook, and it's spending a lot of pages justifying the idea of negative numbers in addition and subtraction and with pictorial representation and has comparably few problems to do.

In a US math textbook, this material would have been done before grade 7, but in less depth. There would be a whole lot of rules, algorithms, and rote practice.

Reply on news.ycombinator.com