Increasing Retention Without Increasing Study Time [pdf]
files.eric.ed.gov
files.eric.ed.gov
Optimizing "study, retention phase, test" for greatest knowledge retention at a delayed test time, is very different from optimizing for greatest value of knowledge learned.
To optimize learning value, learn things that are immediately useful, you can immediately incorporate into learning something else, and ideally both.
The sooner and more you use something, the greater its value AND the greater your retention will be.
If you have to learn something valuable but with no short term use (how to handle a rare brain surgery complication), find a way to use it. Create an ongoing useful project that will revisit that knowledge during the "retention interval" (e.g. a concise summary of rare situations you need to handle, for you and others, that you can revisit and improve with additional and updated knowledge).
So optimize "topic choice", "topic progression", and "study, (optionally) test, use, use, use", for total value of learning.
"Use" is motivation, test, study review, and value realization put together.
If you don't use something after you learned it, you miss out on:
1. Learning how it is actually applied
2. Discovering how the knowledge is useful for you personally, in ways you may not expect if you don't actually experience using it
3. Deeper understanding and mastery of the knowledge
4. Much much much better retention
It is worth creating some immediate use for new knowledge, even the smallest possible useful or creative project, for better retention alone.
> learning a topic deeply as needed
That is the ultimate use-driven learning model.
As for non-sequential, I agree. The more we manage our own learning, the more it is a directed graph (i.e. prerequisites translate to many follow up paths), and eventually just graphs (many ways to order topics, and alternate combinations of prerequisites for each topic, in any complex area).
> By the end of Grade 3, know from memory all products of two one-digit numbers.
[1] https://www.thecorestandards.org/Math/Content/3/OA/#CCSS.Mat...
It is quite sufficient (and probably superior) to learn the facts through use, for that makes them robust.
In the US AFAIK they are called "times tables"; in Germany "one times one" („Einmaleins‟). It's funny what people emphasize for the same thing.
It encourages finding content from native sources, learning new words in context from materials that interest you rather than textbook content, and helps you build your own personal corpus of sentences as you encounter them in the wild. Will soon expand to more media types than just web and epub, to include YouTube/comics/HDMI input/game emulators etc
If you find flashcards from others, you also get easy tools for discovering more sentences from source materials that might interest you or from your own corpus. More coming for this and other word/kanji-level tracking analytics. Offline-first and privacy friendly.
I don't know how to contact Khaz sorry I've never talked w/ him. There's a subreddit dedicated to his approach r/ajatt so you might find leads there. I'd also check archive.org of course
If you try out my apps pls drop any feedback, easiest is Discord (chat or issue tracker) https://discord.gg/4aF9yuASzb Thanks - recently went full-time on this so looking to grow it quickly and have lots more on deck
It seems to me this paper is bringing to light the idea of spaced repetition for learning and this has become quite popular in the productivity/learning culture of today (e.g. Ali Abdaal).
> Alternatively, mathematics textbooks could easily adopt a format that engenders spacing.
I tutor middle school students in mathematics and this is definitely being implemented in their textbooks! At the end of each chapter, there a is normal chapter review practice test followed by a "Cumulative Practice" which reviews topics from the previous chapters in the book. These are especially beneficial to my students as, like the paper highlights, it promotes long-term memory of those topics.
> For example, although computer-based instruction typically provides extensive retrieval practice and rapid feedback, it offers a currently unexploited opportunity to schedule study sessions in ways that optimize long-term retention.
There is an immensely popular software called Anki which implements exactly this "spaced repetition"-type protocol.
Anki's data model is rather strange, which partly relates to its flexibility but has some unexpected downsides and trap-doors. I would love a slightly more opinionated SRS tool! (Maybe with a bit more UI polish, rather than rewriting the backend in Rust)
You can simply dump everything into a single collection, use basic 'front' and 'back' cards...and review. Yes, it may not be 'optimal' depending on how you mark cards, but for most people, it's good enough. Better than nothing, and probably an improvement as an addition to whatever they are currently doing.
If you want expert features, you can. You can drill, tag, organise, create different templates....but none of that is really necessary, or particulary pushed on you as something you should or have to use.
I mean, sure, if that's your thing and you enjoy it, go for it. But it is not necessary, and (SM2 vs FSRS arguments aside), it works right out of the box with the default settings and algorithm.
Like anything, trying to get that last 10% optimization is WAY more work than getting the first 90%. But "using Anki" (in any way) is such a multiplier over NOT using it that just taking that step is enormous.
This is a common complaint, but feels like a "delicate flower" argument. The UX is pretty spartan, but a fancy whiz bang UI isn't necessary to remember facts.
Not everyone needs all that, and the people that do (which is fine, btw), is a reflection of the person more than the tool.
(then how do you practise listening to your L2? Eavesdropping, ancillary encounters, news, movies, etc. give plenty of opportunity, but speaking requires you to make an extra effort.)
I understand time constraints and end goals may prohibit the 'best' approach.
What I do not understand is how can you say something is more 'efficient' if the yield in understanding is less than what you would get with another method.
Hopefully this will clarify my thought process here:
If 'Best' is to teach others and requires 10,000 hours to yield 90-99% understanding.
In contrast, 'Efficient' method requires 2,500 hours to yield 30-40% understanding. However, there is diminishing returns meaning that doubling your hours to 5,000 does not return you with 60-80% understanding, rather maybe closer to 50-60% understanding. With 7,500 hours closer to 65-70% and 10,000 hours may around 75-89% understanding.
Here you've spent the same amount of time but did not achieve the same level of understanding. I think you may have a dynamic 'Best' vs 'Efficient' curve and to switch between those options to optimize maximizing your level of understanding in the least amount of time.
Consider these two scenarios -
Goal: remember where to look up information when it comes up in $JOB Metric: how much you remember, how quickly you find the info
Goal: discover new hyper-efficient method of training an AI (or insert popular ML topic here). Metric: percent improvement vs current pubished best practice (deliberately vague) Required understanding to make progress: "like a Ph.D. from Stanford"
Now you can possibly measure something.
The idea achievement of "90% understanding" is VERY topic dependent. Simple topic? Sure 100% understanding, I remembered the Latin names of all of the plants in my house. Complicated topic? The information for "100% understanding" might not even be written in the textbook - it probably includes things like seeing the interconnections between the topics and being able to apply them in slightly different contexts.
Make sure you read the studies so you know what they're talking about. In this area, I think summaries are frequently misleading. You have to know what the real evidence is that substantiates the claims. (I cannot tell you how many times I have looked at the evidence and just rolled my eyes - obviously not applicable in settings where I wanted it to be.)
Edit: See this comment (not me) - https://news.ycombinator.com/item?id=41275869
Bjarne Stroustop, creator of C, famously rates his C knowledge as 7/10
How effective something is comes with what is best and compromise effectiveness for efficiency.
So what’s this about efficiency? You should be more worried about effectiveness, not efficiency. Particularly in this profession.
I've already heard that line somewhere but can't remember where and who said it.
“Every man has two deaths, when he is buried in the ground and the last time someone says his name. In some ways men can be immortal.”
1. https://www.goodreads.com/quotes/9556005-every-man-has-two-d...
A joke in Russian universities:
- A teaching assistant tells a student: "Look, I've been explaining it to you for so long, that I myself understood it!"
I figured it out, but I’ve no idea if I got him sorted out for his final.
The professor is interrupted in the middle of his lecture by a student who asks about one of the previous week's problems:
— Could you please work an example for us?
— No problem!
The professor writes a problem statement, then scratches his head, tugs on his collar, rubs his chin, and at last scribbles a complicated integral on the board.
— There you go!
— Excuse me, professor, but could you please be a little more explicit?
— What do you want from me? I just did the problem 3 different ways.
A soviet maths prof finds out he'd make more money as a labourer in the shipyards, so after more than a few carefully-distributed bottles of vodka he manages to change profession. A month or two working in the yards, and he sees a notice on the bulletin board offering a bonus to workers who sign up for a special "maths for the proletariat" evening school course.
Figuring it's easy money, he signs up, brings a novel, and sits in the back of the class reading instead paying attention. Then the prof calls him up to the brownboard, and asks him for the circumference of a circle.
Suddenly, our ex-prof blanks. Frantically he scribbles and scribbles on the brownboard, only to wind up with ... -2πr. No, that can't be right, but what went wrong?
Just when he's about to crack trying to figure out his error, he hears a friendly voice from one of the front-row students:
— Psst, colleague, swap order of integration!
I think overall on average they would do a decent job of explaining it in writing to equally brilliant people. Not so sure about "dumbing it down" for the "others".
That all sounds reasonable and smart, but the real reason I do it is the Curse of Knowledge. People in a system can’t see it from the outside. They make assumptions, use opaque or even misleading jargon, and employ circular logic. The new guy doesn’t know the lingo, or the circular logic. Their explanation will make more sense to the next hire than anything I can say. And having it written down this way can also give me new perspective on the system. Maybe it doesn’t have to work this way.
One must capitalize on this brief period because smart programmers are flexible and adaptable. Very quickly they will acclimatize themselves to the mess that surrounds them and they will become as blind to the deficiencies as the rest of the team.
People who don’t “get” the code ask questions that contain feedback they often don’t even register as feedback. By the time they distill it to an actual criticism, they’re often so wrapped up in the problem they can’t be constructive, or they present an XY problem. But if three people ask you the same question about your code? You have a design issue. Fix it. Fix it now.
That puts me in a weird relationship with FAQs. FAQs aren’t informational, they’re confessional. Here’s all the times I fucked up and won’t admit it. Let me explain why I am right and you are wrong.
But your phrase is spot on, well defined, and the subject of ongoing research.
https://en.wikipedia.org/wiki/Curse_of_knowledge
Cheers.
This guy wouldn’t have helped me though, since I never attended the class.
Perhaps it helps the topic was "increasing retention". :-)
I come from a similar cog psych background as the Bjork Lab, so am a big fan of their research, but books like 10 steps come from instructional design, which is a bit more focused on the big picture (designing a whole course vs individual mechanisms).
Just to name one example, folks might look into research on conceptual change theory (eg Chi or Posner). This theory helps explain why a concept like electricity is so challenging to learn. The reason, in brief, being that naive conceptions make a category error and think of electricity as a thing rather than a process. And this theory then informs instructional practice. Specifically, teachers should be aware of difficult concepts and should design activities that force students to confront the contradictions between their naive models and more accurate/complete ones.
Mickie Chi also has fascinations research on active learning (ICAP) and related work on the effectiveness of peer learning.
What error might I be making? For the future, is there a list of types of errors?
-----
Ignore this:
Spaced repetition, if you can stick with it, works. The details around schedules etc. don't matter as much as doing it in a way that you can keep up.
Also writing good prompts is important. I have yet to write more extensively about this but here is a start: https://news.ycombinator.com/item?id=41126734
https://www.scotthyoung.com/blog/the-best-articles-on-learni...
For instance, on memory:
I've always thought that the real problem is information relevancy. People need practical uses for remembering something beyond synthetic bullshit exams. Efficient techniques are great, but nothing demotivates more than not having a reason to learn beyond being told that you have to.
Nobody has to reach for flashcards, extensive notes, or advanced techniques when trying to learn something they're actually interested in, it's retained almost immediately and effortlessly. There has to be some kind of subconscious gauge of information relevancy that physically controls the level of absorption, a sort of "learning rate" if you will.
When you're motivated about something you can practice, you may remember more by doing it (which also acts a spaced repetition). But good luck trying that with, say, astrophysics or macroeconomics.
When motivated, you're also more likely to pick up other books on the subject, which is another form of spaced repetition.
Basically 70% of the semester most students are not studying 40 hours - they are doing 30 hours of real work, and perhaps only 15 hours effectively. And for good reason: Nothing is bridging or rewarding them in a way that interests/motivates them, for courses where the interest isn't natural.
A bridge to motivate them would be ideal. In 2021 I started using GPT-3 to generate motivational "reasons to learn a concept" cards for my flashcard app, - Revision.ai - which you can read about here in the 3rd item: https://www.revision.ai/articles/20ThingsRevisionAIDoesForBe... - the reason we disabled them was simple: we could never quite time the cards right to help the student when they needed it. When the app is closed, they aren't motivated - they don't see them. Mid study session? Showing such cards (or AI generated examples) interrupted the flow [https://www.instagram.com/p/CVVlIuVg31W/] We have also tried recommending relevant short/mid length youtube videos for visual/"breaks" from overwhelming learning. That did not boost student success either. I guess it doesn't address that you are still being told to do it, not naturally flowing into learning.
I welcome any technical or conceptual ideas you have to improve this and help increase interest amongst students. We have found that turning lecture slideshows into sets of exercises with clear visuals[https://www.instagram.com/p/C5ByftwiJ00/], breaking content up, and showing progress does motivate students to study more(and in a semi-related dissertation I wrote, possibly reduce Test Anxiety and tension). Please let me know of any ideas you have!
It combines reading and flashcards, such that it tracks every word and kanji you read and learn in order to show you analytics on what you need to learn to be able to read something or achieve JLPT goals, and highlights unknown/learning words in texts. Next up for the flashcard part is to replace the SM2 algorithm with FSRS for the flashcards, as well as having flashcards get passively reviewed simply by reading content.
I also suspect people are missing out on speed of learning when reviewing flashcards one at a time for hours. Besides actively recalling reviewing flashcards passively while reading, I’ll experiment with other review techniques like seeing a page at a time of vocab / revealable answers. Our minds absorb at the periphery of our vision and scanning/inputting a bunch of information at once, too. I’m unconvinced flashcard UI is the final expression of forgetting curve research based learning apps
Also working on Reader features such as manga/pdf/youtube/game emulators, plus expanding to all languages.
https://supermemo.guru/wiki/School_damages_your_brain
As someone who also publishes in neuroscience I can tell you this is total and utter trash.
I know this person as an author of "super memo" algorithms for spaced repetition learning, so yes, he most likely have a bias, probably even financial stake to some degree, although not sure, I don't follow him in any way, so don't know if this ever was or still is the case.
In the past, I've used some of his algorithms in particular, with quite interesting results - subjectively speaking, not in any scientific regime, And, obviously, those are just one of many. I also remembered this was his area of interest in general, so assumed his wiki to be of some interest in the context of discussed article.
Thanks for your critical opinion, it will contribute to my priors about his writing.
Then I have to go back and review.
Repeat.
> Because people forget much of what they learn, students could benefit from learning strategies that provide long-lasting knowledge.
What learning strategies were taught in your secondary educational years?
Unfortunately I kinda need to plan these sorts of days ahead, because trying to nap past 2PM is a good way to fuck my nighttime sleep.
I say this as someone who works on education and as a lifelong learner who has tried many ways of improving retention.
Rohrer, D., & Pashler, H. (2007). Increasing Retention Without Increasing Study Time. Current Directions in Psychological Science, 16(4), 183-186. https://doi.org/10.1111/j.1467-8721.2007.00500.x