A Year of Spaced Repetition Software in the Classroom
lesswrong.com
lesswrong.com
As a method of developing real competency in a subject or practice, though, it offers none of the benefits of immersion as it keeps the student just on the edge of full engagement (and if we add immersion, we lose the time-savings that spaced repetition offers.) It introduces a fragility into the student's schedule that make lapses in 'discipline' more likely (not everyone has a teacher and regular subject periods to keep them on task.) For anything beyond basic key-value learning, it requires too many contortions and restatements of the subject -- to fit the format -- to be feasible for many students/subjects. The packaged decks available online help but usually don't meet the needs of the specific course or subject concentration the student requires. We can introduce variability to space how to solve a problem instead of a 'what' answer, but then we need to consider muscle memory, developing intuition and other things that the spaced repetition model does not address.
Can you say a little bit more about what you mean by immersion? Do you mean - say, having a teacher who can show you a problem from multiple angles, allowing you to triangulate your understanding by asking numerous questions... etc? Things like this?
I do agree that in terms of developing immediate understanding - having access to this environment allows you to obtain understanding quicker. And I agree that it doesn't cohere well with spaced repetition learning.
However, the problematic thing with learning with this way is that unless you go on to use it daily, you lose almost all of it. So all you manage to do is pass tests throughout your life - never developing a full suite of knowledge that you can deploy. What an enormous waste of time!
Given that the energy needed to learn/recall an individual card drops exponentially over time, over the long term this is a vastly more efficient process for retaining information.
The only question is whether it can be used to learn complex and abstract ideas. I believe it can.
Here's how I typically progress with learning a difficult, abstract mathematical concept or problem.
First pass - I'll rote memorise a solution.
Then after a couple of weeks - I'll typically forget various parts of the solution. More often than not these will be sub-problems that I don't completely understand. So I'll create new cards that will provide training on these smaller sub-tasks. In the meantime, I'll rote learn the solution to the larger problem again.
What I find is that no matter how many times I rote learn a complex solution, eventually I typically forget it. But the smaller, easier concepts stick. Eventually my rote knowledge gets swapped out progressively by the smaller units of understanding and I end up being able to work through the solution without remembering it as a whole at all.
This is a frustratingly slow process in the beginning - particularly if you are starting at the very beginning of a new field (as I am with maths). But in my experience it's worth it. For the first time in my life I feel I'm actually really learning a topic in a way that I'll truly own.
Mostly this doesn't add much overhead to the learning process. As I'm working through a math's text book (in ebook or pdf form) I use the windows 7 snipping tool to create two images - one of the question and solution. And then I just insert these into my card program. This takes about 15-30 seconds at the most.
I use mnemosyne as my repetition software which allows me to insert images - and I can also annotate my cards with latex. I have a number of latex macros set up on my keyboard so that I've become pretty quick at inputting latex.
I continue to annotate my cards as I progress. For a complex problem, what starts out as a simple cut and paste from a text book grows into a voluminous set of notes covering every aspect of the problem that I've had trouble with on different repetitions of the problem. These notes become what I use to create the smaller cards.
For abstract and complex material - there is no substitute for creating your own cards imo. You need to read through a text book anyway to ensure that there is no important context that you've missed. And it isn't much overhead to create your base set of cards, cutting and pasting as you go. But your gaps in understanding are your own - and only you can identify those and fill them.
As you say, it requires discipline; but if something is important you'll find the time, if not you'll find an excuse.
Sivers: "Memorizing a programming language using spaced repetition software" ( 2013)
https://sivers.org/srs ( HN: https://news.ycombinator.com/item?id=5015183 )
---
Janki Method: Using spaced repetition systems to learn and retain technical knowledge.
http://www.jackkinsella.ie/2011/12/05/janki-method.html
https://www.oxbridgenotes.com/articles/janki_method_refined
---
Gwern: SPACED REPETITION : http://www.gwern.net/Spaced%20repetition
SRS is a fun way to learn or else I wouldn't have kept at it so long. I would even go as far as saying it's made my personal projects a little bit easier for me to execute on. However.....
What I don't know is how applicable this is in the "real world". SRS systems don't translate well in the "Learn-in-5-days-or-die" approaches to IT training. As for benefits for work-related development and maintenance, I don't have a clue one way or the other.
I've studied a number of different subjects in my life, from foreign languages to computer science to math, etc. I clearly see the benefit of SRS and necessity of memorization in foreign languages (vocab is so critical, and there's really no way to avoid straight memorization of it), but I don't see how you could effectively encode your knowledge of a programming language on to flash cards. And I'm not sure why you'd want to when you can easily look up things in the documentation or on Stackoverflow.
I've occasionally wondered if an Anki deck composed of Clojure idioms would be valuable. It would be dependent on paredit conventions, teaching "shapes" along with symbols.
The reason I started it was because I just kept forgetting the names of functions. I'd know exactly what I wanted the code to do, but then I would find myself not remembering the which word to invoke to do it. The flash cards have helped a lot there.
I'm not sure how you would fit an idiom into a flash card, but try it out and see how it goes.
Why would you memorise foreign vocab when you could just look up what you want to say in a dictionary or phrasebook?
It's a silly analogy, but there's some merit in the comparison. Searching documentation takes time - perhaps not that long, but enough that if you had to do it constantly you'd become frustrated very quickly. If you look at some code you wrote recently, it's almost certain that there's a bunch of standard library function calls in there that you didn't have to think twice about writing. The same goes for basic programming patterns. The more you expand your repertoire of memorised functions and patterns, the easier it becomes to go from design to code.
Right, very much so, because a live conversation using vocab is very much different than anything you'd do while programming.
> The more you expand your repertoire of memorised functions and patterns, the easier it becomes to go from design to code.
Yeah sure, but the criticality is much less, isn't it? Like it's not the rote memorization of function calls, or standard libraries that makes a good programmer. Design patterns, idiomatic language structures (i.e. how do I write truly "Pythonic" code), etc are all way more important than, say, what does library X provide or what are the args to function Y.
Flash cards don't address the important parts of learning a programming language. And if you focus on the important parts, generally you'll get enough experience in the language that the things you'd want to memorize naturally just stick.
My 2c.
Idioms and design patterns are also very well suited to flash cards.
If I had to switch between perl, python, ruby, php, lua, javascript all the time, I would consider Anki to keep the rosetta stone data in memory, but not otherwise.
Stackoverflow and documentation serve to complement my Anki deck. I learn what I need and apply it to my project. After that, Anki cards get created.
Ultimately, I just see "stuff that you can google" as supplements to memory rather than replacements for it.
I think I'm struggling to see how flash cards would be an effective practice session. Why not just code something? Start a hobby project, contribute to something open source, etc. You'll use everything you'd rote memorize in context (and naturally start remembering it).
-------------------------------------------------- Front: Setup a flask view that requires basic authentication. Use the httpauth extension.
Back: How did it go? --------------------------------------------------
I have a lot of cards like this where I put the onus on myself to code up my own samplers.
My hobby is one where I'm playing around with Magic the Gathering data in JSON format. Over the past week, I designed a database, wrote SQLAlchemy scripts to populate it, and built up some RESTful web service views over it in Flask. It's been a lot of fun.
There'a a natural harmony where my performance on Anki reviews boosts my project work and vice versa. If there's a "key thing" to it all, it's that I have a project flow that works for me and makes me happy with the whole process.
There's syntax and there's idiomatic library usage.
My problem wasn't that I'd forget what the technical thing was, but that I'd just forget the vocabulary to talk about it.
Looking things up isn't that difficult, but it's much better to remember a term and say it than it is to halt a conversation while you look it up.
I used Heisig's Remembering the Hanzi to learn the characters and then would practice them in a space repetition system. A few years later I can still write most of them. His method has you create a story and make mental image associations for each character. Eventually you forget the stories over time, but the character usually sticks.
We were introduced to the characters in class and subsequently learned how to write them. So after being introduced to a character I repeated it on paper for a couple of lines. I repeated this exercise from time to time. I also read the stories of the text book out loud and listened to the recording of the texts. But as I've pointed out in my other post, you have to have context to really learn a language. Just the fact that I used several types of learning doesn't mean that they weren't kind of shallow.
The technique you've delineated above certainly sounds like a better approach.
Also after six months, if you're successful in your reviews, Anki should not expect you to review your (old) cards before several weeks, so it's strange that you forgot most of them for your test if it occurred in that range. It can also depends on the type of test I guess, how was it?
Try a different Anki deck, maybe. I've been studying Polish, and a good portion of my cards are phrases or full sentences.
In other words, they have to be memorized. No connection between writing and pronunciation.
The test went well. It was some weeks after the test that I began forgetting.
> however I think you have to make some special effort to actually use the words (in full sentences) when you learn them if you want to be able to reuse them in the future.
yeah, I absolutely agree. I think context is key. Nothing beats being forced to use a language.
Simply put, the biggest single mistake learners of Chinese make in learning Chinese characters is learning them only in isolation and not also by reading connected Chinese-language text in which Chinese characters appear in "compounds" (meaningful Chinese words). The full argument for doing this is developed by the late John DeFrancis (a brilliant language teacher and textbook author) in the front matter of his book Beginning Chinese Reader,[1] which is still a very worthwhile book decades after it was first published. DeFrancis built in a lot of spaced repetition in the book lessons, back in the days before "spaced repetition" was a trendy term, but he crucially also makes sure the lesson texts illustrate the various ways that Chinese characters enter into compounds to form high-frequency words in the modern Chinese vocabulary. Try it if you'd like to more Chinese in the most efficient way possible.
[1] http://www.amazon.com/Beginning-Chinese-Reader-Part/dp/03000...
P.S. for a more extensive discussion of how to learn natural human languages effectively, see a comment of mine from about two years ago that was very popular with other readers of HN:
I'm right in the middle of the John DeFrancis series (just started Intermediate Chinese Reader volume 2). These books seem to be almost forgotten and I haven't found any modern equivalent, so I was starting to feel slightly lonely in my quest for Chinese reading fluency :) I started them after trying
* regular flashcards: not very effective for me, as all the accumulated knowledge would disappear very quickly after I stopped practicing
* "modern" graded readers: they're nice, but far less structured than the DeFrancis' series, and they often stop at a beginner/intermediate level.
What's truly unique about the DeFrancis' series compared to other graded readers is that the texts are adapted to the vocabulary lists, rather than the other way around. Therefore characters and compounds can be introduced logically, in a coherent manner: for example each new character comes with a cluster of words that use this character, making the whole much easier to remember. The downside is that the texts often feel contrived and boring, but if you're serious about learning to read it feels much more effective and gratifying on the long term. The approach used by most other readers is to start from a simple and enjoyable story, and then adding a glossary containing the more advanced words in the story. But although the story itself is more natural and enjoyable for the reader, the vocabulary itself is much harder to remember.
It would be awesome if someone was willing to put in all the hard work it took to craft the DeFrancis' series, and write something similar with a more modern and less academia-focused content.
Also, for absolute beginners this method has the advantage of forming a solid understanding of the core of the language (those 1000 words that see over 50% usage in regular conversation) and also learn accent really well - cards being in audio format. It feels as if I were a small child learning his native language.
It's not really so much a downside as an unrealistic expectation, like expecting to keep weight off after you've stopped dieting.
Yeah, it doesn't work that way with SRS. If you're not actively using what you learned with your SRS (like say using Chinese on a daily basis) of course you're going to forget it! It's the same with bodybuilding: You have to constantly train your muscles and eat enough calories or they will shrink back to what they were before. It's not like you can gym, get as big as you want and then think if you stop you'll retain whatever you built :)
On one side you have the biology/chemistry guys who tend to use a lot of flashcards, and have a reputation for "just memorizing" facts. Then on the other end you have the physics/computer science guys who often refer to themselves as "lazy" for lack of memorizing and have a reputation for only focusing on conceptual ideas.
It seems to me that the people who succeed utilize a mixture of both approaches ("memorization" and "conceptual understanding"). I'm not sure when these two approaches suddenly became mutually exclusive.
As an example, although there were some classes in Biology that required a heavy amount of memorization, without the flash cards it would have been very hard for me to make the conceptual connections during lectures, while reading, etc.
Knowledge and understanding is kind of like a spider web, the more you have, the more you are able to grasp.
Purely focusing on conceptualizing the topics at hand has never worked out as well as I had hoped.
To further that point, during exams some classes allowed 1 sheet of paper with anything you can put on it. I created some super dense cheat sheets in my time and I am fairly sure that I never referenced them but once or twice in 4 years of school. But again the active of creating them and sorting out important and unimportant information was key in solidifying knowledge for the long term. If someone handed me those cards or sheet they would have been useless to me in the long run is at least how I see it.
The other thing I learned in school was anything I crammed for I knew I was not going to learn. It may have been enough to get through the next couple of days but it was not going to stick. I took some summer school courses and found it to be a waste as it was too much like cramming as it was so condensed.
I guess I did well in school by using something like SRS which was allowing time between study sessions for information to settle and allow subsequent review after time. There is probably more than one reason for university courses to be on alternating days as it probably facilitates this kind of thinking. The only thing is that SRS seems to be focused on much shorter time periods than a day or 2.
When I was preparing for the GRE in 2010 I first wrote down all the words and definitions from both the Princeton Review and Kaplan GRE vocab books on a legal pad. That didn't work as well as I'd hoped for quizzing (it's a pain to try not to read 90% of the page) so I wrote a really basic data entry script in Python that I used to input the words/defs and then a second one that would generate random questions based off of them (definition + 4 words as MC).
By the time I was done with typing them all in and debugging my little quiz script, I'd say I already had active recall on >90% of them. Having that working quizzing tool at the end to practice recognition was just icing on the cake - creating it in the first place was more useful to me.
The most important takeaway for me was accepting that the tougher something is to remember, the greater the benefit will be when we actually remember it.
Probably not a millionaire-in-5-years or one-man-lifestyle business. More of a improve-the-world kind.
This is a classic problem in language pedagogy.
Some old comment of mine from almost a year ago https://news.ycombinator.com/item?id=8510165
https://news.ycombinator.com/item?id=8251569
https://news.ycombinator.com/item?id=8253351
I think "completing the circuit" is among the most important aspect of educational theory. Recognize and produce needs to happen in any subject, and a good student recognizes that SRS is but one tool for recognition, but others are needed to complete that, and production is an entirely different side of the coin.
> Given the seemingly objective nature of Anki it might seem counterintuitive that the benefits would be more obvious in writing than in multiple choice, but it actually makes sense when I consider that even without SRS these students probably would have known the terms and the vocab well enough to get multiple choice questions right, but might have lacked the confidence to use them on their own initiative. Anki gave them that extra confidence.
Not that you're wrong, but the author also points out here that SRS can help production by reinforcing the student's knowledge and boosting their confidence in what they've learned when it comes to using it.
I have also developed a web service (http://slaypni.github.io/flashward/) for memorizing Foreign words using the library. Through the experience of using it by myself, I felt that spaced repetition helped me to remember many words for learning English.
I am feeling that most (free) SRS are using relatively simple algorithms (for example, Anki is based on SM-2 which is developed in the late1980s). Although I am not sure SM-15 is better enough, there could be some room to improve it. It is nice if the library was helpful for some developers who create new SRS.
I'd be interested in materials I may be missing.
I switched to using Anki, and the reading at least became quite cemented - I still struggle with writing at times, but it's a significant improvement over what it was previously.
Daily study is hugely important - if I go out and have a crazy weekend, I hate myself Monday, and my study session becomes 3 or 4 times longer than it would otherwise.
What I currently do is learn new kanji via Kanji in Context, and then create cards for them in my anki deck as I have learned them.