Spaced repetition for study and learning

I've gotten into a rhythm, four new sentences per day which at my memorization rate works out to ~60 cards to review a day. Thus 20-30 minutes per day of review. It has become such a part of my day that skipping a day makes the day feel like a massive failure.

Its interesting how 4 cards can translate into 60 even with high memorization rates: for every card you add you'll see it again in 2 days, ~4 days, ~8 days, ~2 weeks, ~1 month, ~2 months, ~4 months, ~8 months, ~1 year... A review session is thus made up of multiple standing waves. The effect is noticeable you'll see a workload increase as the next standing wave of review starts. Right now some reviewed cards are getting scheduled next in ~8 years!

Using flashcards to learn mathematics, computer programming, physics, or anything requiring more creative or analytical thought is not so useful. For these subjects the number of raw facts required is usually smaller, and the thought processes, concepts and problem solving ability are much more important.

However, spaced repetition could be beneficial to these more creative and analytical subjects if it was applied more dynamically. In this case the units to learn could be the rules, laws, concepts which can manifest themselves in different ways within different problems, instead of being static flashcards. Does anyone know of software which works in this way?

My interest in this stems from http://readlang.com, a site I'm working on at the moment for learning foreign languages. I use SRS to keep track of words, but in future it could be nice to think about applying SRS to grammatical concepts too.

Problems arise in practical situations when you want to pass an exam. Or a couple of them in a certain period of time.

Proponents of SR will say that it takes at least 6 times the amount of time/effort over learning "short term". So if you are learning for an exam, you better have plenty of time to spare. And you better start early, like 6 months before the exam. All that makes it pretty useless in a lot of situations I've been in.

Short term learning isn't as bad as it looks. If you really "cram" for an exam, and you think you forgot all of it a couple of days or weeks later, that's not what happened. Instead, your brain is now primed for the material, and when you come into contact with it later, a lot of information is still there and re-learning will be easier.

Anyways, I ended up dumping SuperMemo because once my item database grew I became too lazy to spend 30 minutes (or more!) each day going through this drill. I might get back to it at some point, though.

Edit: By the way, SuperMemo offers a few more interesting tools.

One is "Incremental Reading", where you can import an article or text file into SM and SM will let you incrementally process it. For example, it might pop up as part of your repetition routine, and you can read a few paragraphs (you read as much as you want), and pick out the important parts of what you've read (parts you want to remember). Once you're done reading, you continue and the whole text will come up again and again, until you finish reading it and then you can dismiss it. The parts you cut out from the text become separate learning items and they too will come up in your daily routines, which let you further work on them until you ultimately reduce them into a Question & Answer form.

Another tool is sleeping monitoring. SM will let you track your sleep data, and it will tell you how your sleep affects your memory performance.

The anecdote where Hamming went to Bode's office to talk about Turkey was like a lightning ! Is it possible to see so many great names in one paragraph !

I've always seen some analogies in things with half-life and the charge and discharge of capacitors.

I'll give an example: I've always thought about learning as a capacitor charging through a resistance, and thought about the RC constant. In my mind, it takes relatively a short period of time to get to 63.2%, but it takes a very long time to get from 63.2% (it will never reach that, since it's an exponential).

What this tells me about learning ? Pick a field and then decide how much do you want to master it. Then know that it'll take you longer to get from 63.2% to 100% (which you will never achieve). An example would be the fact that English is my fifth language, which I only started speaking in 2009. Do I want to be an English major and write in a Kipling way, or do I want to have a command good enough to get my message to you (and possibly learn other languages) ?

It depends, but I find it a fascinating subject.

The other funny thing is that I used a mix of cramming and spaced repetition to study: I used to skip college for a year, and then get a month and go hard-core. Five courses. I'd draw a pentagon with the five courses and study two courses a day. 1,2 - 3,4 - 5,1 - 2,3 - 4,5 - 1,2 ...

You'll notice that I go back to each module each other day. The results I got from that are just astounding, as I was capable to memorize entire pages, chapters and I would see them. I would know exactly where I've written my notes on which pages. I aced the final exam of physics and had the best grade with questions that were never asked before (it was theory, and no 'values' or 'calculations' were there). I was writing things I've never written before, but I knew my maths checked out because I just knew. These weren't exercises or something.. I've been accused of cheating many times. A part of me found it insulting, the other part found it flattering as you only accuse someone of cheating if what he did was an accomplishment.

http://jeffhuang.com/Final_HaloLearning_CHI13.pdf

But wait, let's take a step back. Does it really matter if I can recall all this engineering material? Supposedly my brain "knows" enough of it such that if there were a big project in a field requiring that knowledge, then I could relearn it all, because I've been trained to think in a way pertinent to that material. Is my time better spent just learning new topics and building things, which is what I currently do? Why dwell on having perfect recall? It only seems to benefit test takers.

I would love to hear counter-arguments for why SRS for engineering materials would be a good idea for those no longer in school, maybe then I'll be motivated to build some gamified spaced repetition software. It'll need to be fun and reward-based, imo.

This means, every time I successfully recall the answer to a question, I increase the interval of when I will be challenged with it again by about 2.71828 times since I had seen it previously.

This is with a recall failure rate of about 5%, which I'm trying to maintain. (If it's nearing 0, then you're probably wasting time being challenged too frequently.)

This is on a data set used daily for 30 minutes, reviewing about 50 older questions and encountering around 3 new ones per day.

> At least with physical fitness there isn’t a precisely dismaying number indicating how far behind you are!

Many sports are precisely measurable and in such sports, the measurements determine who has won, to various resolutions. Shortest running time over a given distance, heaviest weight snatched, furthest distance jumped and so on.

Athletes in such sports can very accurately track their performance over time. As of yesterday I demonstrated that I can clean & jerk 25kg less than my lifetime record, for example.

FluxCards provides spaced repetition learning with easy to edit and create own cards. You can sneak the answer line by line, you can finger-paint and the stats are fast and meaningful.

It is free to use with 0.5, 1, 2, 4, 8, 16 days intervals. Beyond that you need the premium version but your cards are yours, so if you consider it not worth the fee, you can export your cards to SD and use them in Anki.

And yes, spaced repetition rocks. I can only encourage any flash cards programmer to implement it, as all these apps that implement flash cards but no spaced repetition learning are just not worth your time.

I'm also building a language learning startup that uses SRS. If you're interested in trying the system out in beta, or being notified when it's ready for general consumption, email me at languagebeta@jh.id.au and let me know which language you want to learn.

One interesting thing we've found early on, was that when we encouraged people to actually "take a break", not to repeat too much, but instead let your brain process and come back to it later, people didn't like it. They wanted the ability to keep training and training. This resonates well with what the article describe about students' cramming tendency. So some time you have to manipulate even a solid algorithm, to fit with reality.

[1]https://www.kenhub.com

Spaced repetition is pretty amazing. Adding records feels like you're entering them into your brain, provided you use it every day. I managed to go from knowing almost nothing to scoring 99% in a Japanese kanji exam once, entirely thanks to Anki.

My thesis is available here: https://github.com/jordwest/Thesis-Report/raw/master/report....

The online SRS I built is also available: https://github.com/jordwest/Membit

I always wanted to make my own SRS, one that had ease of use , stability and data integrity at its core rather than cutting edge features. This year I finally set aside time to make it: feel free to try it out and let me know what you think. It is very simple (there is only one deck), but multiple decks will be the next feature to be added.

http://benkyo.co

On top of that platform is an implementation of a rather interesting way to learn Japanese reading which it would also be great to have feedback on:

http://benkyo.co/iikanji/

One thing that I've wondered is whether it is a good idea to try to think up mnemonics for tricky words when using spaced repetition. It certainly seems effective in helping recall but I worry that I might train my brain to always rely on going via an indirect path to get to the information and that this will slow down my ability to understand real language.

Has anyone seen any research along these lines?

Email me at sean@einstein.com

It's not so useful in college now that I'm majoring in applied math, but occasionally I'll use it for a humanities class. (throwaway b/c I always use throwaways)

Thanks for all the references as always.

I think it's doing all right though. Here are some of the decks I'm most proud of:

- Using lilypond snippets and a midi-to-wav generator, I wrote a perl script that generates a deck for every (non-accidental) note on a guitar fretboard. It shows the fret position with a question mark, and then when you identify the note, it plays the pitch and shows the note on a treble clef. I'll probably expand it to include accidentals soon.

- Also using lilypond, I wrote a jazz theory deck for major chords. Questions like, "What chord quality and inversion is this?" where it plays a chord, and answer would be m7b5 (half-dim) 2nd inversion. My ear training has improved a ton. And, what is the key scale for bII7/iii in B major? (Plays the chord audio). Answer: B melodic minor or E lydian/mixolydian, playing and showing the scale on treble clef. And an effect like gwern described has happened. I haven't actually memorized the answer to every card in the deck (since the perl script generated about 2500 of them). Instead, I've gotten faster of doing the actual music theory processing in my head to calculate/recognize the right answer almost instantaneously. End result is that it has improved my speed in reading lead sheets. (It has not, however, improved my melodic improvisation abilities. :-) )

The rest of my decks are basically based off of Coursera courses. Unfortunately, I haven't finished any of them yet. I routinely run into the problem where partway through the course, the amount of time I need to spend keeping up with the new cards limits my time available to watch new videos and do new assignments, so I don't finish the course. I am now taking the Scala course again, and I'm hopeful that since I'm fully reviewed up through week 4, I might be able to power through and finish it with a deck that fully reflects the course.

This also means that I am improperly creating a separate deck for each subject matter, when Anki doesn't let you scramble them. I should be putting them all in one deck, but I kinda don't want to before a deck is "complete". I'm also a couple of chapters into "Learn You A Haskell", with the rest of the book on pause.

Finally, I can echo that practicing helps a lot. For instance with the Haskell deck, I got up to fully reviewed, and each card was still kind of hard. Then I spent a couple of hours playing with the first Project Euler problem. After that, the entire deck was much easier and faster. So, retaining familiarity still is somewhat different than learning the concepts. If I have a fully reviewed Anki deck from Odersky's Scala course (on coursera), it won't mean I know Scala. But if I go start a Scala project, the Anki knowledge will help me a ton, and then afterwards, the experience from hacking on the project will make the Anki cards much easier, maybe even to the point that I could delete a bunch of them.

BTW, that's feature I wish I had from the deck - some kind of advice for when it might be appropriate to delete a card. I worry that I'm doing the equivalent of writing pointless unit tests sometimes.

[0] http://sivers.org/srs

'cperciva noted this effect from his time at Oxford...

"When I tell North Americans about my time at Oxford University, one aspect of its undergraduate program inevitably surprises them: Final examinations. Rather than writing examinations at the end of each term, students in most subjects write a single set of examinations at the end of their final year, on which their entire degree performance is measured. (In recent years, some subjects have switched to annual examinations.) As terrifying as this might sound, the system has some merits: Whereas many students at institutions with termly examinations study the day before an exam and forget the material the day after the exam, attempting to 'cram' three years worth of material at once is largely an exercise in futility; as such, Oxford's examinations often provide a better measure of how much material a student has learned and retained. Given some of the conversations I've had with — and code I've seen from — people who have received computer science degrees from major universities and are actively working as software developers, I think that last part is very important: There's a lot of people out there who have forgotten some very basic — and very important — material" (http://www.daemonology.net/blog/2012-10-08-software-developm...).

You can obviously practice spaced repetition regardless of your school's exam structure, and Cal Newport specifies a method for doing this in "The Art of Stealth Studying: How To Earn a 4.0 With Only 1.0 Hours of Work" (http://calnewport.com/blog/2007/10/03/the-art-of-stealth-stu...).

However, most students are unaware of this method nor have the disciple to follow it on their own so to realize its benefits on a mass scale, we would need to revamp the education system around these principles.

And as 'qwern noted, sleep is a key factor in spaced repetition learning. Research shows sleep is when the brain prunes itself by separating signal from noise...

"Sleep researchers at the University of Wisconsin-Madison School of Medicine and Public Health believe it is more evidence for their theory of 'synaptic homeostasis.' This is the idea that synapses grow stronger when we're awake as we learn and adapt to an ever-changing the environment, that sleep refreshes the brain by bringing synapses back to a lower level of strength. This is important because larger synapses consume a lot of energy, occupy more space and require more supplies, including the proteins examined in this study."

"Sleep — by allowing synaptic downscaling — saves energy, space and material, and clears away unnecessary 'noise' from the previous day, the researchers believe. The fresh brain is then ready to learn again in the morning" (http://www.sciencedaily.com/releases/2009/04/090402143455.ht...).

I'll propose an alternative to rote memorization for learning: critical thinking.