Trying to study textbooks effectively: a year of experimentation | Hacker News Reader

Trying to study textbooks effectively: a year of experimentation

lesswrong.com

156 pointsyoucould63 comments

I'm a big fan of spaced repetition systems. I recommend them strongly but would also like to offer some advice against common failure modes:

* Adding too much. Either from pre-made decks or just making cards for absolutely everything.

* Adding things you haven't learnt yet. SRS works best when you basically know something and want to still know it in 6 months time. Make cards at the end of a math study session when you have completed multiple problems. Don't make cards when you first encounter a piece of information.

* Treating SRS as a goal in and of itself. I see this in language learning a lot. "If I just memorise these 6000 words I will be fluent in this language". If you want to learn baseball you can't just swing the bat all day, you're gonna have to play some baseball games.

This is great advice. I've used Anki to memorize technical knowledge (not language) for ~10 years. Here are some additional points with some overlap:

* Keep trying. If you're having a bad time with it it's because you haven't figured out how to use it well. SRS must be useful for at least some of your knowledge. Don't add too much and get overwhelmed. Add fewer cards and try to figure out what makes a good card vs bad.

* Stop adding cards if you get overwhelmed. Do all your reviews for a while and the daily number will reduce to something more manageable. Let the magic of SRS work itself out. You get the real benefit when it starts spacing things in years.

* Start slow. Develop your skill of making good cards by making a few and observing your mistakes then ramp it up as you gain confidence.

* Delete or fix bad cards aggressively. Flag them if you can't fix them immediately.

* Use cloze deletion. This is like "[Paris] is the capital of [France]" and you get two cards where one or the other is blanked out. Much easier to create cards this way. Be careful not to make too many.

* Use image occlusion. It's like cloze deletion for pictures. Really excellent for visual stuff.

Using a spaced-repetition-system is probably the most evidence-based way we can get to long lasting memory. The success I’ve personally had with it makes me feel like it’s similar to downloading information to your brain, Matrix-style.

I’ve used Anki for language learning for several years, and I’m consistently surprised by what I can recall, and how that directly translates to reading content in that language.

For books, I’ve been using Readwise for at least a year as well — it will surface highlights to you in an SRS-like manner each day in a more lightweight way.

I agree with some of the other comments I’ve seen that we don’t need to remember everything. The author says they tried Anki, but dropped it because of the sheer amount of review it would require. But this is the point, in some sense. Studying with spaced-repetition is effective because it is tedious. So I’d only recommend it for information you really want to internalize, or languages.

I’m not trying to say SRS didn’t work very well for parent, but it doesn’t work “matrix style“ well for everyone.

I’ve sunk countless hours into Anki and I don’t feel like I have a lot to show for it.

If I could do it over I would use Anki less and read more out loud, and write notes on paper as if trees didn’t matter.

I feel like we are often met with the biggest success stories on the web with regards to tools and it can feel very discouraging when things don’t work as well for yourself. So my point is just to chime in and say “some of us also fail using these tools”

Would you mind posting some examples of the kind of cards you have in Anki?

I've experimented with a wealth of homemade and premade decks.

A lot of vocabulary decks based on textbooks used in class, e.g.:

    人 -> person

But I find that for more complex words, I reconize them in Anki, but not in texts.

I also did a large premade deck of example sentences from a grammar book, which I actually found to be very helpful for general language understanding.

I've also failed trying to complete Heisig's Remembering the Kanji, using Anki.

Have you tried using Wanikani? I have been using it for less than half a year and I already know more than 600 Kanji, most of which I can recognise in written text also. Of course, it helps if you reinforce the material through some other means, too, e.g. by working through a textbook.

disclaimer: not affiliated, and WK is not a perfect product (they have some weird quirks), but I've found it to be rather effective. You can also the first three levels for free.

What worked best for me was putting in sentences rather than single words.

So I might put in: ハンバーガーを食べる

Then there's two good ways to practice this. One is just reading/recognition. Can you understand the sentence?

The other way is to make it into a card to practise output: ハンバーガーを_____ (eat)

The part in brackets is a hint to help you get the correct word. Your hints at the start will often be direct translations of words, but you can also use images, emoji, dictionary definitions in the L2. Anything is ok, or combinations.

I often make my output cards very easy:

ハンバーガーをた____ (eat)

While it may look too easy, I still have to remember the word I'm studying.

In general words taken from real books, TV shows, youtube videos tend to stick in my mind a lot better than words from textbooks, wordlists and grammar books. Those tools have their place but they're not the be all and end all.

Good luck! And remember it's a long road. I've made 6000 flashcards over 10 years to learn a second language and I'm still far from "done".

I don't want to say your wrong about your own experience, but I think sometimes SRS can subjectively feel very unproductive even when it's working great because of the way it constantly surfaces material at the edge of your knowledge. It doesn't often give you the satisfaction of smashing through a series of problems you can answer easily. If it did, it would be wasting your time.

I also think there is a real skill to formulating good flash cards when using SRS that I for one don't think I've yet mastered.

There seems to be a lot of research about SRS but I never see any research about what makes an effective SRS flash card. Maybe it’s different for every person and/or subject. My foreign language skills improved considerably once I stopped putting english on my cards at all and used only the target language + pictures. But, like the person you replied to, using Anki for anything other than language learning has been mostly a waste for me. I do suspect it’s more to do with the decks I’ve created than with the SRS process.

I think this realization is one of the most helpful parts of Anki. Remembering stuff is always harder than we think it is, SRS forces you to face that fact head-on. It doesn’t mean “don’t use SRS”, I think SRS is still worthwhile, it just means you have to be more intentional about what you decide to commit to memory.

Personally, after 10+ years of consistent SRS, I focus only on vocabulary/terminology and fundamentals. Trivia is never worth it.

I don't use Anki, but I do use Space Repetition. I've explored different styles of stacks, and found that even for apparently identical underlying information, some formulations of that information work and are acquired easily, and other formulations simply don't work.

The underlying "atoms" you embody in the cards matter, and possibly matters a lot.

I am just finding more about this space . Correct me if I am wrong, My understanding is Anki is helpful when you want to remember some extremely detailed information (might even be considered as trivia by some). My usecase isn't like that. I just want to remember the main concepts of everything. Don't want to remember things like who developed that thing or when was it developed.

Have you been able to do this ?

Could you give an example of a "main concept" you'd like to remember?

Take notes on the main concepts / highlights, then make anki cards (: (I know it's easier said than done)

Shameless plug but I built an app [0] to help with this workflow. You can write note cards in markdown and then review them with spaced repetition.

[0] https://mochi.cards/

This is only what I use Anki for (main concepts). Just make cards for the concepts. For me, trivia is not worth the effort to remember, concepts + terminology are (terminology helps me understand and talk about the concepts, and search for the trivia when I need it).

https://en.wikipedia.org/wiki/Spaced_repetition

Spaced Repetition

Spaced repetition is an evidence-based learning technique that is usually performed with flashcards. Newly introduced and more difficult flashcards are shown more frequently, while older and less difficult flashcards are shown less frequently in order to exploit the psychological spacing effect. The use of spaced repetition has been proven to increase rate of learning

Correct me if I'm wrong but whilst there is a lot of evidence that reviewing learning material at regular interval is more efficient than doing it all at once, there is nothing that says that flashcard is THE implementation of spaced repetition.

There are some way to get things into long lasting memory that may be less documented academicaly but do work : learning by doing for instance. No one learns Java by putting all the elements of the language into Anki and learning them with Anki. One learns Java by writing Java programs.

I have tried many times using SRS and that's not something for me. I need context and enjoyment. Flashcards are too articial a learning environment for me.

SRS and learning-by-doing aren't mutually exclusive, in fact, I find that doing them together is much more effective than doing either one individually.

The catch is that, in my experience, SRS won't help you learn material you've never seen or used before. For programming, I make Anki cards for a given language/library/algorithm during or after the process of actually implementing something with it; then I can add my own code snippets into the card. For natural languages, I make a card only after I've seen or had to look up a particular grammatical construction or vocab word; then I can add my own example sentence, or the context where I first saw the term, into the card. So long as I'm regularly reviewing, then even for rare idioms or vocab, I can remember things I last saw years ago.

SRS is not as effective if you're studying prebuilt material by rote without any other interaction with the content. Rather it should be one part of a broad portfolio of activities you do to learn. For coding it should complement things like implementing projects, learning new libraries, reading open source code, reading technical books on the subject and so on. For languages it should complement things like talking to native speakers, reading news articles in the target language, listening to podcasts etc. where you use content from your other things and turn them into Anki cards.

Ah, the holy grail of not forgetting what you read and learn.

For this author, visualizing worked well, for others maybe not.

The sad fact is that all memories fade over time unless recalled.

My opinion: commit only the most important ideas to long-term memory. The rest can be looked up.

Indeed there's a legend that Albert Einstein refused to memorize his phone number because he said it was already in the phone book.

That makes me feel better about not knowing my address. I know how to get home and that's good enough.

Anyway, I use a PO Box ... that I also can't remember.

This reminds me of the Extended mind thesis. As long as you're capable of looking it up, it's part of your mind for all practical purposes. Storing it literally in your brain wouldn't add benefit in many (not all) cases.

This does not really apply to language learning.

It doesn't, and it's interesting to note why - the latency of recall is important with language because it's so often used in daily life and because low latency is needed for fluid conversation.

For more scarcely needed information, the latency of recall (200ms vs 10s) doesn't matter, and so it is often preferable to store it outside of the brain. It might still be important to store a pointer to that information with some metadata in the brain so that it can be used in conceptual reasoning. But the actual data can be stored elsewhere (e.g a phone number on a smartphone).

After playing with memory techniques as a child I've come to the conclusion that forgetting is a feature, not a bug.

If you're not remembering something your mind's "garbage collection" system has come to the conclusion that it's not relevant enough for active recall.

I'm for changing approaches to the material to change it's relevancy score, but forcing my mind to remember something's turns out usually to be pointless.

Also... A mind full of irrelevant visual associations from some mnemonic techniques seems littered and distracting.

> My opinion: commit only the most important ideas to long-term memory. The rest can be looked up.

Mine's similar: By default, take notes and figure out a way to search them quickly whenever needed. Only memorize those things where lookup adds too much latency/is impractical.

For note taking I've found Vimwiki to work well, and fzf.vim for searching. However effectively taking notes is something to be learned, and it's definitely not just "summarize everything you've learned".

> Only memorize those things where lookup adds too much latency/is impractical.

One particularly useful form of this is keywords and jargon: If you memorize the formal names of nebulous concepts, you’ll be in a much better position to look up the details when you need them. For example:

“Encoding scheme derived from a tree of symbol frequencies” => “Huffman Coding”

An internet or literature search for the latter term will turn up relevant results much more quickly than searching for the former directly.

> The old spark returned! It was fun again! I breezed through a chapter on transcription, and had no trouble banging through the homework immediately afterward. Not only did I understand it better as I went, I was having more fun.

Unfortunately, fun and ease is not synonymous with effective use of learning time. Novelty can be a good motivator for initial learning, but without spaced repetition retention is lower. People love the feeling of mastery, so doing the reading and homework in quick succession is a great form of dopamine. IMO you can either bite the bullet and recognize most of what you read will be forgotten in a year, or recognize that most of learning is boring.

When I ran through a stats textbook last summer, the authors clearly had spaced repetition in mind. Each chapter was divided into subsections with small problem sets as you read, and an end-of chapter question bank covering chapter contents (and then a cliff-notes chapter summary at the end). Each of the three major sections included a comprehensive question bank that are basically an exam. End result is that with a pace of about 10 pages per day, you'd do some reading and some problem solving, but every few days you'd need to reach back in your head to cover pages you read a few days ago. The end of chapter homeworks were often much slower going than the question sets sprinkled in, and the exams were equally rough.

To supplement that, every weekend I'd go through chapters completed that week and add some Anki cards. I don't necessarily need to know how to rapidly calculate correlation, as much as feel confident I can re-learn it later if I need to because I did so once already.

I've tried breaking textbooks down into Anki cards, and it takes forever and afterwards I'm left feeling underwhelmed. But, if you're going to do it, I've found that my favorite method of review is TTS on the Anki app with swipe gestures, and I just go for loooong walks while grinding my flash cards, just holding the phone at my side and swiping with my thumb '1' or '3'.

One thing I tried that I liked was reading the entire textbook out-loud into a set of bluetooth headphones chapter-by-chapter. I then process the raw audio file using Audacity's 'truncate silence' tool, as well as increasing the tempo of the audio file by 1.5-2x speed. I read it mechanically, straight-through as quickly as I possibly can on the first run. Then I make 1-2 passes back through the entire book using the audio file of my own recorded voice sped up in order to pace myself.

That way I go through the entire book like 3 times. Once slow, twice fast. I might do my second or third pass months or years later. Doing it mechanically means that I can grind through nonfiction technical books that are hundreds of pages long pretty quickly, and I know exactly how long my reviews will take, since it's the length of my audio file. The largest book I've pulled it off with was 900 pages. It was a sufferfest, but if you smoke a little pot while you're doing it, you can get into a nice rhythm and it's kind of fun.

I don't remember every single detail of the book using this method, but I've found cranking through them relentlessly with some time in-between to be very, very effective from a practical perspective. It scratches the 'done is better than perfect' itch, and when I need a specific detail I know exactly where to look.

First of all, you have to sit down with a piece of paper and a pencil next to the book and ban every electronic device in the vicinity. Then focus and do the exercises.

That's the method I recommend. It takes time, though, that I sadly no longer have. There are probably ten textbooks on my shelf waiting that I can't tackle for lack of time.

What worked best for me was to have paper next to me as I read, and every time I felt like I did not fully understand what I read, write down the questions I would need to answer to feel better about my understanding.

Often, the answers would appear later in a chapter. Sometimes, re-reading the chapter would allow me to catch answers to questions I'd missed. If questions remained after reading and re-reading, it was time to talk to someone else who had already studied the material.

For those in the thread interested in spaced repetition, Andy Matuschak has made a good writeup of some things to keep in mind for creating good prompts / cards:

https://andymatuschak.org/prompts/

I'd have liked to see at least a nodding reference to Mortimer Adler's classic, How to Read a Book. This isn't strictly a study guide, but that itself is a strength, in that Adler explicitly recognises different types of books and different levels of reading.

Records transmit knowledge. Not all human knowlege is equally facilitated by explitic (spoken or written) transmission. Different domains and topics exercise different skills: memorisation in the case of history and a substantial portion of practice lore, structured knowledge in the case of most of the sciences, and what might be considered cerebral skills development in areas such as maths, logic, rhetoric, management, negotiation, sales tactics, and relationship management. These are learnt in different ways, and books play different roles in their education.

Online: https://archive.org/details/howtoreadbook0000adle

Numerous HN submissions: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

Notably: https://news.ycombinator.com/item?id=12209446 https://news.ycombinator.com/item?id=177214 https://news.ycombinator.com/item?id=1002360

For math textbooks, one technique I've settled on is essentially summarizing main points and (importantly) drawing diagrams from the text. It's more work than just reading but I find the concepts stick very well especially for subjects like differential topology and category theory, since you remember writing it down and see it in your own handwriting.

I feel like this misses the point a bit.

Memorizing a text book has very little to do with learning a text book.

Read it again using another technique, and you may have a different feeling.

Memorizing and learning are the same process to me. What would be different is attempt at retention.

However, this is not strictly necessary because when you relearn a subject, you often relearn it faster due to previous experience(memory) with the subject matter.

If your goal is to retain skill/knowledge over time, than that's a different matter.

I don't think retention is the same as learning either (although it would be pretty depressing to learn without retention).

Learning is gaining the "wisdom" of the material. If all you're doing is rote memorization, you're missing the forest for the trees.

Having the wisdom is tough without having the knowledge.

Having the knowledge is difficult if you don't have the information.

I don't completely subscribe to Bloom's taxonomy of learning as the complete picture, but has at least some kernel of applicability.

The idea that there are different interdependent levels to mastering some field sounds true to me.

I'd agree if we were living in the 1800's, but we're not. We're living in the age where everything is googleable (Including the text book). You don't have to memorize things to have information.

Even in the 1800's, in a good library, I'd have more information available in a searchable, indexed form than I could quickly read into "working memory" to be able to "process" with.

Having a Grey's anatomy book in front of me, all available to read, does not make me able to perform anatomy at an advanced level.

It's similar to having data in secondary storage, that still needs to be loaded into a process's working memory, before it can be worked on. This is specifically true of condensed terminology that's used to compress further explanations.

Following the anatomy example - if I don't have the terminology to describe the different positional views available without doing an external lookup first, following a basic description becomes very difficult and involves so many lookups as to become impossible to follow.

I can't find the article/paper, but I remember reading something where someone calculated the trade-off for a specific API between knowing a percentage of the API method signatures of by heart versus looking up each one using the IDE documentation each time. It made a very big performance difference in the end.

BTW: knowing what you don't NEED to remember and can just reference when required, is probably part of the skills making you an expert.

Yep. Learning these things make it easier to google, not harder.

You might find this interesting:

https://www.amazon.com/Mind-Over-Machine-Hubert-Dreyfus/dp/0...

It’s a critique of AI but gives a thorough take on the progress from novice to expert.

if I want to learn something I want to understand it, so that in the end I'm having a high signal to noise ratio but if I memorize things I don't necessarily understand them leading to a low signal to noise ratio

It is difficult to remember something that you don't understand.

Yea this article sounds like a good strategy on how to get correct answers on a test of things you don't understand.

I just think the categorization of "textbook" is nonsense to start with. Of course you want a book with text as opposed to a picture book.

You have to start with the right book. The variability between books on a subject is so huge and probably even different person to person based on prior knowledge.

The advantage to memorizing a textbook is you can learn without it. So if you can memorize quickly but learn much more slowly you can use time that's otherwise lost (like the two hours of mandatory daily driving in the US) for learning.

I've been finding that I'm coming up against the same frustrations as the author.

I'm reading a bunch of textbooks and adding notes as I go to anki. I find that so long as I'm equally as willing to _drop_ a card from anki, it works well. However if I focus too much on this then I lose the flow of just reading and absorbing.

What are you reading?

Off the top of my head:

* Crafting Interpreters

* Real-World Cryptography

* The Algorithm Design Manual

* Some materials science book but I've forgotten the title

* Introduction to Probability, Statistics, and Random Processes

* (just ordered) AI: A Modern Approach

But when I say 'reading' I really mean that I've read the first few chapters and have gotten stuck on most of them.

Do you happen to publish your anki cards? I'd love to see examples of what your questions look like -- we have some overlapping interests! :)

Sorry I don't have any public decks, but I'd be happy to share them with you if you can think of some way of doing it (my email is in by bio).

I kind of doubt they'd even make sense if they were because they're crafted to the way I think, and mostly terrible.

Uhhh.. i think that they're crafted to the way you think is a interesting viewpoint on it's own.... byt the email on your profile didn't work..

Just replied to you. I'm in Australia so my timezone is different.

I have been recently integrating Anki into my book reading. If you are willing to share your Anki cards on books there has been a relatively new subreddit that might interest you. https://www.reddit.com/r/ankibookclub/

Thankyou! I'll join that sub.

I've made the Skiena deck public now [0], but remember it is terrible and no guarantees about correctness etc

[0] https://ankiweb.net/shared/info/236103555

I've been doing this as well, but I specifically try to come up with good questions and add those. Usually at the time I add the question as a flashcard, I don't know the answer yet, so I leave that field blank. Later on every time the flash card shows up, I refine the answer and return to the textbook if necessary. I find that the effort required to come up with interesting questions is very beneficial to the learning process by itself. I also sometimes include the name of the book and page numbers so I can quickly return to the relevant section.

This is a fantastic idea! I'll be incorporating this.

I've also run into this frustration and come to the conclusion that for me, no learning style actually works except for putting what I've learned to use, and doing it quickly. Granted, that means I'll probably never become a doctor, but I cannot force myself to memorize and "learn" something that I am not using, but if I currently _need_ to know something in order to use it, learning and retaining it is natural and easy.

Seeing how many blog posts this person published advocating different techniques along this journey, my main take away is that people should invest advice from blog posts extremely critically, if at all.

Reply on news.ycombinator.com