“The Suck” (Learning Anything by Writing It Out by Hand)
scottscheper.com
scottscheper.com
When learning to program, I found typing out code samples had a similar effect to writing them out by hand, and felt more natural to the domain. But in other subjects in school, it felt like my comprehension was lower when I typed lecture notes instead of handwriting them. I was never sure whether that was something inherent to hand writing vs typing or it was just because I had more of a barrier between typing and thinking than I did with handwriting.
It always seemed clear that the value of writing something verbatim was that it forced your brain to internalize it in a way you don’t need to when reading. It takes longer to hand write than to type — maybe that is the value of hand writing vs typing? Or when typing I needed to think more about the act of typing, whether that was thinking about formatting/positioning or just being less natural than handwriting? Maybe programming is just a special case where you always think through typing?
I lean towards thinking it’s somewhat domain specific. Clearly with physics or math if you had to typeset equations with LaTeX that would get in the way, and typing would be a barrier to understanding. With programming, every idea is expressed through typing, and so typing is a natural way to imprint ideas on your brain.
I suspect it's time to digest the concepts and when it's slower to handwrite versus type that results in better comprehension.
Daniel Oppenheimer et al.: Fortune Favors the Bold (and the Italicized): Effects of Disfluency on Educational Outcomes (Proceedings of the Annual Meeting of the Cognitive Science Society, 2010), https://escholarship.org/uc/item/4wd1s7hj
"Study 1 found that harder to read fonts led to increased retention in a controlled laboratory setting. Study 2 extended this finding to real-world classroom environments."
I'm not an expert on this topic, so I don't know whether this result was replicated since. If it indeed works, then kudos to my university's physics department for making use of this effect - they had a habit of handing out physics cheat sheets and various MATLAB code snippets that have been photocopied for 15+ years from one copy to another.
For me, yes. It's a question of speed. I type much faster than I can handwrite. For a complex unit of code, it's better for me to dump it as fast as possible so I can hold it in my mental pipeline as a whole.
Long ago, learning touch typing was a huge boost to my programming capacity.
For other stuff, I find it better handwritting so the ideas have the time to sink in. Programming seems a more complex activity.
For some of my undergrad courses we were allowed a 'cheat sheet' which is a single 8.5"x11" sheet of paper that we could write anything we wanted on and could bring in to the exam. It became a skill to carefully put an entire course's worth of information on a single sheet. But for me, interestingly, once I had made this sheet I rarely referred to it in the test. The act of making it was enough for me to remember most of it.
I'm way too lazy to write anything by hand though.
My process is to open about 20 browser tabs on a subject and cull them off one by one as I create flashcards from the information (process it, write bite-sized chunks in my own words). I learn via flashcards for about 15 minutes per day for a while. Nothing strenuous. That cements the 10,000ft view into my head before trying trying the hands-on stuff, so that I don't give up out of frustration. From there I try to use the information for real.
[1] https://youtu.be/fb_v5Bc8PSk actual study method starting at 4:04. What to do with the papers around 6:16
I would have preferred citing actual research, not an appeal to historical methods.
Given that some of the conclusions that word for word copy may be less efficient than summarized copy[1], it may be a less efficient and less effective way of learning than going through a reading and summarizing every paragraph.
https://www.scientificamerican.com/article/a-learning-secret...
That kind of agrees with the summarized copy idea you suggested.
Verbatim copying is good for bringing your attention to the material, it helps you notice more, and doesn't let you mind wander. While summarizing is good for cementing what you've noticed and learned.
For example, if I try to copy part of text I see the writing techniques used there a lot better. I don't see how summarizing can give me the same effect.
I'm almost ready to try both techniques simultaneously, although it seems like an overkill, so much writing.
I can at least provide some links. [1] is my own research on giving students optional typing practice in a CS2 course. [2] is Mickie Chi's overview of the ICAP framework which categorizes learning activities based on their level of engagement (Interactive, Constructive, Active, Passive). Chi's work notes that higher modes of engagement provide more learning gains, or I > C > A > P.
Copying would be considered an Active exercise and theoretically would not give as much learning gains as a Self-Explanation exercise ("summarized copy", Constructive). However, much of the research into self-explanation shows that lower-performing students do not provide good summarizations/self-explanations. Thus, in my [1] work, I make the argument that for these students, completing a lower ICAP mode (typing practice) is a better use of their time. While it does not provide as much learning gain as a Constructive activity, it can still give students some gains that could potentially elevate them to a mental model that can successfully complete Self-Explanations.
In your research area, is there a significant textbook or summary paper you would recommend that summarises current findings well?
What would you recommend to a complete amateur orienting themselves?
It depends on what you're referring to by "research area". My focus is specific to novel CS exercises like typing exercises, Parsons Problems, coding problems, etc. In that regard I really like Teaching Tech Together [1] as a broad, here's a blanket review of CS education and its exercises. If you mean more generally to just CS Education, then Teaching Tech is still starting point I think, as it provides a nice literature review of the domain as well.
[1] http://teachtogether.tech/
> What would you recommend to a complete amateur orienting themselves?
It'll depend (again) on what you mean here. If you mean learning about CS Education research, then the link above will be great. Then its about going down rabbit holes from the citations to read in more detail about those findings.
If you mean simply learning CS, the biggest recommendation I can make is carve out 1-4 hours a week (depending on your schedule) and commit to learning to code via MOOCs, tutorials, videos etc. Find a CS1 syllabus from a university that has a schedule on it and follow that. A lot of learning can be traced back to "time on task". Ignoring the recent HN post about how years of experience doesn't equal most skilled coders, that article is looking at what research calls "experts" vs. "novices" (beginners). We know spaced repetition works, we know cramming (trying to learn it all immediately) doesn't. Following the syllabus' schedule will space out your learning, force you to recall the information, and produce better learning in the long run.
The idea is to make it a part of your weekly "routine" to the point where if you DON'T do it, you feel weird. For example, I've trained martial arts for 15+ years. Somewhere in that time, I'm so used to going to train that when I take nights off, its weird because I'm just USED to training. Even my body wakes up cause its used to needing adrenaline. That needs to be a part of any learning process.
No worries there.
I'm simply curious about the state of the art of the pedagogy.
CS education is a fascinating situation. You have an interaction of strong mathematical, socioeconomic, and generally academic backgrounds, and often the complete opposite studying it. Plus very little institutional knowledge around pedagogy or anything else relative to almost all other fields of academia. And yet, there are all sorts of interesting factoids around the place, like https://www.youtube.com/watch?v=7J-wCHDJYmo.
I don't pretend get how it all fits together.
Thank you very much, Teach Together looks like the perfect starting point, and I can follow the citation breadcrumgbs from there.
Anyway, I'd love to see some research comparing, for example, the retention gained by writing stuff out by hand, to just reading the material multiple times until an equivalent amount of time has passed. It seems possible to me that what The Suck achieves is forcing you to slow down and take some time to absorb the information, but the method doesn't seem efficient/optimal in principle.
That matches my experience. If I'm being forced to do something, my brain is focused on compliance, meaning doing whatever it takes to get the authority figure off my back. But if it comes from intrinsic motivation, I'm focused on achieving a practical result as best I can.
My prior is, having a ritual to set your brain in learning mode helps activate it. For most of us (at least over 30 or so) the ritual of written notes was taught. But there's definitely been enough time to get some adults who used typed notes to learn.
The most interesting test to me would be a 2x2 comparison. People who learned to <type|write> notes, test them on recall of information <typed|written>. All four combinations so we can really suss out what makes the difference.
https://sci-hub.se/https://journals.sagepub.com/doi/abs/10.1...
Say it was very effective from retention/ memorization perspective but very inefficient in terms of how long it took.
I became dependent on this method for my preparation though. Which ...sucked I guess. Often felt like a handicap when I needed to quickly go through some material for review.
Through this lens, much of conventional education feels like a desperate babysitting tactic to keep the young busy.
<rant> For this reason I’m completely biased toward believing all of the positive stories about homeschooling coming out of the pandemic. The socialising argument against homeschooling has always seemed suspicious, and I completely buy that real socialising happens outside of the confines of school. From my own experience, this always feels true … extracurricular sports, music or out of school social events always seemed to be more meaningful and relevant than the classroom dynamic.
</rant>
The pandemic changes were dramatic precisely because they struck at norms, and that was the question and argument the whole way through: "it's not normal to have a pandemic" vs "the pandemic is the new normal." I remember overhearing a couple in the park in April 2020, one saying to the other, "when things are back to normal," and just thinking, yeah, fat chance. Norms are adopted for your survival, not the other way around.
With school the defined norms are actually quite strict and take up a huge part of the day. Most of the assignments are related in some way to meeting a norm, and this makes a certain kind of person - someone sufficiently willing to compete within and master the unspoken rules, or anxious enough not to fail - a "star student". But take the class outside of the classroom, and it's totally different.
From my perspective, it's not norms themselves being bad, just the particular set we've been herding along institutionally. We need some, some of the time, because the alternative is everyone being a philosopher, which besides creating unworkable arrangements, many people don't have it in them to try.
Once this occurs, it may be too difficult to reason about alternatives because competing within the unspoken game, as you put it, is too dominant a social factor and becomes the social air that we're all breathing. The educational institution and culture arguably becomes (and now the unworkable armchair philosopher comes out) a meta-norm or "Grundnorm"[1] from which others such as what makes a worth-while life, who has "prestige" and power (both real and earning) etc.
For a few reasons, I think allowing this to go too far is undesirable for a society. And with the West having transitioned in modernity to this idea that nearly everyone should be getting a tertiary education of some sort I do wonder if some major scaling issues and negative secondary effects come into play.
From my own peek into the workings of academia, one of the more disheartening things I observed was how Professors, the most successful researchers we have, were increasingly being burdened with duties not too dissimilar from those of high school teachers. Seemed to me to be a mismatch between the scale of the research and education sectors. And, I suspect, we're yet to see the knock-on effects.
~~~~
I have very bad vision which wasn't picked up for various reasons, so learning to write did not go smoothly (my handwriting is and always has been barely legible). So being forced to write out everything I learned by hand was a real bottleneck for me.
Things went a lot better after I discovered touch typing, and nowadays I learn best via keyboard shortcuts and typed commands.
The idea of "writing it out" is to give the student familiarity without higher level "figuring it out on their own". I also teach martial arts, and so copying solution steps, or drilling, involves showing students how to do a technique and then having them do it. Both worlds are attempting to build muscle memory so if the instructor says to "do a tenkan" or "write a for loop", you have enough exposure to that concept to do it without thought.
While it's commendable, it should also be noted that all this was done prior to the information age we're living in. These are physical objects that at the end of the day take up sizeable space in your house, gather dust, need cleaning/dusting etc. And you can't throw a computer at the problem of search and retrieval, at least yet.
I wish there was more research into active learning by typing as well. Typing does force you to think, especially if you don't type something you learn verbatim but actively try to process info and possibly summarize it in your own words.
Speak of the devil and he shall appear! This is a link to my research of "active learning by typing"! The jist of my results were - students performed better in the course, initially lower performing students benefit from the practice and by sampling having more examples to refer to, regular completers of typing practice submitted less buggy code to GitHub.
I think it's a powerful way to learn and remember, but requires some time and effort. Alas, I am not doing it very much nowadays, but that's also because my learning is much less structured and much more "operational" (I need to learn this in order to do that).
Edit: typos
For the interesting stuff where I was paying attention and did the coursework in time I didn't need to.
I wrote C code by hand. Mind you, I wasn't typing it nor copying it from anywhere. I was writing it, by hand, from my head. Hear me out...
I was on the collegiate ACM programming team with Russ Cox and Matt Caywood. The format was you were given like 6 puzzles, 4 hours, and one terminal. Russ insisted we follow this process:
1) One person picks the easiest and gets the terminal.
2) Other two teammates choose two other puzzles and each of us would write our code, by hand, pen and paper. ("The New York Times crossword puzzle is a form of meditation. But doing it in pen if pencil is available is grandstanding." - Glenn O'Brien, from _How to be a Man_)
3) When one was finished, we would acquire the lock on the terminal from the other teammate, transcribe our code, and hit run with our fingers crossed. No collaboration. No cross-talk.
4) Repeat until victory is achieved and your competitors are decimated. (We laughed with no small amount of schadenfreude that the MIT team lost in the regionals because they all stood around their sole terminal, imcomprehensibly bickering at each other.)
We got 8th place in the world, so---uhm---thank you Russ for the joy and pain? I wouldn't recommand to any CS student that this slow form of torture is necessary by any means to assert that you are damn fine coder. But, GOD DAMN you feel like a beast when you type C into a terminal, run compile, and it works with no bugs.
You get a special satisfaction of bravura and elan that would reduce Salieri to tears reading the scripts of Mozart and realizing there were zero hand-writing errors. https://www.youtube.com/watch?v=RfJ0QyiI68c (This is historically apocryphal, but makes for great screenwriting.)
Through this rigor, you learn something very valuable, albeit something you might learn in more conventional and less tedious ways: You learn always---at least in those fleeting moments---to write correct code at the optimal level of conciseness. Because you simply don't have the time to fiddle around the way you might if you have a fast wpm typing rate, since you write slower than you think. And learn to think faster than you can write.
I guess this is similar to the form of discipline you would get when you programmed punch cards and sent them via post, except under time pressure it was blitz chess than punch-card postal chess.
And don't get me wrong. I use Jupyter notebooks and putz around a lot these days. I'm too old for some shit, and putzing around is fun.
I believe it was 3 hours, and in my day it was 10 puzzles. I still have my crib sheets, and the printed-out problems sets, which were printed on brightly-coloured sheet of paper. I agree that, once you apply yourself to this, it is not that difficult to learn to code more proficiently on paper than you would with an IDE, in less time. That's not to say we didn't have bugs in our algorithm designs.
Although we did not get eighth place in our region, much less the world, I too think it was pivotal in my development as a programmer.
I also have typing exercises as a practice for students in my courses and part of my research. While I haven't formally analyzed how much time they take, I can say that students take roughly 7-10 minutes to complete 30-50 line typing practice.
Though with that said, I also rote memorized a lot of chemical structures, pathways and several rows of properties of the period table. It’s essentially the same as learning a language. You need a certain “vocabulary” before you can reach a higher order of reasoning.
But I would also be careful with suggesting “copy everything verbatim to learn it”, since it helps you to mostly memorize it, but not necessarily to reason about it. Memorizing the first 5000 words of a language is useful. Memorizing the dictionary is not.
Once you have a handle on it, though, it's like jet fuel for learning. You can synthesize things literally five times faster.
Only it wasn't.
He also did say - which you may have scanned past -
"I aim to have these emails be more along the lines of "infotainment." Meaning, they aim to contain brief bits of information while also being entertaining."
* Org Mode [0] is an Emacs plugin
* Anki [1] is a standalone app