Whoops, yeah, that's a typo. I think I originally had 7x3 as the example. Thanks for catching it.
1,329 karma · joined January 19, 2024
Chief Quant, Director of Analytics at MathAcademy.com. Instead of optimizing return in the stock market, I optimize learning efficiency in students' brains.
TLDR: I learned from those kinds of resources myself, and while I came a long way, for the amount of effort I put into learning, I could have gone a lot further if my time were used more efficiently. That's the problem that Math Academy solves.
1. New tasks are selected only as you complete existing tasks (so if you come back after 2 weeks, you need to complete some existing tasks from 2 weeks ago to get new tasks selected based on your knowledge profile right now).
2. We are often able to implicitly knock out due reviews with new lessons. We're not just doing plain vanilla spaced repetition. We're doing a highly efficient novel version of it that we call Fractional Implicit Repetition (FIRe). I have a writeup on this that gained some traction on HN recently: https://news.ycombinator.com/item?id=40954571
Regarding spaced repetition, keep in mind the following:
1. New tasks are selected only as you complete existing tasks (so if you come back after 2 weeks, you need to complete some existing tasks from 2 weeks ago to get new tasks selected based on your knowledge profile right now).
2. We are often able to implicitly knock out due reviews with new lessons. We're not just doing plain vanilla spaced repetition. We're doing a highly efficient novel version of it that we call Fractional Implicit Repetition (FIRe). I have a writeup on this that gained some traction on HN recently: https://news.ycombinator.com/item?id=40954571
Happy to answer any questions you may have.
Campbell (1987) The role of associative interference in learning and retrieving arithmetic facts
https://www.researchgate.net/publication/247981688_The_role_...
Yeah, that makes sense. It sounds like you're talking about "listening on paper" (I made a distinction between "note-taking" and "listening on paper" at the end of the post).
You don't have to be blazing fast but you do need to have sufficiently tight feedback loops so that you can get through a high volume of action-feedback-adjustment cycles. It's the compounding of a massive number of those cycles that produces a massive gain in performance.
Also, in order to make mathematical discoveries -- or more generally, in order to grapple with advanced content at all -- you need to develop a level of automaticity on your foundational skills.
1) Maybe you can do it with scaffolding, but you are unable to do it without scaffolding.
For instance, a musician might not be capable of playing a difficult section of a musical piece at full speed. So they might practice it while playing slowly (a type of scaffolding), and then gradually ramp up the speed while maintaining accuracy.
2) Maybe you can do it sometimes, but not consistently/accurately.
For instance, a gymnast might not be capable of landing a particular flip consistently with proper form. But maybe they can land it 50% of the time with shaky form. So they might practice improving their consistency and form on this skill.
---
Working on things outside of one's repertoire, is a core aspect of deliberate practice. Non-experts are often misled to practice within their level of comfort. This tends to be more effortful and less enjoyable, which can mislead non-experts to practice within their level of comfort.
For instance, Coughlan et al. (2014) observed this as a factor differentiating intermediate and expert Gaeilic football players:
"Expert and intermediate level Gaelic football players executed two types of kicks during an acquisition phase and pre-, post-, and retention tests. During acquisition, participants self-selected how they practiced and rated the characteristics of deliberate practice for effort and enjoyment.
The expert group predominantly practiced the skill they were weaker at and improved its performance across pre-, post- and retention tests. Participants in the expert group also rated their practice as more effortful and less enjoyable compared to those in the intermediate group.
In contrast, participants in the intermediate group predominantly practiced the skill they were stronger at and improved their performance from pretest to posttest but not on the retention test."
---
The idea of practicing outside of one's repertoire can be generalized to the idea of engaging in a cycle of strain and adaptation. This is done in, e.g., Ericsson (2006). Here's a snippet:
"When the human body is put under exceptional strain, a range of dormant genes in the DNA are expressed and extraordinary physiological processes are activated. Over time the cells of the body, including the brain (see Hill & Schneider, Chapter 37) will reorganize in response to the induced metabolic demands of the activity by, for example, increases in the number of capillaries supplying blood to muscles and changes in metabolism of the muscle fibers themselves.
These adaptations will eventually allow the individual to execute the given level of activity without greatly straining the physiological systems. To gain further beneficial increases in adaptation, the athletes need to increase or change their weekly training activities to induce new and perhaps different types of strain on the key physiological systems."
---
In general, in the phrase "deliberate practice," the word "deliberate" is not just an adjective. "Deliberate practice" has a very specific meaning in the research literature.
The way you describe it -- "Practice is certainly still deliberate. It's planned. Every workout has a purpose." -- is not as strict as the meaning in the research literature.
For something closer to a proper definition, I'll quote (Ericsson, 2006):
"The core assumption of deliberate practice (Ericsson, 1996, 2002, 2004; Ericsson et al., 1993) is that expert performance is acquired gradually and that effective improvement of performance requires the opportunity to find suitable training tasks that the performer can master sequentially -- typically the design of training tasks and monitoring of the attained performance is done by a teacher or a coach.
Deliberate practice presents performers with tasks that are initially outside their current realm of reliable performance, yet can be mastered within hours of practice by concentrating on critical aspects and by gradually refining performance through repetitions after feedback.
Hence, the requirement for concentration sets deliberate practice apart from both mindless, routine performance and playful engagement, as the latter two types of activities would, if anything, merely strengthen the current mediating cognitive mechanisms rather than modify them to allow increases in the level of performance."
---
I don't know much about serious running, but based on how things work in other domains: if "performance-improving adjustments on every single repetition" is incomprehensible at the level you're looking, then it's an indication you need to zoom out a bit.
The same confusion can happen in, e.g., deliberate practice in math, if you zoom in too much. When a student solves a math problem, do we really expect every single pen stroke to involve feedback and improvement? No. You have to zoom out to the level of the problem.
Correct me if I'm wrong, but I would expect that in running, the appropriate level to view these deliberate practice cycles is not the level of a single step, but rather, a cohesive group of a taxing "deliberate practice" runs and easier "recovery" runs. At this level, it looks more like that cycle of strain/adaptation that is characteristic of deliberate practice.
(And that level seems to align with what's discussed in the literature -- for instance, I was just skimming Casado et al, 2020, Deliberate Practice in Training Differentiates the Best Kenyan and Spanish Long-Distance Runners, which mentioned that "systematic training ... included high-intensity training sessions considered deliberate practice (DP) and easy runs.")
---
References
Casado, A., Hanley, B., and Ruiz-Pérez, L.M. (2020). Deliberate Practice in Training Differentiates the Best Kenyan and Spanish Long-Distance Runners. European Journal of Sport Science, 20 (7). pp. 887-895.
Coughlan, E. K., Williams, A. M., McRobert, A. P., & Ford, P. R. (2014). How experts practice: A novel test of deliberate practice theory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(2), 449.
Ericsson, K. A. (2006). The influence of experience and deliberate practice on the development of superior expert performance. The Cambridge handbook of expertise and expert performance, 38(685-705), 2-2.
https://onlinelibrary.wiley.com/doi/full/10.1111/j.1553-2712...
In the article, the key underlying assumption is that the further you go in math, the more energy it requires to learn the next level up -- and everyone's "energy vs level of abstraction" curve is shifted based on their cognitive ability and degree of motivation/interest.
Here is a quote from the article that gets at the main argument:
"As Hofstadter describes, the abstraction ceiling is not a “hard” threshold, a level at which one is suddenly incapable of learning math, but rather a “soft” threshold, a level at which the amount of time and effort required to learn math begins to skyrocket until learning more advanced math is effectively no longer a productive use of one’s time. That level is different for everyone. For Hofstadter, it was graduate-level math; for another person, it might be earlier or later (but almost certainly earlier)."
https://www.justinmath.com/your-mathematical-potential-has-a...
Thanks for the kind words about Math Academy! It's true that we focus on students who are trying to acquire math skills to the highest degree possible -- we teach math as if we were training a professional athlete or musician. We maximize learning efficiency in the sense that we minimize the amount of work required to learn math to the fullest extent.
I realize that there are many learners who only want to devote an hour or two per month, but, at least right now, such learners would be better served elsewhere. It's a totally different optimization problem -- maximize surface-level coverage subject to some fixed, miniscule amount of work -- and as a result it would require different different curriculum and possibly different training techniques (or at least, differently calibrated techniques).
But it's definitely an idea to think about in the future. :)
However, the learning itself has to occur bottom-up. Especially in math. Math is a skill hierarchy, and if you cannot execute a lower-level skill consistently and accurately, you will not be able to build more advanced skills on top of it.
Not everybody can learn every level of math, but most people can learn the basics. In practice, however, few people actually reach their full mathematical potential because they get knocked off course early on by factors such as missing foundations, ineffective practice habits, inability or unwillingness to engage in additional practice, or lack of motivation.
(My comment here is basically the intro to a detailed article I wrote on the topic: https://www.justinmath.com/your-mathematical-potential-has-a...)
For instance, a student struggling in calculus may think they know algebra because they got a decent grade in algebra class, even though they struggle to solve a quadratic equation and they've forgotten how trig works.
-- Maybe they got saved by grade inflation, or
-- maybe they did learn these things but they've gotten so rusty that they need to effectively re-learn them again, or
-- maybe they learned and still remember everything from their algebra class, but the class was watered-down and cherry-picked the simplest possible cases of problems within each topic (e.g., quadratic equation always has leading coefficient of 1 and is solvable via factoring) ...
There's a million different ways that a student can look at a list of prerequisites and mistakenly think that they have learned them, especially if the prerequisites are listed as a handful of high-level categories as opposed to hundreds of granular atomic topics.
However, the learning itself has to occur bottom-up. Especially in math. Math is a skill hierarchy, and if you cannot execute a lower-level skill consistently and accurately, you will not be able to build more advanced skills on top of it.
I wrote about this recently here if you're interested: https://www.justinmath.com/how-to-learn-machine-learning-top...
I'll quote a snippet below:
“My biggest mistake when starting my doctoral research was taking a top-down approach. I focused my efforts on a handful of research papers on the frontier of my chosen field, even writing code to solve problems in these papers from day one. However, I soon realized I lacked many foundational prerequisites, making the first year exceptionally tough. What I should have done was spend 3-6 months dissecting the hell out of all the key research papers and books written on the subject, starting from the very basics (from my knowledge frontier) and working my way up (the bottom-up approach).”