Want to Remember Everything You'll Ever Learn? Surrender to This Algorithm
wired.com
wired.com
I also started every class with a question/answer period. The rules were simple, the questioning will last at least 10 minutes, and you don't want me to ask the questions. :-) Anything that had come up in the questions that seemed to be a point of confusion was sure to be added to the next homework set.
I won't go into what else I did with that class, but the end result is worth thinking about. First note that I gave a ridiculously hard final. Other grad students who saw it thought that the class would bomb. Secondly they aced the test. What do I mean by aced? Well I had a bonus question which fellow grad students thought nobody would get. 70% of the class got that question, and a good fraction were over 100% on the test. So they must have studied hard, right? Nope. I ran into some students several months later. They told me that they tried to study for the final and stopped after a few minutes because it was useless, they knew everything. And several months later they still knew much of the material cold!
The thing is that none of what I did was very radical. The principles have been known for a century. Psychologists have been trying to get people to listen for that long. I learned about it in the 80s from a university course I watched on TV. (British Columbia had a TV channel devoted to lectures for correspondence courses.)
Yet, despite how dramatic the effects are, nobody listens and nobody takes advantage of it.
However, middle school art teachers seem stuck in their little art project. I am not sure about high school, but it doesn't look like they teach the skills.
A great book. One of the exercises is to copy a picture turned 180 degrees, so that you're drawing exactly what you're seeing without the interference of your mind's prior knowledge of the subject.
I have a copy that I pass around via samizdat, because the mandatory drawing classes here are useless to students without the basic principles.
(Also worth noting: 'The Memory Book' by Lucas and Loraynne. Now back in print, an falling-apart copy got me through high school. I can still memorize a deck of cards in under ten minutes.)
I guess my teacher had an advantage with a psychology degree though.
I'm already updating webassign to follow your suggestion, current material + old stuff each week. Anything else you did (beyond SRS) would also be useful to know.
Why can't you be similarly creative with your class and run with it?
As for why not "be creative"? I don't see how that suggestion is helpful. Btilly wasn't creative, he implemented proven techniques. I want to do the same, I just don't know what those proven techniques are (beyond the ones he mentioned in his post).
That's what I hate about people. What makes people pay attention isn't the ideas themselves, but whether it looks like you worked hard in writing it. I've made comments on HN that no one paid any attention to, and then turned them into blog posts and had them get to the top of Reddit or Digg or whatever. Exact same ideas, all I did was visually change the text to make it look more professionally laid out, and carefully rework some of the phrasing to make it sound better to the ear. (Plus add a couple really good first sentences and a good headline.)
I've found that if you actually leave a couple minor mistakes in your blog post it gets more votes, because it look like you're thinking at the edge of your intellectual ability, and people like that. It's the exact same thing Seth said in his post about the Chris Bliss Diss video:
http://sethgodin.typepad.com/seths_blog/2006/03/its_how_you_...
"Today, I got a video, featuring Jason, who just might be the best juggler I have ever seen. Same music, similar routine. Except... five balls. Not three, five. Infinitely more difficult. And Jason makes it look easy.
The thing is, even though I know how much more difficult Jason's routine is and how skilled he is, the very ease of his delivery makes it less likely an audience would give him that same ovation. Interesting how important effort seems to be."
Those changes are non-trivial. Layout and writing style are integral components in the user interface for comprehending text.
There are a lot of hn news posts I've made that I think I could make into blog posts or even articles. But I can't claim they already are in a form people will pay attention to.
For example, the posts I'm most proud of generally get few mods whereas simplistic but irrefutable objections to some other story tend to get the highest mods. But that's logical. People just aren't reading the ten or fifty posts on a paper to find the gems. They might read the first that way if you're lucky. Otherwise, they are more likely to skim. Even on the web, the writer has to present their ideas so they are accessible.
(And here I go again. This post will get far less mods than statements like "where's your evidence")
Context has always been of the utmost importance for any sort of information to have impact.
Further, it's easier to point someone to a blog post than a forum comment.
In the Chris Bliss video, he's on a stage with all kinds of lights and scenery, performing an original routine in front of a crowd. In the Jason Garfield video, he's copying someone else's routine in a high school gym with no crowd. Sure, Jason's performance may be harder, but Chris Bliss's performance is more interesting. Its not that Jason made it look too easy, its that he made it boring.
Instead the takeaway could be this:
You users don't care if what you did was hard, they care about whether they like the product. And that's as much presentation (i.e. UI) as nuts and bolts.
You're bitching that people consume ideas in different media, in different forms? I'm confused by this.
There are a lot of things you need to learn to master a mathematical topic. You need to learn definitions and terminology. You need to learn key theorems. You should be able to apply procedures and derive formulas. At a higher level you need to learn available proof techniques and the key ideas behind important proofs.
Of course learning these things intellectually is not quite sufficient. You need skills as well. However failing to learn these things will keep you from learning the topic. And that is where most students fall down.
This is particularly true in subjects that build on themselves like mathematics. If you missed an important point earlier, you may develop a bigger misconception about a current topic that will make it impossible to learn a future one. Anyone who has tutored students should be aware of this.
As for what to review, students did not get the same questions over and over again. Instead I gave them questions from the same section over and over again. Since I only asked a few questions from the current day's section, there were always plenty of questions left to go back to for future homework sets. As a bonus doing many kinds of questions at the same time makes connections obvious. When a single homework set asks you to solve a set of equations, find a basis for a vector space, and invert a matrix, you see how they tie together.
I'm slow, I think, and I can not do well in math. So I just let math get off my life, and trying to be a computing hacker as well as an entrepreneur.
I've proposed for Mnemosyne 2.0 that there be a card type which is just some Python code which is evaluated: http://groups.google.com/group/mnemosyne-proj-users/browse_t... The idea is that if you want to learn, say, multiplication, you would write some Python code that generates 2 random ints (as the question) and their product (as the answer), and now the user must solve it. You get much the same benefit as if you generated several hundred problems by hand or by script, but without cluttering your deck or risking memorizing some.
Thanks for sharing.
I'm sure that if I had stayed in academia I would have sorted out those kinds of issues. However it is clear to me that I'm much happier not being an academic.
I think we can all recall a teacher or teachers in our life who truly made a difference in teaching us something - sometimes _despite_ our interest (or lack thereof) of learning it.
I was forced in University to take a course in Propositional Calculus (it was a breadth requirement for computing science) - it was mid-day so I attended the class (unlike early morning classes in which my attendance was abysmal) - I went in with zero desire - no coding, or even math. But, the timing was right so I just sat in the class.
The instructor spent thirteen weeks walking us through Reductio Ad absurdum proof, tautologies, conditional proofs, etc.. He gave us homework assignments that built on earlier knowledge, and just as we were getting into new material, he'd throw some earlier sequent proofs at us to solve and refresh our memories.
Here is the thing - I had _no_ interest in this course, the only thing going for it was it's timing, and repetition. I was _killing_ myself trying to learn integration by parts, linear algebra, discrete math, spending 5 hours outside of class for every hour in class trying to master those other course, and basically just sitting in this one distraction course. I was a C+ student in the math courses (after _massive_ effort) - I walked out of the Propositional Calculus course with an A+.
Sixteen years later I look back at University and can honestly say that the most useful course, that I _continue_ to have rock solid knowledge of PLUS use almost every single hour of the day was that stupid Philosphy 210 Propositional Calculus course. For the life of me I can't remember much of graph theory, and wouldn't be able to invert a matrix if you held a gun to my head, but, I'll sit in a meeting and see a ton of possible alternative and immediately start applying Disjunction elimination to break us out of a log jam.
There really is something to this spacing method.
I think in general, the knowledge domain is important. Linear algebra is a very unified field by itself but when you get courses that involve a grab bag of algebra, linear algebra, trigonometry and calculus, I think the "stew" they present is incredibly hard to comprehend on a high level and students wind-up either failing or learning by a horrible kind of rote.
Uh, this is from biased personal experience. I indeed only taught one college math course when I doing my math MA, it was "Trignometry". It was a complete failure. I still blame the "grab bag" quality of the material but perhaps I was to blame instead.
Q: "What is "the flow experience" and what does it have to do with motivation? A: The flow experience is when a person is completely involved in what he or she is doing, when the concentration is very high, when the person knows moment by moment what the next steps should be, like if you are playing tennis, you know where you want the ball to go, if you are playing a musical instrument you know what notes you want to play, every millisecond, almost. And you get feedback to what you're doing. That is, if you're playing music, you can hear whether what you are trying to do is coming out right or in tennis you see where the ball goes and so on. So there's concentration, clear goals, feedback, there is the feeling that what you can do is more or less in balance with what needs to be done, that is, challenges and skills are pretty much in balance."
http://www.edutopia.org/mihaly-csikszentmihalyi-motivating-p...
Hey cool, thanks for the link. It's been a long time since I was in school but back then every homework assignment was 100% what we covered that day. So if the "lessons" became unclear many folks just got further and further behind and for many there was no recovering at some point because the teachers never wanted to deviate from their schedule. Btillys 1/3 1/3 1/3 structure really resonates with me because I can imagine I would feel confident and never get in over my head as the complexity of the work escalates. (And if I fell behind I could use his 10 minute improvisational time to get clear) But back to the confidence, Video Games Designers want the player to be challenged but also to feel confident and of course this balance has to remain throughout all levels of the game. Video Game Designers have long used MC's work to help them incorporate this balance. So, yes I do believe the relationship between ability and challenge is part of MC's agenda- there is more, of course. It is my understanding that Video Games are the most common contact most people usually have with MC's work.
I made a connection between the successful feeling of playing video games with the successful learning experiences in Btillys class's and it would be interesting if Btilly is aware of MC's work or if he had ever thought of what he does as MCesque. In closing, I think you and I are saying the same thing here. Thanks for the comment..
But despite being very useful for a half to two-thirds of her students, there is always about 1/3 of the class who does terrible in this course. How did you get these people to keep up?
Anyway, I'm glad you posted this. It's a good pick-me-up motivator as I'm about to be trying to do the same thing!
http://www.techcrunch.com/2009/09/10/cerego-raises-3-4-milli...
And I bet you must be a good self-teacher. I wish I have ever had you as my teacher!
The advantage of holding "fancy degrees" is usually that you have been exposed to a wide variety of viewpoints and insignt that self-taught people might have. People tend to study what they are interested in. Unfortunately, this can produce self-taught academics with horse blinders on.
Imaging you only got your news from Reddit & Huffington Post, saying you were becoming a self-taught news commentator. You'd probably end up incredibly informed... about liberal viewpoints, perhaps not realizing that a large portion of the country doesn't share them. Getting taught in a formal education system (providing the system works) would expose you to a wider variety of views.
http://www.wired.com/medtech/health/magazine/16-05/ff_woznia...
Bottom line is that to maximize retention, there's an optimal time for being re-exposed to information: right before you're about to forget it. Seems like re-covering the previous material in the class may have accomplished this.
... But if we did that, who would drink Coors and drive forklift for Walmart and not complain about it?
So I think that a huge function of the educational system is to educate a large part of the population badly. I think that the teacher training system serves this is as well, by selecting for mediocre teachers and then making them more mediocre via training. Additionally, in the schools themselves a lot of effort is made to classify young people into dumb and smart categories, usually unfairly (http://en.wikipedia.org/wiki/Pygmalion_effect), and this classification stamps them for the rest of their lives and creates a population of hopeless lower class workers.
If there are SOME good teachers in the system like btilly, then SOME kids go on to get out of their class, which is perfect in that (1) we need to claim that it is possible to pull yourself up by your bootstraps, and (2) we need to recruit SOME kids to grow up to fill management positions, but not too many that they all can't find careers. Historically, underemployed educated people are the people who become union leaders.
The beauty of it is that the people who fail think it is their fault!
(Sorry to rant about the educational / class system in general, rather than the topic of how to teach advanced math, but I couldn't stop myself.)
Call me paranoid, but there you go.
"But if we did that, who would drink Coors and drive forklift for Walmart and not complain about it?"
Robots? If the education system elevated more of the population, there would be a shortage of blue-collar workers, more pressure for system automation, and more well-educated minds to develop the robots. Of course, there would be no shortage of immigrants to fill the blue-collar positions, but perhaps in the long term, a "better" equilibrium would be reached.
The dirty little secret of education is that it's "ok" to cram and forget. Most classes reward that; education is about "being able to do something once" vs. actually remembering and applying it.
I hope you write more about your findings!
I've tried to do something similar with my work so far - a working codebase that I'm just learning, a polishing codebase that I basically know my way around in, and various tweaks and bugfixes that I have to do for previous projects. It's a bit harder in the corporate world though - while Google engineers are given a lot of freedom to pick their projects, they're still subject to the needs of the business, and sometimes a project will come up that's such a great opportunity for professional visibility that I'd want to take it even if it involves working hard on something I already know well and dropping the polishing of stuff I just learned.
I suspect the OP was teaching an intro-level course, designed to get freshmen up to speed on basic linear algebra, filling any gaps they might have had in their high school curriculum.
Anki: http://ichi2.net/anki/
Wikipedia entry about this technique: http://en.wikipedia.org/wiki/Spaced_repetition
http://www.mnemosyne-proj.org/
it works on Win, Mac and Linux - written in Python
FlashcardDB allows a choice between two spaced repetition algorithms: Supermemo's SM-2 (like Anki, Mnemosyne) or the Leitner System. New card decks are shared for free.
If you're looking to memorise a lot of data, it's definitely worth a look (runs on Mac, Windows, and Linux).
There was a recent thread here about whether SRS will evolve to appeal to non-programmers: http://globalmaverick.org/archives/169-repetitio-est-mater-s...
http://www.supermemo.com/english/contents.htm#Articles
Really interesting stuff. The website and the software user interface (at least judging from the screenshots) are terrible, though.
All in all I find spending some time on studying docs useful in contrast to the pure work-by-doing approach.
tl;dr: if you will spend more than 5 minutes looking it up or trying to remember it over the next 10/20 years, then it's a net gain to memorize it in a SRS.
I find this plausible, but I'm not sure 5 minutes is right - I understand from Anki users (Anki's statistics include 'how much total time ever have I spent looking at/on this particular card') that a card will cost you more than 5 minutes, and more like 10 minutes.
http://www.spicyelephant.com/decks/523 http://www.spicyelephant.com/decks/577 http://www.spicyelephant.com/decks/659
You probably have to register for an account if you want the site to apply supermemo-type techniques. I don't recommend using the site in the long term because it will eventually force you to register for $20.