Hacker News Reader

Comment by spicyj | Hacker News Reader

At Khan Academy, we're really excited to work with the College Board to provide awesome, free test prep to everyone. We're putting huge emphasis on really learning the material instead of practicing test-taking skills that won't be useful afterwards.

Here's a little more about the partnership if you're interested:

http://techcrunch.com/2014/03/05/khan-academy-gets-major-par...

View on HN

"Instead of teaching to the test we're going to put huge emphasis on really learning the material instead of practicing test-taking skills that won't be useful afterwards."

Now how do we get the schools to actually do that?

You provide them with a premade test-prep curriculum that is interwoven with genuine educational videos, perhaps in a subtle way, such as providing coarse links to review material in response to incorrect answers. Then you hope that educators will either see the value in your cross-links or won't have the time to scrape your question bank and eliminate the cross-references ("condensing" the review material).

Disclaimer: I have no actual idea if this is what they're doing, I'm just saying that it could be done this way.

Changing the incentives would be a start. If you give schools very specific metrics, and then judge them on how well they optimize those metrics for the least money, it's no surprise that they are going to... try to find the most direct way of optimizing those metrics. In fact, the ones who don't do that will be penalized!

"Beware your metrics, for they will improve."

Ideally you test better, such that teaching to the test is identical to teaching well. That's the theory behind the Oxbridge tutorial system. Unfortunately good tests and scalable tests seem to be mutually incompatible, and non-scalable tests incommensurable.

People in discussions like this often assume that it's easy to make test scores go up without teaching the material.

Here are specific tests from some large states (the first I found via google):

http://www.nysedregents.org/Grade8/Mathematics/20100505book1...

http://www.nysedregents.org/Grade8/EnglishLanguageArts/04261...

http://www.cde.ca.gov/ta/tg/sr/documents/cstrtqmath7.pdf

How do you make scores on these specific tests go up to any significant degree without making students learn the material? Please explain the techniques in detail.

Well, all of the tests are multiple choice. Switching away from the format would certainly encourage a better understanding of material. Given that the format of the 4 choices is general:

1 correct answer, 1 or 2 close but not correct answer, and 1 or 2 obviously wrong answers.

Now that format makes thing significantly easier to get the correct answer than simply presenting the problem and asking the student to solve the problem. I'd imagine some of you are thinking "Well hell, they would still need to know the formula", but they also give you formula sheets with the tests. Since they give you a list of possible answers and a list of applicable formulas, you can pretty quickly plug in some numbers and determine the correct answer. Does it still require some base understanding? Yes. But I can tell you with absolute certainty that my grades would not have been as high had I not known how to game the testing system in such a manner.

I'm only a few years removed from high school, so I cannot imagine things have changed too much, especially for the better. For reference I am from South Carolina, albeit I went to one of the more performant/richer public school systems in the state.

Teaching students how to get into the ballpark of the right answer and reject obviously wrong ones increases their understanding. It's an important skill and there are adults out there who embarrass themselves by failing to do it (for example http://www.buzzfeed.com/charlesclymer/5-things-more-likely-t... ).

Same thing for taking a reference work (e.g. a formula sheet, a wikipedia page) and figuring out which piece of information you need and applying it.

Remember, I'm not asking if the tests could be better. I'm asking what you do to make test scores go up without making understanding go up.

I'd be at least moderately convinced if you could even show me two teaching techniques, A and B, where A is superior for test scores but significantly inferior for understanding.

I used to play rock-paper-scissors with my child. Could win every time. You could see on his face his simple logic.

You can learn to take tests - I did that for the GRE, got a 99% but hadn't ever studied 2 of the area tests I took. There's a logic to test questions, you can learn it.

There was no understanding involved, I can assure you. Just a recognition of what is considered a 'fair' question, which lets you eliminate answers that are too convoluted.

"I did it, it's totally possible, I'm not going to support this claim at all but I assure you it's true."

It would be far more convincing if you actually skimmed the NY Reagents test and presented a teaching strategy which you believe would raise scores but not increase understanding.

Note that if such a strategy does not exist for the NY test but does exist for every other test, then we have a solution to all the complaints about testing - use the NY test for everyone.

As a former SAT prep teacher, I'm very excited to learn about Khan Academy's partnership with the College Board. Previously, success on the SAT seemed to depend solely on how much money was spent on SAT prep. Glad to hear that the College Board is interested in leveling the playing field.

Eh, maybe I'm just an outlier but I got a 1370 in 1997 without really trying, let alone actually spending money to prepare. I'm also a 100% product of public schools.

Not sure what you are saying. The claim is that you could have done better had you really tried and taking courses is one way of doing that.

Some feel if they succeed without trying, then this success is more meaningful than those whose foundation for success was built on hard work.

Some also feel their time is better spent on learning and smart work and not dry, repetitive practice of test-taking strategies.

Maybe I misinterpreted "success on the SAT" and "solely". In context, I took it to mean people had test test prep did well, in the absolute sense, and those who did not have test prep did not do well. I was providing myself as an example of someone who did well in the absolute sense with no preparation. I was not saying I could not have done better with preparation.

Yeah I meant more than it is very easy to go from the 400-500's range to the 600-700's range with SAT prep. Especially on the writing section -- they only test on 20ish grammar rules, many of which are subtle and wouldn't necessarily be noticed without prep.

I can see that. I remember quite a bit of the test being formulaic, and preparation for what the formulas are would certainly be worth a few hundred points. I can't speak for the writing section, because it didn't exist in 1997.

I scored very well on the SAT's back in 2001 or so (better than 700 on each section) with no prep. I dont think anything on there was beyond the reach of a normal HS student. I went to public HS and took no advanced math or english classes prior to the test.

The writing section is the only one I think test prep would have helped with. Especially with essay structuring. I just wrote a response to a prompt the way I've always written.

For the writing section, you have to follow an exact structure and use the right amount of words. You can also make stuff up to support your arguments.

It's basically "Write a generic 5 paragraph essay using memorized argument flow."

Yea, with no test prep, I just wrote an essay as I always had. It was my weakest section.

Where are you getting this information? I've taught hundreds of kids SAT prep and I can tell you that it is definitely not "very easy" to go from 400-500s to 600-700s.

I did literally no prep whatsoever (just walked in to the test location the day I was told to), and did very well. I'm sure test prep would have helped my writing score (my lowest one) by teaching me how to write a god awful essay that contained the pieces to get you extra points, but my verbal and math scores were about as high as they were going to get.

The secret to standardized testing is fast reading comprehension. I've always been a fast reader (without sacrificing comprehension). It frees up time to go back and really spend time on the tougher questions that just isn't available if you have to spend a long time just reading the question (it's more than just in the reading comprehension section, since you have to actually read every question to get through the test).

SAT prep has little effect on SAT scores. From research on this topic [1]:

"Does test preparation help improve student performance on the SAT and ACT? For students that have taken the test before and would like to boost their scores, coaching seems to help, but by a rather small amount. After controlling for group differences, the average coaching boost on the math section of the SAT is 14 to 15 points. The boost is smaller on the verbal section of the test, just 6 to 8 points. The combined effect of coaching on the SAT for the NELS sample is about 20 points. The effect of coaching is similar on comparable sections of the ACT. The average score increase on the ACT math section probably lies within the range of 0 to .4 points, while the coaching effect on the English section is about .3 to .6 points"

[1] http://nepc.colorado.edu/files/Briggs_Theeffectofadmissionst...

Hm, it's very possible that SAT prep companies game the system by making their diagnostic tests quite difficult.

There is a lot of bad test prep out there, especially any test prep taught by high school teachers. I have done test prep privately for six years and I can guarantee you coaching can have substantial effects. These studies group all test prep under a giant umbrella, but take elite 1 to 1 test prep and the premium price it costs that parents actually pay and you'll see substantial gains. I have spend $0.00 on advertising in the last six years and I am routinely overbooked because of word of mouth. There's just so many awful teachers and courses out there.

> Previously, success on the SAT seemed to depend solely on how much money was spent on SAT prep.

Most likely, a correlation is not causation thing -- people for whom success on the SAT is most important are likely to spend more money on test prep, but also be best generally prepared. As ueqirat points out, the research that has been done on SAT test prep utility hasn't actually born out a real effect.

I know it's just one datapoint, and SAT II, not SAT, but I took the SAT II writing test around 2000 and got a 600. I then took a test prep course over a few weeks, took the test again, and got a 690, which is a pretty big difference I'd say. I can tell you the test prep course certainly didn't improve my writing at all...

I didn't spend a dime on SAT prep and I did reasonably well (good enough to get into Caltech).

Perhaps you didn't give money to Princeton Review or Kaplan, but I'm guessing you didn't walk home barefoot to abusive parents in Louisiana either.

Good thing there were no questions about extrapolating from a sample size of 1.

Since the parent said "solely" all it takes is a single counterexample :-)

and that's why you are posting on HN, with other members of above-average baseline intelligence.

This is going to be huge for you guys, congratulations and I hope it all works out!

Reply on news.ycombinator.com