Hacker News Reader

Comment by steve1977 | Hacker News Reader

steve1977

I don't know this specific exam, but most of these can be gamed in the sense of learning to the test. So depending on what training resources someone has available (e.g. rich parents who can afford tutors), I'd consider them only partially meritocratic.

View on HN

If this is the case, then why doesn’t everyone get the top a score? The answer is, of course, that it’s not so simple, and you can’t just learn to the test.

That’s just like with sports: anyone can learn how to train himself, and anyone can improve with training, but in the end, some people will end up faster, and some people will end up slower.

My point was exactly that the chances are NOT the same for everyone. A kid from an affluent family might have both better tutoring as well as fewer troubles in life that could deter from learning.

But of course, in addition to that, there is always also a genetic component, as in sports.

The question is what you're measuring. You can have a test that gives you whatever distribution of scores you like. But is the thing it measures competency in the subjects it tests, general intellectual ability, familiarity with the test format, etc.? The worst negative outcome is usually subordination of learning itself to preparing for the exam, which can happen even when the gatekeeping function of an exam still works perfectly.

All scientific research on this topic points to the conclusion that standardized test results are the single best predictor of subsequent academic performance. Some studies suggest that using GPA in addition to test results improves the prediction accuracy, but the marginal increase is very small, and it increases variance.

Everyone is well familiar with the downsides of standardized tests, but so far, nobody has proposed any alternative that better. Learning to the test is not great, but what’s the alternative? It’s not like anyone knows how to teach things that results in more actual knowledge and skills being attained despite lower test results.

> All scientific research on this topic points to the conclusion that standardized test results are the single best predictor of subsequent academic performance.

And academic performance is measured how? With standardized tests?

Obviously, yes. This is not circular: it is by no means tautological that people who did well on test X will do well on a completely different test Y that tests different knowledge and skills. The fact fact that it does gives strong evidence to the value of using these tests for admission.

It depends on what skills are necessary to succeed at test X and Y. While the subject matter or details thereof might differ, it's possible that things like "knowing how to learn to the test (i.e. cramming)" or "reading between the lines of what the teacher/professor says is relevant" belongs to these skills. And these can absolutely be transferred between test X and Y.

So the question is, how much do these tests actually test skills in the subject matter and how much do they test "meta skills"?

The research shows that the tests are predictive almost precisely to the extent they test “meta skills”. We can even measure how much individual questions test meta skills vs specific skills. You can learn about this by searching for the terms “factor analysis” and “g-loading”.

Then what’s a better, unbiased method of assessing students?

The main problem I raised with the Gaokao isn't that it's biased, but that it has negative effects on the way education is conducted prior to university.

It's not difficult to find first-hand accounts of this; go browse social media posts by teachers in mainland China if you're curious.

There are similar problems of "teaching to the test" in other contexts, too.

I'm not categorically opposed to standardized testing and I never said I was.

Ok so what’s your alternative proposal?

China also bans test tutoring as a commercial service. Without a doubt people will still be able to find tutors if they're sufficiently capable, but the scale of this problem should be vastly altered by that action.

Can you learn to the test without actually learning the material?

Without learning the material? Unlikely. Without understanding the material? More likely.

Feynman made such an observation while in Brazil:

https://enlightenedidiot.net/random/feynman-on-brazilian-edu...

Ok that’s a very good criticism.

My take away is the assessment method must force the student to actually apply understanding and application of the material, and not simply look y regurgitate it.

Thank you.

What you are describing, or the lack thereof, is how the majority of my public education in a Southern US state felt. Ironically, my county had the highest test scores in the state too. Then again, perhaps that isn't ironic after all...

Reply on news.ycombinator.com