Comment by jonnathanson | Hacker News Reader

Comment by jonnathanson | Hacker News Reader

Parent Full thread

Not to mention demographics. Most of these studies pretty much overlook that dimension entirely, and it's usually the most critical.

In the case of Finland, we're looking at a country with a total population of 5.4 million people, 92% of which share the same native language and ethnic background. By comparison, the USA has 313 million people and a heck of a lot more diversity of income, class, race, ethnicity, linguistic background, country of origin, etc. It's also got a much higher variance in population density per city, state, region, etc.

If ever there were a case of comparing apples to oranges, this would be it. There are vanishingly few analogs between the two countries on almost any dimension, and most of the education statistics I've seen over the years have not been weighted accordingly.

I'd be much more interested in apples-to-apples comparisons. How does Finland perform against countries like Finland? How does the US perform against similar countries (of which there probably aren't many)? Even comparing the US to a country like China is problematic, given that only the top some-single-digit percent of Chinese children take these tests in the first place.

The article specifically describes a comparison of Finland with Norway, similiar in size and ethnic composition. Apples-to-apples, as you say. Norway's education policy is much more similar to the US, and Norway's PISA results are mediocre, also similar to the US.

Actually, after controlling for ethnic composition, the US and Norway are not similar. In fact the US outperforms Norway by roughly the same margin that Finland outperforms the US.

http://super-economy.blogspot.com/2010/12/amazing-truth-abou...

Your "authority" is crap. The original article explicitly compared Finland not only with Norway but with individual US states that had similar population and percentage of immigrants. That's about as apples to apples as you can get, considering that US schools are primarily subject to state rather than federal guidelines, and Finland still came out ahead.

By way of supposed refutation, you present a source that is full of strong claims substantiated by little more than the most blatant cherry-picking. The author's "correcting for demography" is no more than throwing out the data he doesn't like. Ignoring the fact that the immigrant factor has already been examined, he pulls tricks like comparing second- and third-generation immigrants in Finland to native Swedes. Notice how the obvious comparison between native Finns and native Swedes isn't made, because it wouldn't support his conclusions. Then he goes on to say "In the case of America, 99% of the population originates from other countries" and suggests that we compare such loosely-defined American immigrants to people in their home countries - without regard for such things as economic differences between the countries. After arbitrarily excluding Asians because they pull the results up, Hispanics because they pull the results down, and undeclareds for no methodologically sound (or even fully explained) reason, he comes up with a highly suspect graph purportedly showing how the US "beats" most of Europe.

Here's the kicker: even after cooking the data that much, Finland still comes out well ahead of the US. No matter how hard he tries, he still can't manage to reverse the original conclusion. In the end, citing that does more to discredit your position than to support it.

Um, I wasn't disputing the fact that Finland comes out way ahead of everyone else. I would have expected the phrase "...roughly the same margin that Finland outperforms the US..." to be a tipoff.

I was disputing the idea that the US and Norway are somehow equivalent. They aren't. The US does far better than Norway.

The exclusion of Asians, Hispanics and Blacks is not arbitrary - it is done because Finland, the UK, Sweden and the other nations discussed by Sanandaji have very few of those groups. Since there are strong correlations between ethnicity and school performance, this is necessary - otherwise we might confuse the effect of the school system with effects of the student body. He has another blog post where he compares the US to Asian nations and excludes non-Asian ethnicities for the same reason - Singapore and Japan don't have many Whites/Blacks/Hispanics.

The goal of Sanandaji's blog post is to compare educational systems, holding student body constant.

But student body is not held constant, not even ethnically. Even within the "white European" demographic bucket there's huge variation among ethnic groups - most relevantly regarding attitudes toward education itself, teachers, and relative levels of funding or social priority given to either. Treating US whites the same as European whites is wrong on a whole bunch of levels. For one thing, we're not all white. For another, the American population is drawn from specific subsets of the European population, and has experienced different patterns of population change since then (particularly evident wrt Jews with their known unique educational profile). Lastly there's the fact that even if non-whites aren't included in the sample their effects are still felt because many white students still share schools with non-whites, and not a few of those "white" students in fact have mixed heritage.

Of course, treating ethnicity - and particularly just white vs. non-white - as the only variable other than the educational system itself is itself offensive. One might also consider the effects that 300+ years of distinctly US culture and history have had on attitudes toward education, teachers, or the relative social/economic priority accorded to either. Or the effects that wealth distribution within the US or wealth differences between the US and other nations might have. The US student body is simply not the same as the Finnish one, so if Sanandaji really wanted to isolate the effect of the educational system then he'd have to adjust for more than one other variable.

Alternatively, he or you might take note of the fact that the current Finnish results are the result of change to the system while holding demographics relatively constant. What works in Finland might not work in the US for all sorts of reasons, but claiming that it won't work specifically because of ethnic makeup is both intellectually and morally dubious.

If you have a data set that distinguishes between different flavors of European, or somehow controls for attitudes towards education, I'd love to see it.

In the meantime, I'll control for the factors I have data on, while recognizing that even more of the variation than I can see might be caused by non-school system factors.

...you might take note of the fact that the current Finnish results are the result of change to the system while holding demographics relatively constant.

This is incorrect. Because the Finns don't do standardized tests (according to the article), and because PISA is relatively recent, we don't know how good or bad the Finns did before these changes.

"I'll control for the factors I have data on"

Of course you will, even though - or perhaps because - the intermediate result is far more inflammatory than informative.

"Because the Finns don't do standardized tests"

The fact that they don't do standardized tests doesn't mean it's impossible to know whether they're doing better. They embarked on these reforms because of a perceived problem, and seem quite satisfied with the results. If educators and researchers believe there has been improvement, based on other evidence, then I'm disinclined to second-guess them based on one semi-informed and heavily biased blogger's commentary.

So your suggestion is to disregard all data without perfect controls? In that case, we have no evidence whatsoever that any nation outperforms any other nation.

The fact that they don't do standardized tests doesn't mean it's impossible to know whether they're doing better.

How can one possibly know this, even in principle?

If you are being intellectually honest, your answer should agree with your principle of ignoring all data unless perfect controls are used.

"How can one possibly know this, even in principle?"

Are you seriously suggesting that standardized tests are the only way to measure school or student performance? What about non-standardized tests showing improvement within the same region or district? What about measuring achievement differences after leaving school? There are plenty of other options besides standardized tests.

In addition to cherry-picking and strawman, you've just added the the excluded middle to your list of fallacies. There are more than two options here, not just standardized tests or mere guesswork. Just because something can't be measured by one yardstick, even if it's the most accurate one (which itself is debatable in this case), doesn't mean other yardsticks won't suffice. Do you really want to turn this into a discussion about intellectual honesty? I'd relish the opportunity to cast your disreputable claims and tactics even further into oblivion.

Hmm. So you trust the results of non-standardized non-normed tests, even though no such tests have actually been cited.

You are unwilling to trust the results of an internationally normed and standardized test (PISA) after controlling for some but not all exogenous variables?

But nevertheless, you are willing to trust the results of the internationally normed and standardized test without controlling for any exogenous variables?

Heh, you had me going for a while. Now I realize you are trolling.

The US and Norway are not similar. Norway and Finland are similar.

Finland is similar to many US states, the level of government responsible for education.

> demographics

Canada scores in the top 10 in all PISA categories, not far off from Finland, and shares much more demographically and geographically with the United States than with Finland.

The provinces of Canada with higher ethnic diversity and immigration (the more prosperous ones) tend to score higher within Canada than the ethnically homogeneous provinces.

The US and Canada have vastly different demographics. In the US black and hispanic children perform far worse than white and asian children.

Wikipedia says that black and hispanic Canadians comprise less than 4% of the population.

It's not obvious that this statistic is an argument against the success of the Finnish system being translatable to America, though. Black and Hispanic children are disproportionately likely to be poor and subject to underfunded schooling, and may in fact be the demographics who would stand to improve the most under a system more interested in equalizing such disadvantages than the current one.

> "The provinces of Canada with higher ethnic diversity and immigration"

What kind of immigration? The USA has the unfortunate status of being home to a huge population of generally impoverished immigrants, thanks to its over-focus on family reunification and humanitarian immigration paths, the ease/prevalence of illegal immigration, and combined with a complete ignorance of skilled immigration.

This is the opposite of Canada, where the immigration policy has for decades strongly favored skilled immigrants - and have let them in in far greater numbers (and greater ease) than humanitarian immigrants. Similarly, Canada has for the past decade or so slowly shut the door and raised the bar on family reunification. It should also be no surprise that illegal immigration is a substantially smaller problem here than it is in the USA.

The somewhat inconvenient and blunt way to put it is: Canada has, for the most part, received a socially desirable demographic of immigrants, and the USA has not.

On top of this, America has to deal with the legacy of slavery - which has created a huge population that continues to be marginalized (despite advancements) to this day. You can't oppress and systematically destroy a population's chances of success for nearly 200 years in a row and then magically expect them to pick right back up a mere 4-5 decades later. This race dynamic drives a huge part of American demographics, and in Canada this issue may as well not exist.

The issue of race in the US is a labyrinthine beast that the vast majority of Canada could not even begin to imagine. And for that Canadians are lucky.

It's misleading to discuss matters of ethnic demographics in such terms as "diversity" and "immigration." The fact of the matter is, while immigrant and/or minority status can be a variable in its own right, it's usually just a proxy for effects that are specific to the minority and/or immigrant group in question.

Reply on news.ycombinator.com