Hacker News Reader

Comment by temp-dude-87844 | Hacker News Reader

Disaggregating the results by race is real cute but also misses the point: blacks and asians and hispanics are residents of the state too and so leaving them in to evaluate the state's overall achievement (relative to other states) in education is fair. If anything it ought to be more problematic that Texas fails so badly at educating hispanic students when a fifth of them live in Texas.

Besides, the race in big-gap states is an unwitting proxy to income, so disaggregate by household per capita income (relative to local cost of living) if you want to disdain. This methodology would work well in all states and expose the challenges of anxiety and the quality of a student's home life in their education attainment.

View on HN

"Disaggregation" doesn't mean taking them out--the ranking includes everyone:

> Think about that: White students do better in Texas than in Iowa. Black students do better in Texas. Hispanic students do better in Texas. Asian students do better in Texas. Given these facts, it is absurd for U.S. News to rank Iowa higher than Texas in terms of educational performance.

This, of course, makes sense. Students from disadvantaged backgrounds face unique challenges. Under the traditional methodology, states can do a crappy job of educating these students so long as they have few of them. States that have large populations of disadvantaged students, by contrast, are penalized in the rankings even if they do a relatively better job educating those students.

I'm surprised to find no mention of SES or income in the article. There can be similar Simpson's paradox effects within racial groups between states (for instance, Iowa might have more low SES white people than Texas). I would have expected this to be mentioned in the article for why they chose not to account for it in some form.

Simpson's Paradox is the relevant keyword here. Averaging a across non-homogeneous groups (multimodal distributions) makes garbage statisics

> If anything it ought to be more problematic that Texas fails so badly at educating hispanic students when a fifth of them live in Texas.

Texas doesn't do badly at educating Hispanic students -- it does a superior job at that. It has low overall performance because Hispanic students have terrible performance, and Texas, even though its Hispanic students are better than average, has a very large number of them.

This kind of thing is generally known as Simpson's Paradox. https://en.wikipedia.org/wiki/Simpson%27s_paradox

All good points, and clarifies my wording about Texas doing a good job (relative to other states) of educating Hispanic students that suggested otherwise. But the widely divergent results between the aggregated and disaggregated view isn't sufficient to declare, like the article does, that the ranking using aggregated view are using a flawed methodology. They're measuring different factors than the authors of the article, and the nuances of both are equally squashed under the pressure of producing a simple metric.

Cohorts are useful because they capture a group with some shared attribute, and racial cohorts are particularly useful because they're overwhelmingly consistent through time and datasource for a particular person. The disaggregation into racial cohorts in a ranking like this would be a useful indicator of how the outcomes of a particular person would change solely by moving states. But it's not an effective indicator of evaluating a state taxpayer's contribution to the quality of students coming out of the state's schools -- students who presumably perpetuate the results of the quality of their education in their life, producing a myriad of poorly-understood impacts on the state as they do so. Both of them are worthwhile questions to answer, but they're very different studies. But one calling another flawed on the basis of this is jarring.

Why isn't it an effective indicator for evaluating how much state taxpayers are getting for their contribution to the state schools? Can you elaborate on the difference between the two questions?

The "best states to send your kids to school" and the "how much value are students of our state deriving from my tax contribution vs. in other states" are almost mutually exclusive, because in the second case you're deliberately averaging every student out in the whole state, good students and bad, while in the first one you're benefiting from the positive lurking variables associated with other cohorts, yet only being penalized for negative contributions of your own cohort.

The first methodology benefits from disaggregation by racial cohorts such that the reader can optimize given their particular circumstances, but the magnitude of contributory factors to that success are opaque and still averaged statewide. Most obviously, this also includes funding, where funding received by particular students -- and especially not funding sourced from parents of students in the same cohort -- are not separated out. They shouldn't have to be, as schooling is a public good, but if they aren't, then the metric doesn't accurately reflect the impact of the socioeconomic condition and proportions of the cohorts in the state, but splits them out the results solely to avoid having to consider races other than one's own.

The latter question is typically what is being studied, although pop science coverage often leaves the reader free to interpret it as the former.

> But it's not an effective indicator of evaluating a state taxpayer's contribution to the quality of students coming out of the state's schools -- students who presumably perpetuate the results of the quality of their education in their life, producing a myriad of poorly-understood impacts on the state as they do so.

The opposite is true: the disaggregated ranking is the only good measure of "a state taxpayer's contribution" because the taxpayer has little to do with e.g. the fact that Hispanic students face additional challenges because they're from an ESL household.

What you're really saying is that the composite metric is a better measure of the average "quality of students coming out of the state's schools," which reduces to the assertion that it's better to have a school system full of privileged white kids than minority kids who face challenges. I don't think we should be ranking school systems that way.

I would suggest you read up on Simpson's paradox.[0] The study is not eliminating their performance just categorizing them.

[0]: https://en.wikipedia.org/wiki/Simpson%27s_paradox

It depends on what question you're trying to answer.

"How well does the average student do?" vs "If you raise a family there, how well will your children do?"

For the former question, you don't want to control for racial mixture. For the latter question, you do. Most parents should care more about the latter question than the former.

> race here is an an unwitting proxy to income, so disaggregate by household per capita income

Poor white students do better academically than well off black students. So splitting by race is more informative when discussing academic performance.

Do you have some evidence to back that up? There are some dumb rich people, no doubt of every color, however there is a clear correlation between income and performance in every reputable source I’ve ever seen.

I find it hard to believe that on average rich black kids are doing worse than on average poor white kids.

Here's a study[1] that breaks down SAT scores by race and income in 2003. Take a look at table 2.

[1] https://pdfs.semanticscholar.org/e7a0/dfdee8ad5b91e1d179e9e2...

I believe that study shows that the affect of race on SAT scores when a more complex model of income and parental education is taken into account is halved (i.e., the observed 100 pt difference drops to 50). So I think it still doesn't excuse the original post from taking income into account.

Yes, a multifaceted approach that takes into account lots of factors would be ideal. It’s still better than existing rankings though, which just use a statewide average.

Reply on news.ycombinator.com