Hacker News Reader

There was an earlier Hacker News submission about the Gates Foundation research on teacher effectiveness,

http://news.ycombinator.com/item?id=4559682

linking to an article that reported details of the methodology.

http://www.theatlantic.com/magazine/archive/2012/10/why-kids...

I looked up other research on the matter for the reply I posted in that thread. From the article submitted then, this is one way this process has been validated:

"The responses did indeed help predict which classes would have the most test-score improvement at the end of the year. In math, for example, the teachers rated most highly by students delivered the equivalent of about six more months of learning than teachers with the lowest ratings. (By comparison, teachers who get a master’s degree—one of the few ways to earn a pay raise in most schools —delivered about one more month of learning per year than teachers without one.)

. . . .

"The survey did not ask Do you like your teacher? Is your teacher nice? This wasn’t a popularity contest. The survey mostly asked questions about what students saw, day in and day out.

"Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward:

1. Students in this class treat the teacher with respect.

2. My classmates behave the way my teacher wants them to.

3. Our class stays busy and doesn’t waste time.

4. In this class, we learn a lot almost every day.

5. In this class, we learn to correct our mistakes."

Here is earlier reporting (10 December 2010) from the New York Times about the same issue:

http://www.nytimes.com/2010/12/11/education/11education.html

Here is the website of Ronald Ferguson's research project at Harvard:

http://tripodproject.wpengine.com/about/our-team/

And here are some links about the project from the National Center for Teacher Effectiveness:

http://www.gse.harvard.edu/ncte/news/NCTE_Conference_Using_S...

Simply put, don't assume that what the Gates Foundation was investigating was the same kind of student opinion survey that I have filled out as a postsecondary student. (But note that I'm not so sure that those surveys are as bad or as useless as college faculty often claim they are.) There is a research base for the primary school pupil and secondary school student ratings used in the Gates Foundation studies, and I have every reason to believe those ratings would help school effectiveness--so much so that I use the same questions to invite my clients of my mathematics program to evaluate my teaching from that point of view.

Other comments in this thread are about the more general issue of college rankings as they currently exist. As a parent who has occasion to look at my children's college search process for four children, I really like the site College Results

http://www.collegeresults.org/search1b.aspx?institutionid=11...

which aggregates data that colleges are required by law to report to the federal government into user-friendly data look-ups that allow direct comparisons of similar colleges along many dimensions. For me as a parent, one of the most interesting data views is a view of "comparable colleges" for a college of interest, sorted under the Finance and Faculty tab for a ranking of colleges by instructional expenditures / FTE (full-time equivalent students). That comparison often reveals that even the "scholarships" (discounts from list price) that colleges offer still leave parents spending far more for their children's higher education than the college itself actually spends on educating students. That's a raw deal that more parents ought to know about. Colleges hire expensive consultants to learn how to confuse parents on the issue of value,

http://www.maguireassoc.com/services-challenges/optimize-net...

and parents have to defend themselves by looking up comparable data.

I can't find anything in those links about their survey being tested beyond 12th grade -- have they done that yet? It would be interesting to see if the same survey is effective in a college setting, especially one where classes can exceed 300 students.

And this is my hypothesis on why home schooling otherwise normal children is so much more effective than primary school education.

1) Kids between the ages of 5 and 13 often do treat their parents with respect.

2) Kids between the ages of 5 and 13 often do what their parents tell them to do.

3) Home school kids stay busy and don't waste time because the parent(s) aren't going to let the time go to 'waste'

4) Home schooled kids stay on subjects until they understand them and move on as soon as they do, this means little down time or 'review' for other students slowing them down.

5) Home schooled kids go through an correct all the mistakes and talk about how they made them in the first place and work on ways to avoid them in the future.

These all relate to the relationship the child has with their parent/teacher, the teacher is really invested in the child's success, and the precise pacing of subject introduction which is tailored to the student's ability to take in new concepts. The more you generalize more students per teacher, more teachers per student, the harder it is to keep these things optimized.

Assuming an involved parent (with the accumulated educational knowledge/value associated with a proper education of their own) rather than purely an "unschooling" approach (which is increasingly more popular, if risky) - this is likely true.

You are effectively providing the student (your child) with a full time, one-on-one tutor (you) which is significantly more effective (even with a less experienced teacher) than an instructor being asked to teach 20/30/40 children at one time.

A teacher in a full classroom of 30 students simply cannot pay attention to a child's advancement as precisely as a teacher working one-on-one.

Primary school seems to be about social skills just as much as academic skills. How does homeschooling address that?

In my experience (and granted the Bay Area has a fairly healthy home schooling community) is that there are plenty of opportunities for socializing which are not necessarily 'school'. One of the things we found for our daughters was Reikes Nature studies [1] (can't say enough good things about that program), activities at our church (that happen for all kids, home schooled or not) and get togethers with other kids who were part of the community.

Generally we found filling in for the 'good' part of school socializing wasn't a problem and missing out on the 'bad' part of school socializing was a benefit.

[1] http://riekes.org/nature-awareness/

Everyone says that, but I've seen no data to back it up. In fact, most studies show the exact opposite, that home schooled children have better social skills. People just assume that since they went to school, everyone else should too. Is it really in a child's best interest to spend the majority of his time with hundreds of other poorly supervised children, learning to emulate their behavior rather than the behavior of trusted adults?

As soon as you get out of school you realize that adult life, for most people, is absolutely nothing like school. It's shocking just how much nicer adults are than children. The only social environment school really prepares you for is an institutionalized one (e.g., prison).

And don't even get me started on the distractions, by the time I was 13 or so I spent so much time worrying about girls (and other social stuff, but mostly girls) that learning was the last thing on my mind (I still made a 4.0 in high school, but I didn't actually learn much). I also don't think that those juvenile relationships prepared me in any way for actual adult relationships.

Which studies? I'm a little skeptical that home schooled children have better social skills than kids that attend public school.

Its a common question that comes up, and as far as my wife and I could figure out it has never actually been an issue here. One of the things people don't recognize right away is that rarely are you the only person home schooling, there can be lots of people in your area doing the same. One of the programs we did was for science a bunch of us did a one science topic for the week, and everyone in the group's kids would go to that parent's house for some particular expertise or investigation. These were 5, 7 even 10 student groups of similar ages working on the same material. Similarly for Reikes which was 15 - 25 home schooled kids once a week meeting up at a county park to discuss the ecology, bio-diversity, flora, fauna, land management, lots of stuff.

If the impression is home schooling is 1 kid sitting at home all day doing the same things they would do in a class room, you are not seeing what is going on around here. Groups of kids tackling problems and learning about history, math, communication, societies and communities, and all the material you'd normally get in school, just in chunkier bits with the opportunity to go deeper into the topic if you're interested and just pick up the required bits if you're not. Lots of reading, lots of field trips (the Sierras are fabulous for doing geology field trips), museums and such. Oh and lots and lots of reading.

I'd love to see some more rigorous work on this.

Nonsense!

Homeschool kids are from a selected group where their parents are extremely dedicated

Look at public school kids who also come from the group of extremely dedicated parents

This. And consider that homeschooling in no way scales, and is not very effective for kids from troubled families (for obvious reasons).

One can never guess what is going to be projected into one's writing.

Tokenadult pointed out some things that MET figured out were highly correlated with effective teaching, he quoted them with : "Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward:

My wife and I home schooled our kids through "8th" grade, they went to the local high school. I got to meet a number of kids being home schooled, and their parents. I recognized all five of the factors that the MET identified as being present in a home schooling situation. That is all.

What did you think I was saying?

You are confusing cause and effect, the factors are already present in families who are dedicated/resourceful enough to consider homeschooling, so homeschooling simply has no effect in that case.

It is already known that the most important factor in whether a kid learns or not is their home situation. These studies are on the effectiveness of schools assuming the home factor can't be changed easily by society.

While you are right that the background of homeschooled children may be a confounding factor, you are overreaching when you (seem to) claim that this rules out the possibility that being homeschooled contributes as well.

Perhaps. Separating out the home factor of homeschooling to get some useful truth seems quite impossible.

Can you say more about this? Or can you talk about your reasoning that makes 'home factor' the dominant factor?

The biggest impact on a kid's future is their socioeconomic status and/or the determination of their parents. Schooling provided by society is not even a close second, though it does make being a good parent easier (you can delegate 6 or 7 hours a day to society, whereas you'd have to make some really tough decisions if that option wasn't available). We know this is true because social mobility is not that great even in the states; your background matters.

Home schooling tends to just reinforce what is already true about the home environment, so there doesn't seem to be any new variable their. Of course, we could argue that schools can have adverse effects and that you are avoiding those by homeschooling, but since home life is so dominant, you'd expect kids to just regurjitate what they've learned at home anyways, at least until middle or high school where teenagers begin to break free.

I don't know what you mean by 'home factor' but my idea would be to take children from similar backgrounds who are homeschooled or not and see if that variable correlates with some measure of performance. It's not perfect of course because homeschooling parents are self-selecting, but a fully controlled experiment is not going to be feasible; however, this does not make the hypothesis any less likely, just less testable.

Hmm, that wasn't my experience. There is a huge variation in the family types who are home schooling in the Bay Area, it runs the gamut. Two of the kids in the Reikes class were homeless, their Mom was doing this because while she was homeless she did not want here kids to get behind in their education, and California has some really strict rules about letting your kids go to a public school if you don't have an address in the district. Perhaps the biggest cohort were kids where the parents had an issue[1] with the public schools but didn't have the resources or desire to send their kids to private school. I do know that there was a group who leaned more toward the creationist side of things but that wasn't the groups in our circle so I can't say much about them other than that you could find them if you wanted to.

"It is already known that the most important factor in whether a kid learns or not is their home situation."

My experience would support that hypothesis. Of all the kids I've known over my life those who had parents that were supportive of their educational goals did better than those whose parents were absent or destructive. That said though, it didn't change the fact that for kids who were otherwise nominal (reasonable support from parents, no major mental health issues) learned a lot (and I believe more) in their home school environment than they would have had they gone to public school.

[1] Issues ranged from teachers who were 'bad', district policies (no pocket knives allowed), school choice in their district (gang issues), and 'bad influences' (drug use or sexual experimentation).

You should look into the details of MET Project that is mentioned as the source of Gates' claim. As is often the case the reporter mangled or left out many of the details.

In addition to student surveys and teacher evaluations teacher performance measured as "value added" in students test scores. These three components were combined into an overall "teacher effectiveness" measure. This was all done using what look like very good experimental practices (random assignment of students to teachers, multiple schools across the US...).

Although not perfect the "teacher effectiveness" measure was quite predictive of future "value added". So perhaps we are closer to a solution. Of course this study was in elementary and middle schools and it may be more difficult in colleges (students walking out on surveys).

The project's methodology and results are well documented in a number of reports[1] including a detailed research report for those literate in statistics[2].

[1] http://www.metproject.org/reports.php

[2] http://www.metproject.org/downloads/MET_Validating_Using_Ran...

"There's also no incentive to provide useful feedback from the student's perspective."

Some departments of the university I'm studying at got around this problem by bribing the students with printing credits after filling out the lecture feedback surveys. Other more general surveys on the campus entered all respondents into a prize draw for vouchers or cash.

Sadly my department did nothing like that and as such our response rates are as abysmal as your experience describes.

> Students are notoriusly bad at rating their professors

I wonder whether this was because they weren't asked the right questions.

The Air Force survey [1] seemed to have more broad, abstract questions like "The instructor's effectiveness in facilitating my learning in the course was good/bad/etc." or "Value of questions and problems raised by instructor was good/bad/etc." or "Amount you learned in the course was lots/little/etc."

The Gates survey [2] seemed to have more fine-grained, specific questions like "My teacher knows when the class understands." and "My classmates behave the way the teacher wants them to." and "The comments I get help me know how to improve."

[1] http://www.economics.harvard.edu/faculty/staiger/files/carre...

[2] http://www.metproject.org/downloads/Asking_Students_Practiti...