How to Analyze False Claims about Charter Schools
dianeravitch.net
dianeravitch.net
a) It's true that statistics are frequently misused by journalists and advocates, BUT:
b) It hardly impugns the schools themselves. Taking a subset of more promising students without behavioral issues out of failing public schools and giving them better resources at charters is not necessarily a bad idea.
Independent of that, it's also problematic because you're essentially giving up on the "failing schools public schools", by taking resources away from them.
> The hard-to-teach students need to go somewhere.
That's obvious. The criticism is that, in the current system, laws and tradition have dictated that "somewhere" is Oswald Cobblepot High School down the street, and if that school is a hole, sucks to be you. Blame the voters that didn't pass the bond issue five years ago. Or get rich and move to a nicer neighborhood.
And that last point is especially important. The rich have places to send their problem students. And they can afford to move to overpriced neighborhoods that effectively weed out at-risk students from the student body. The current system (unintentionally) creates public schools full of hard-to-teach students.
As I see it, this exacerbates the problems of rich schools and poor schools.
And one of the side effects of the current system is that rich parents and poor parents don't even live in the same school district anymore. And housing prices are both inflated and tied to the quality of the local schools, which is insane if you think about it.
http://www.huffingtonpost.com/2013/06/25/oecd-education-repo...
Anecdotally, I know several teachers. Their salaries, for their education level (Master's degree) and cost of living (NYC area, often teaching in the city), are very low. One particular example I know is a NYC public school teacher with a Master's in education and several years experience making $45k.
Many members of this community are fine with the notion that paying programmers more will tend to attract better programmers. I see no reason why we can't extend that to public school teachers.
I don't even know if I would agree that giving more money to all schools would solve this problem. What's to keep those standardization study materials companies from raising their prices to increase their profitability?
How about if we instead take apart the centralized institutions that allow testing companies to maintain monopolies on testing and standardized testing study materials?
Once again, this isn't a problem rooted in money for education. This is a problem of entrenched power structures that prioritize maintaining their power over educating children.
Why not a school with behavior experts? Wouldn't a concentration of students with specific needs make it easier to address those needs because you can concentrate those specific resources there? To take an extreme example, look at the success of schools for the deaf or schools for the blind.
Also, the article mentions that many of the charters have suspension rates between 20 and 50%, which suggests that either a.) the students at these schools do have behavioral issues and/or b.) the charters use aggressive suspension policies to get students to leave.
If journalists are concerned about the quality of reporting around this issue, explaining the suspension rate seems like a great story. It would be hard work, though, since schools rightly don't talk to strangers about specific disciplinary incidents.
Transfers of facilities space? The school recently took out a loan and bought an empty business complex to renovate as our new campus because the campus we were using was an old run down school that the district had already closed several years prior to us moving in. They closed it (and two others in the district) due to budget cuts. Closing these schools caused teachers to lose their jobs & class sizes at the remaining schools to increase.
I can try and find the exact reference, but a quick google search brings up a ton of articles on socioeconomic mixing.
It is a similar problem to survivor bias when researching financial fund performance.
The incentives to improve performance on statistical measures like test scores will tend to cause overly strict schools to survive whether or not the administrators started out as true believers in strict discipline or just figured out along the way its effect on test scores. Once the relationship is understood, administrators who are watching performance will be reluctant to move towards lenient policies that bring test scores down.
What if the expulsion rate (dropping disruptive students) and parent engagement is part of the 'secret sauce' of charter school success? Why should data that is based on these factors be considered 'bogus statistics'?
There is some good, common sense advice in this article but it considers too many fundamental factors 'bogus'.
This is a bit of a tangent but my fundamental problem with the traditional public school system is this: why should my child's education be primarily determined by where our domicile happens to be located?
I used to work at educating blind children. Most of those I worked with also had other disabilities as well. The school I worked at was a boarding school and first rate. It concentrated experts into one place and offered continuity of instruction beyond anything a public district could provide. Student progress was, on the whole, amazing compared to their progress at a public school. This is an extreme example, granted, but why wouldn't a similar approach work with other kids who need extra help and attention? Why not have behavior specialists at one charter school working with disruptive kids and engineers teaching kids that are interested in STEM?
Nobody assumes that. That's not what public school is meant to do.
Public school is meant to give the minimum level of education to the largest amount of students possible. "Best of their ability" is too costly- that's what private schools are for.
Public schools have special education teachers, and sometimes have entire sub-schools within existing schools just for special education kids. (My mother was a special education teacher.)
Since your school was a boarding school, I imagine it also cost significantly more per-child than we currently spend in public school. A lot of the gain from your experience, I think, was probably from having good instructors trained in how to teach students with those particular disabilities, and probably a good teacher-to-student ratio. If we were willing to pay public teachers more, public schools could attract the same experts, and if we were willing to employ more public teachers in total, we could achieve a similar ratio. We could then achieve the same thing in public schools as in the boarding school.
No we couldn't. There are only so many experts to spread around.
If the criticism is that it doesn't give a fair chance to some kids, why aren't we focusing more on giving more chances and more types of chances to people instead of forcing everyone into the same mold?
If the expulsion rate and parent engagement is part of the 'secret sauce', then everyone needs to be aware of that.
Charter schools are impressing people because they claim to get results, and people assume they're getting better results with the same students. This would be interesting if it were happening, but it doesn't seem to be happening. Anyone can take the most motivated, compliant, best-supported students and do well with them.
What if the key variables are harder truths? What if it's the education level of parents, implying that natural intelligence (or something correlated) is the most important thing. What if it's the marital status of the parents, implying the stability of their homes is the most important thing? What if it's the crime rate in their neighborhood they live in?
Honestly, it's incredibly important to know if we're wasting our time and should be concentrating on lead paint, exercise levels, or healthy marriages.
It seems to me that reporters don't really approach those kinds of issues either. We like to control for those sorts of variables as it they don't matter.
The question is: are charter schools doing better with these students than is being done with them in public schools?
On the other end of the spectrum: would it be better to filter kids with behavior issues to their own school where it would be easier to concentrate experts to help them?
How do we maximize the area under the student education curve? I don't claim to know but at least under the charter school system we have the ability to tweak the dials.
>The majority of these stories praise charters, while often demeaning public schools.
Charter schools are public schools - they are just less regulated ones.
No, they are not. They are run by private corporations. That's like saying McDonalds is a public restaurant because they are open to the public. Please don't mix common understanding of what words mean with some plausible other use of those words, it does not actually help your cause.
[1]http://www.gadoe.org/external-affairs-and-policy/charter-sch...
http://www.miamiherald.com/news/special-reports/cashing-in-o...
In North American usage a "public school" is a school open to the public and run on public funds (tax monies) instead of funded by charging students for access.
A charter school is just an independently run public school. Some are run by parent companies that are for-profit corporations running multiple schools. Some are not. I have a daughter enrolled in a charter school started and run by teachers and parents. No parent corporation is involved and it shares facilities amicably with a traditional public school.
You end up with a phenomenon where some schools get used as dumping grounds for the kids that the other schools don't want to deal with or whose parents don't care or don't know any better. I witnessed this when I had a part-time teaching job in the Boston Public Schools. I spent a year at Jeremiah Burke High School, which has the reputation as one of the worst in the state. A huge proportion of kids there either did not speak much English (Cape Verdean Creole was very common), or had severe learning or behavioral issues.
The problems that exist in public education are incredibly complex, and it's not clear to me that any simple, ideologically-tinged solution is going to solve them.
This is exactly why I want competition. A few experts sitting around a conference table for a year are not going to just deduce the solution, just like the best VCs invest in many companies instead of just picking only the Facebooks. These problems are far too complicated for a centrally architected solution. In the end the best solution might be one that most people never even imagined.
Interesting. My experience with how the media presents charter schools is quite the opposite. It seems like every few weeks there is a story about a corrupt charter school management team embezzling funds or over-enrolling of students or suspect academic results. Of course with stories on actual events, such as an investigation into missing funds, it's difficult if not downright impossible, if a media outlet so desired, to spin the story into something positive. But I don't recollect reading anything within the past few years that actually praised charter schools, and that's not to mean the traditional public school system is squeaky clean and a model of best practices. I suspect there is an equivalent level of corruption there. I've wondered if it's because I live in Philadelphia where this a strong teachers union and a traditional public school system that is, for all intents and purposes, bankrupt. However, I would expect that these conditions would lead to the opposite in how charter schools are portrayed, if only for the sake that they are an alternative to the current system that is failing miserably.
The charter school debate is a sub-segment of a larger issue, which is this push towards the privatization of everything that is consumed by the public and paid for by tax dollars, from public school systems to prisons to health care programs, such as Medicaid. Personally, I disagree with this push. It creates perverse incentives and from a political economy point of view, I think there are some goods which can't and should not be privatized, but that discussion is for another time.
A public school with selection criteria is not, by itself, a bad thing. I love that schools like Stuyvesant HS and Bronx School of Science exist, but that's because we all know what their selection criteria are.
I don't like charter schools pretending that the only thing it takes to be part of the school is winnning an entrance lottery. That might be the main way to get in, but it's not all it takes to stay enrolled. Many charter school advocates seem to be ignoring, or ignorant of, the fact that these schools actively weed out students who are difficult to reach.
It's okay to build some schools that require a more serious commitment to learning; it's not okay to build schools like this and pretend they serve everyone, or do a better job than schools who have to keep every eligible student enrolled.
I'll go down the checklist item by item.
"Does the story compare the demographics of the student population served by charter schools to the demographics of local public schools?"
What aspect of "demographics" is most relevant here? There are already charter school programs (not many, but a few) that do better with mostly pupils who come from poor homes and parents without higher education than many other schools do with pupils who come from high-income homes.[1] A lot of public schools in the United States serve up mediocre results while serving well off pupils.
"Does it include data on the charter school attrition rate?"
This is a fair item to put on the checklist, and should be reported more in stories about schools in general. In other words, a story about public schools should also report on their attrition rate, and on what rate at which pupils move in and out of the school district, and in general how stable or unstable enrollments are at all schools being compared. (The United States has a highly geographically mobile population, and it would be a very unusual school that has the same group of pupils enrolled by sixth grade as was enrolled in first grade five years earlier.)
"Does it include data on how the students who leave the charters compare to students who leave public schools?"
Again, this is a question well worth asking, but we shouldn't guess, until we have seen the answer to questions like this, whether charter schools or public schools would come out better in a comparison of this kind. We should be checking the facts in each individual case all around the country.
"Does it include numbers of students expelled?"
Since the early 1990s, I have been aware of homeschooling advocates who talk about "push outs" (not "drop outs") from the public school system, kids who are basically told to like it or lump it in the public school system. People leaving the public school system should always be looked at closely by researchers to figure out why they left, and whether they are later able to find a more fitting school situation in which they can learn better.
"Does it include numbers of students suspended?"
As above, this is a question to ask about any kind of school. Moreover, at the extremes, a school that never suspends any pupil is possibly a school that is not trying hard enough to maintain a learning environment for all other pupils enrolled, so a researcher would want to look at grounds for suspension, procedures surrounding suspension, what corrective steps are taken when a pupil is suspended, and so on. I've certainly seen suspension abused in public school settings.
"Does the story focus exclusively on test scores?"
This is a general defect of reporting on schools. At the broad statistical level, and especially for international comparisons of school systems, test scores[2] are mostly what we have to look at. What I also look at, as someone who has lived in more than one country and who reads more than one language, is the actual item content of textbooks and the attitudes toward learning shown by teachers and pupils. I think some school systems in other countries compare very favorably to those in the United States (not least because pupils in those other countries begin foreign language study much earlier than pupils in the United States, a fact not usually reflected in international testing programs). Educational researchers should be looking more closely at the actual item content of school textbooks and at the specific teacher practices in each classroom.[3]
"If so, has someone, with educational expertise, visited the school to determine if the school focuses on test prep at the expense of a rich curriculum?"
As above, we should be looking in detail at what actually goes on in the classroom. Indeed, we should be looking at whether persons employed as teachers are actually teaching anything, which we shouldn't assume sight unseen.[4]
"Are the test scores reported outside of school assessments such as the SAT/ACT or does the story only report test scores of exams that are proctored in-house?"
This is also important, and also a question that should be applied to news stories about schools in general, not just about charter schools. So far many of the worst cheating scandals in school testing programs have originated in regular public schools.[5]
"Does the story account for the fact that, due to the need to apply to the charter school, parents of the students at charters are, on average, likely to be more engaged in education than the parents of students at public schools?"
The policy response to this is to make school choices much more widely available to many more families. I have seen some examples of helpful reforms where I live. Minnesota, where I now live and where I grew up, has had largely equal per-capita funding for public school pupils statewide since the 1970s. The state law change that made most school funding come from general state appropriations rather than from local property taxes was called the "Minnesota miracle."[6]
Today most funding for schools is distributed by the state government on a per-pupil enrollment basis.[7] The funding reform in the 1970s was followed up by two further reforms in the 1980s. First, the former compulsory instruction statute in Minnesota was ruled unconstitutional in a court case involving a homeschooling family, and a new compulsory instruction statute explicitly allows more nonpublic school alternatives for families who seek those. Second, the Legislature, pushed by the then Governor, set up statewide open enrollment[8] and the opportunity for advanced learners to attend up to two years of college while still high school students on the state's dime.[9]
And Minnesota also has the oldest charter school statute in the United States.[10] We haven't had big problems with charter school performance here.
Parents in Minnesota now have more power to shop than parents in most states. That gets closer to the ideal of detecting the optimum education environment for each student (by parents observing what works for each of their differing children) and giving it to them by open-enrolling in another school district (my school district has inbound open-enrollment students from forty-one other school districts of residence) or by homeschooling, or by postsecondary study at high school age, or by exercising other choices.
The educational results of Minnesota schools are well above the meager results of most United States schools, and almost competitive (but not fully competitive) with the better schools in the newly industrialized countries of east Asia and southeast Asia. It's a start. More choices would be even better.
"Does it exclusively or primarily cite reports funded by pro-charter or conservative think tanks?"
Here it's regrettable that we don't know the identity of the author of this checklist. Do we know whether or not the author is funded by pro-public-school interest groups? Disclosure of that would be helpful here.
"Does it include quotes from academic scholars or does it just cite charter school advocates?"
I can recommend a website full of the writings of actual academic scholar to Hacker News participants interested in education reform. That is the Education Next website,[11] which is better supplied than most English-language websites about education policy with information about education policy outside the United States, which is important for a reality-check on issues discussed in this country.
"Does it identify advocates or simply call them 'experts' or 'researchers'?"
This is a funny question for a commentator who insisted on anonymity to ask.
"Does it compare the resources available to charter schools to those available to public schools?"
In Minnesota, and in all states I have checked, the resources available to charter schools (funding per student, and availability of buildings) are strictly less than the resources available to public school disticts, by law. Yes, lets's check that detail carefully.
It's commendable to receive a suggestion that we should check news stories carefully. We should always do that here on Hacker News. Let's check stories on education policy especially carefully, as public school spending is a huge part of state budgets[12] and a lot of interest groups are lined up behind the public trough.
[1] http://educationnext.org/when-the-best-is-mediocre/
[3] http://www.amazon.com/Knowing-Teaching-Elementary-Mathematic...
http://www.timssvideo.com/timss-video-study
[4] http://www.pbs.org/newshour/bb/students-take-a-stand-to-topp...
[5] http://www.ajc.com/s/news/school-test-scores/#top-stories
http://online.wsj.com/articles/for-school-tests-measures-to-...
http://www.newyorker.com/magazine/2014/07/21/wrong-answer
http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/08/...
[6] http://www.mnhs.org/library/tips/history_topics/18public.htm...
[7] http://www.house.leg.state.mn.us/hrd/pubs/mnschfin.pdf
http://education.state.mn.us/MDE/SchSup/SchFin/index.html
[8] http://education.state.mn.us/MDE/StuSuc/EnrollChoice/index.h...
[9] http://education.state.mn.us/MDE/StuSuc/CollReadi/PSEO/index...
[10] http://www.amazon.com/Zero-Chance-Passage-Pioneering-Charter...
[11] http://educationnext.org/
Given that, I've ignored most data I've seen about dropout rates. Do you have any perspective on this?
PS I always appreciate your comments on education. Thanks for continuing to offer your perspective, even though these articles often get killed.
There are certain base assumptions that we all make that need to be addressed - for example, the article compares the pass rates of charter schools to public schools in Chicago. Specific language is used - lower pass rates are "worse", which, then, implies that higher pass rates are "better". This is one of those scenarios where base assumptions need to be challenged.
Simply passing means nothing. For example, 19% of high school graduates can't read [1]. So while you can pass uneducated people through an education system, that does not make it "better".
Charter schools pass less students, but maybe that's because they don't pass students who can't read. Instead of looking at pass rates, maybe we should address other metrics - do public schools or charter schools graduate less people who can't read?
Essentially, when reading metrics, think critically about whether or not the metric is a "vanity metric" (such as pass rates) or if it actually means something (like literacy rates of graduates).
---
Notes
[1]: http://www.statisticbrain.com/number-of-american-adults-who-...
Yep. The same logic can be used on the (supposed) higher suspension rate at (some) charter schools. Is it possible that maybe some non-charter schools just don't care to discipline problem kids as aggressively?
In fact, the point in the article [1] seems to indicate that US Schools do more poorly than international schools, whereas you posit that (some) charter schools do better than public schools with the same demographics.
Furthermore, unless those exemplary charter schools are forced to take any students applicants (which seems contradictory to their "charter") as opposed to those who "qualify" or "via lottery".
There is no question that the US education system could be improved. The problem I have is that there is a lot of money betting with Charter schools and against traditional public education. Investors aren't stupid, they expects an ROI. And that makes me skeptical that it's really about the kids and not the lucrative potential of privatized education.
Charter schools are required to take applications for anyone who wants to apply. They even take applications from outside the district. There is no "qualifying" for entry. However, as with any school facility, space is not infinite. One of the things that lets charter schools stand out is that they usually have smaller class sizes than the non-charter schools. They are not going to just cram more kids in. So if 1500 students apply for a school that only has room for 1200, then a randomized lottery is performed. I don't know how other charter schools do it but ours has the most opening for kindergarten (obviously), very few spots for 1st-4th (only available spots are to replace students that leave the school), a few more in 5th grade (a class size increase allows for more new students) and then back to very few spots for 6th-8th. But the charter does have other provisions that allow for the lottery to be not truly random. It is weighted a little. Students that live within the district are weighted higher than students from outside the district. Students that have a sibling already in the school are also weighted higher.
Also, if the lottery can be "weighted" then how is it a lottery in any way shape or form? It's essentially administrator's choice, is it not? Public schools essentially have no choice as to who can attend except for rare exceptions (expulsions, not immunized without waiver, etc).
Essentially charter schools are in a position like Uber - they get to play by different rules but then get to claim they're better when compared with handicapped public schools in the same area.
Think of it like a raffle. Every applicant gets 20 tickets. But in-district students get +2 tickets. Students with a sibling get +1 ticket. The raffle is still random but some people have slightly higher chance of winning.
> It's essentially administrator's choice, is it not?
No, it is not. There is no administrator making any choice. There is an algorithm that sorts all the anonymous applicants into a list for each grade. Once the lists are set, they "draw a line" at the spot where that grade is full. Everyone below the line is now on the waiting list. In-district students get a slightly better score than out-of-district students. Compare and contrast, it is very difficult to attend a non-charter school that is not the one assigned by the district (based on your address). Probably impossible for out-of-district students. The fact that charter schools will even accept applications from outside the district means their enrollment is more open than non-charters. Siblings that live in the same house would automatically be going to the same non-charter school anyway. So allowing charters to do the same gives them no advantage. But even then, there is still no guarantee that you will get in even if your sibling is in. You just have a slightly better chance. What ever imbalance the charter school's class size makes is negated by the fact that they get significantly less money per student than non-charters. That is part of the compromises that are written into the charter.
Whether that is true or not depends on the particular charter terms, and the rules governing what terms can be in charters vary from state to state.
I have a good friend who I have had since college days. Has been an NYC public school teacher for a while. Loves to teach - onel of those folks that is constantly formulating ideas, reads voraciously, argues passionately, etc.
I see him as caught between the attempt to corporatize education and a bloated, syphillic (sp) union... My ideas not his.
On my side I had kids in public school until 2nd grade where I saw disturbing things that made me jump ship to a charter where my boys thrive.
This is anecdotal - I do not want to try and prove/ disprove anything by this.
I wish we had a viable pub school option locally but we don't at the moment. Our school survived a scummy attack on charters locally in which false claims were made about segregation. This was disproven but we were made aware of the fact that the other side has more resources and deeper legal pockets then the charter school does.
As I said I will sit down and read this later. I am interested in something that comes up when my buddy and I talk about this stuff.
Somewhere I picked up a concept from French which I will anglicize as "Deformation Professionelle" (my apologies for butchering it - I will properly google it later).
ed. http://en.wikipedia.org/wiki/D%C3%A9formation_professionnell...
I see this when tech and teachers discuss education.
I sit and talk to him about tech issues - 'net neutrality', privacy issue, (going back a decade) attacks on open source software, etc. And he doesn't know about these things.
And I (at times) wax enthusiastic about things like Khan Academy or charter schools without seeing possible drawbacks.
I look forward to giving this a proper read later - am glad to see this on HN.
And then the justification for why bother:
It grows tiresome to dispute every tendentious article written on charter schools. But let's see how the ICBS strategy would help us evaluate a sample story.
At the end of the day, school choice doesn't matter. The fact that Charter schools are relatively new and just finding their way doesn't matter.
I'm all for apples to apples comparisons. But when you have a huge basket of fruit you're trying to normalize to apples, it's really telling which ones are picked.
I'm saying that in the big picture, those probably aren't the specific statistics that many people even care much about at this point.
It's like an article comparing a Tesla to a Ford pickup truck that focuses solely on 0-60 speed and fuel costs but never bothers talking about towing capacity, load capacity, ability to refuel anywhere, etc.
If Charter Schools are attended by (e.g.) wealthier students than public schools, or students with more involved parents, then a charter school's superior aggregate performance on a standardized test tells us precisely zero about the educational quality of the school unless one can somehow compensate for this confounding variable. Yet this is routinely overlooked in the media.
You seem to be saying that we shouldn't care about controlling for variables like this and should, instead, look at other ones. OK. But you don't tell us why, or what statistics would be preferable, or give any evidence to suggest that publications like the Economist actually consider your preferred measures in lieu of those suggested by the article (remember: the article is about how to evaluate the claims made in the media about charter schools, not about the merits of charter schools themselves -- though I readily grant that the author clearly has a view on the latter topic as well).
You seem to be saying that we shouldn't care about controlling for variables like this
I'm saying that those variables and the questions that this "expert" chose to put forth miss the mark on the questions that should be asked and the variables that should be examined.
But you don't tell us why, or what statistics would be preferable
I did tell you why. My guess is that if you thought that choice was important in education then my thinly veiled sarcasm about choice would have caught your attention.
Regarding statistics, the statistic alone that typically Charter schools receive less per student than mainstream public schools renders questionable every other statistic that this article mentioned.
I agree that statistics are massively abused in politics, and ironically these sorts of post hoc ergo propter hoc arguments are especially prevalent when it comes to issues of education. Take, for example, the recent push for universal public preschool.
I didn't call it a "hit piece" because I necessarily disagree with the statistics the author chose to present. I called it a hit piece because of the specific statistics the author chose to present and the condescending manner in which the author presented them.
> It was, as is typical among news stories, incredibly naive. The writer didn’t ask the right questions. Maybe he already believed in the charter “miracle” story and didn’t ask any questions.
The author concluded that the claims in the story were probably false (or at least not supported by evidence) on the basis of the framework described in the article. That's what is meant by "didn't ask the right questions." (Of course, given that the author is well versed in this area, it did not take much effort to spot the mistakes.)
Also, I don't understand why so many leftists hate charter schools so much. First, charter schools are public schools, not private schools. Second, charter schools are created to help the poor, not the rich. What's wrong with those leftists? I just can't understand. Do you really love teachers union so much that poor kids mean nothing to you?
Parents should be involved in the education of their children. If they take the (tiny, tiny) extra effort to fill out the application to a charter school in addition to enrolling in their designated neighborhood school then those are the ones that have a chance to win. You can't blame the school for the parents not involved enough to make an effort. (and really... the effort is not that great)
This is in theory, many charter schools are not following this principle, see for example: http://dianeravitch.net/2013/01/14/do-affluent-white-neighbo... http://www.bloomberg.com/news/2011-11-15/taxpayers-billed-fo...
Charter schools are funded with public money, but they are run (and often owned) by for-profit management companies.
http://www.miamiherald.com/news/special-reports/cashing-in-o...
Many people see them as just another way to get public money in the hands of a few private companies.