Test Scores Cast Doubt on U.S. Education Reform
nytimes.com
nytimes.com
I’ve only got a decent sample for one state, but here the state decides what its common core standards and teaching methods will be (rewriting the national standards, I suppose to justify someone’s job) then those are filtered through districts (admin, curriculum advisors) which hopefully turn that into something useful for classroom teachers, who are to use what the district provides. Materials, plans, and so on, are typically not among what’s provided.
Districts usually whip their focus or emphasis between tested subjects (fuck untested ones, who cares, obviously) based on whatever did worst in the most recent test results. Districts can implement all kinds of their own practices or frameworks for discipline and classroom management, materials, teaching methods, class scheduling, class structure in a broad sense, and so on. These are often drawn from popular or trendy books from education researchers and consultants. Admin (principals, superintendents) are often not much on systems or game thinking so tend to partially implement whichever of these they’ve decided on (because some part is inconvenient or otherwise not something they want to do) usually in such a way as to cripple them.
Nepotism is common. Small- to medium-scale fraud is common. Administrators will find a way to spin all their decisions as positive to protect their jobs and pad their résumés for the next one, and no one with the power to correct the record will do so because it’d make the district look bad. Principals do similar things as they all want to be superintendents or at least a principal in some district easier & better known than the one they’re in now.
It’s a mess and it’s impressive it works as well as it does.
Centralization and standardization is generally only useful when you know what needs to be done, it just needs to be done everywhere. There is considerable evidence that we _don't_ know what needs to be done differently, so centralization is the last thing we need.
I know lots of school administrators are effectively incompetent at implementing reforms & any kinds of systems, really, directed from higher up or otherwise (frankly, it's usually even worse when "otherwise"), due to some combination of ignorance or misaligned incentives (the balance of these varies but one or both are often present in some quantity). Whether it's worth bringing the few who have a clue and are choosing to use it to-heel in order to attempt to fix the many who don't and aren't, I do not know, but so far as I know anything like that's pretty far outside the Overton window so I wouldn't worry about it either way. Just recording how this stuff looks when it hits the state, district, and school level, since not everyone has insight into that, and it's largely a mess.
Probably it's worth looking at how schools are run and their governance structured in other countries that are doing better than us, if the core motivation of all this anxiety is our falling behind them.
- The boredom of being imprisoned all day long
- The slavery of feeling forced to do uninteresting homework
- Lack of free time after school to pursue your own interests or learn your own material
- The toxic culture in school, and being forced to be around people you hate all day long.
- Lack of sleep, etc...
...all to regurgitate a bunch of short-term memory into a piece of paper, and forget it soon afterwards.
For most honest people, school was mostly boring and overall a waste of time.
I believe older children and teenagers have the ability to choose material that comfortably challenges their own ability better than any software or person. Students aren't lazy by default. They're just lazy and unmotivated when forced to focus on what their brains consider boring.
As for more/better teachers: the best education is already available for free online (such as Khan academy). I'm all for face-to-face interaction, but that could happen almost for free with older student mentors, and ocassionaly with paid professors.
Also, if maximum benefits mean "highest scores" or "highest efficiency of time", I'd rather youngesters learn to rest and have more down-time.
I'm not suggesting "I have solved the education problem". Just thinking outside the tiny box our system has been stuck in for decades too long.
The single biggest problem in the US educational system is that the people in charge of educational policy need to stop and have a nice thoughtful think about the streetlight effect.
This is all crazy what we've been doing to the educational system for the last 30 years, and if we can produce functioning adults from it, it will occur despite the emphasis on test scores, rather than because of it.
We had one of the finest education systems in the world, and we're taking all the wrong lessons from other developed nations on how to "improve" it.
And in the mean time, we're causing severe burnout among all the hard-working and passionate teachers who are on the front lines of trying to deal with all the issues with the educational system today.
I've been acquiring a new language over the past year or so. I went from beginner to a decent (for my purposes) level of fluency in about a year. The other day we had a friend we hadn't seen in a while over for dinner. She's a school teacher in the very language I had been learning, so, naturally, the bulk of the dinner conversation between us proceeded in that language. And, naturally, she commented on my rapid progress, when most her students can't read a comic book after eight years of study, and wanted to know more about my study technique.
I kept it positive because I didn't want to trash her whole career, but I can say that my only real secret sauce was that I studiously avoided just about every practice in mainstream language education. I didn't do what schools do, and I didn't do what Duolingo and Rosetta Stone do. Because, at the start, I spent a while reading up on the academic research on how people learn new languages. There's a lot of it, it's about as well established as any sub-field of psychology can be, and it is, to a remarkable degree, summarily ignored. The mainstream has no choice but to ignore it, because most of what's been discovered does not lend itself well to the establishment of standardized curricula or easy, objective assessment criteria. That means schools don't want to take that advice because it's incompatible with their mandate to assign letter grades, and Duolingo doesn't want to take that advice because it's impossible to gamify.
The people I know of who've successfully learned a second language that way are invariably characters on TV shows. Having spent much of the past few years of my life helping two small children learn their first language, I have my doubts about whether the "immersion is magic" approach is really even the most efficient way to pick up language number one.
Of course, the most important thing is to have a motivation for reading English text in the first place and wanting to understand it. If you don't have that motivation you won't have any lasting success.
Though, I know of methods that allow you to learn any language in six weeks, but it involves talking to a native speaker.
I believe the idea of common schools originated in the USA, or at least it did as far as Western countries are concerned. Well into the 19th century, the European attitude was that parents were responsible for providing for their own children's education, and nobody else's. Working-class children were largely shut out.
More contemporarily, the US is still in the top 10 by at least some rankings. But talking about those rankings doesn't sell ads or get the voters riled up or help rich parents feel good about all the money they're spending on private schools, so there's a tendency to cherry pick the rankings where the US is merely in the top 10%. Ideally followed by intensely fetishizing Finland.
(Fetishizing Australia, which tends to rank even higher than the nordic countries, never seems to happen. Funny, that.)
Our Ivy leagues are still generally the most resourceful, best equipped, and best staffed in the world.
It's changing though.
Many PISA questions contextualize academic knowledge in real-world scenarios such as questions about purchasing apartment or intravenous drip rate in medical infusions.
https://www.oecd.org/pisa/pisaproducts/pisa2012-2006-rel-ite...
A question in reading test is about the effect of the word 'free' in making purchasing decisions:
http://www.oecd.org/pisa/aboutpisa/PISA%20for%20Schools%20sa...
A question on bee colony collapse disorder in Science test:
https://www.oecd.org/pisa/pisaproducts/PISA2015-Released-FT-...
No test can substitute for one-on-one evaluation, but one-on-one evaluation is also subject to (different) biases. As far as standardized tests go, these are pretty good ones.
More importantly, I believe the results would likely correlate relatively well with how well the students apply knowledge in real-world settings.
I would call into question both assumptions. The USA isn't perfect by any means but we have one of the most transparent bureacracies in the world. You can't say the same for a country like China. You have to wonder if the test scores are accurate and represent a comparable cross section of society.
Also, common core sucks. When my son was in 3rd grade, the teacher and parents felt like the math textbook was terrible. Mainly because it was all word problems. I have a master's degree in engineering and I found some of those word problems to be ambiguous. The result is that if you were struggling with reading in 3rd grade, you were also struggling with math. Because it was common core, it couldn't be changed at the class or school level. Common core's goal seems to be to remove teachers and parents from the equation of how best to educate your child.
I faced a similar problem when my son was in 3rd grade (I also have an M.S. in Engineering). All of the work seemed to be geared to helping kids develop an intuitive understanding of math and numbers, which is a great goal, but the way it was taught was to teach the same "intuitive" methods to all kids. We all intuit things differently and my humble opinion is that parents and teachers aren't fully prepared to understand and teach math this way. Attempts to promote intuition simply become methods that are different than the ones we learned.
I was able to work around those things and help my son, and he was able to develop a fairly good intuitive understanding of numbers. Where the approach falls down (again, in my opinion) is where structured approaches are required, such as in long division. By the time 5th and 6th grade came around, my son didn't have the reps at doing it and had some struggles.
https://www.oecd.org/pisa/PISA-results_ENGLISH.png
A quote from the Economist article on PISA results (which the NYT article is also based on):
“In these parts of China the average pupil’s maths score is 591, compared with an OECD average of 489, suggesting local teenagers are roughly three years ahead of the OECD average.”
Note that the total population of the four Chinese provinces that rank global top in all three PISA core subjects is about 180 million.
This following bit of insight from the same article can be implemented and should have impact over time. I would add that small class size might be more important for younger kids. At the junior high and high school levels, larger classes are acceptable when the teacher is engaging.
“Andreas Schleicher, head of education at the OECD, bemoans the fact that lots of countries have, for instance, prioritised shrinking classes over hiring and training excellent teachers, despite evidence suggesting this is a bad idea. As he points out, one place that has given the quality of the teacher priority over the size of the class is Shanghai. Another is Singapore. And they are reaping the benefits.”
https://www.economist.com/international/2019/12/03/after-two...
To handle a larger class, Singapore schools use technology such as an in-class messaging system that allows students to submit answers individually and show them all at once on screen, thus reducing the bandwagon effect and promoting independent thinking. (I saw a YouTube video on it and it was used really well.)
https://www.forbes.com/sites/realspin/2017/01/04/are-the-pis...
Moreover, MA's population of ~7 million is only ~2% of US population, whereas the four provinces' 180 million is about 13% of China's population.
"for the Program for International Student Assessment, or PISA. It shows the state is tops in the country in math, but when you zoom out and look globally, Massachusetts ranks 20th in the world."
https://www.wbur.org/edify/2017/04/10/massachusetts-math-edu...
"I found that if Massachusetts were allowed to report subject scores independently ... the Bay State would rank 9th in the world in Math."
https://www.forbes.com/sites/jamesmarshallcrotty/2014/09/29/...
I think there should be a clear-eyed view toward China's progress in education as well as advanced research. For example, based on some measures of publications in top AI conferences, among the top 10 academic institutions in the world, 4 are in China, 4 in the US, and 2 in Singapore.
http://csrankings.org/#/fromyear/2018/toyear/2019/index?ai&v...
> that the sampling done on mainland China (Beijing, Jiangsu, Guangdong and Shanghai) and other cities was not taken from a wide variety of schools. Rather, the very best schools were chosen and the very best students were cherry-picked from those schools.
So this is more than just focusing on Massachusetts, this is picking only the best students from the best schools in Massachusetts.
I don't find this all that surprising. I've done casual comparisons of Chinese curricular materials for math vs. Canadian (Ontario) materials, and there is about a 3 year difference in content level being presented to the students at equivalent age levels. Of course, there could be other elements like selection bias, etc. playing into the results, but there is actually an underlying divergence in terms of the basic curricular material.
https://www.theatlantic.com/notes/2015/11/more-on-chinese-an...
>In many cases, the teachers would like to take a lesson from American schools and concentrate more on imagination and resourcefulness. They say it is the parents who want to keep the focus on objective assignments, drills, and tests to prepare students for exams.
>Any understanding of Chinese culture has to begin with how a "repeat after me" education focused entirely on objective tests affects communication skills, analytic thinking, teamwork, creativity, imagination and initiative.
Creativity understood more broadly is a tough thing to shoehorn into a math curriculum, because average teachers have difficulty with mathematically creative material that is sufficiently in-depth. The Discovery math curriculum that's popular in North America is emblematic of that; there is a huge amount of time devoted to finding multiple creative ways to visualize/deconstruct basic operations, but actual, profound mathematical creativity like exploring various ways to prove interesting theorems is largely absent.
How common is it to have the local funding of schools outside the US?
Schools are funded by property taxes. In lots of places this leads to expensive suburbs allocating lots of money to schools and leaving neighboring cities and low income areas out to dry.
Is this just another symptom of increasing wealth disparity in the US.
It's interesting to me the degree this article focuses on curriculum and not resource allocation.
Suburban schools do better largely because the kids are easier to teach, not funding. It correlates with richer districts because kids whose parents bought more expensive houses to get their kids into better schools tend to be kids who are easier to teach, for a bunch of reasons.
Those parents tend to emphasize education. Those parents tend to encourage literacy, and discourage bad behavior toward teachers. This creates an environment where all of the children, more or less, have a reasonable environment for learning, but the largest contributing factor is the parental involvement.
Children don't become magically easier to teach because of resources. Resources at home, however, can help the parents focus on their child's behavior and amenability to learning. This means that the problem of ineffective education of poor children must be addressed at home, not in the school.
Second, why do we insist that a single number is going to usefully assess the many skills being learned? Or that any single metric, however informative, should be applied across all the many subgroups that make up a large population like the US? Why the hell don't we stratify students and policies and regions and then compare only across pairs across strata? Only in that way can you learn what works and what don't.
We should be breaking down these numbers by student socioeconomic group, by state, by county, by school district, by education policy, by gender, so that we can compare results within the US to see how different policies on similar student groups turns out, and similar policies on different student groups.
What works and what doesn't? After 70(?) years of standard tests, it seems we still have no idea. We're never going to learn anything useful from any kind of testing if we don't 1) partition the parameter space better, and 2) ask simple (actionable) questions of the results.
Likewise, we need to break ourselves of the insistence that a single education policy is best for all students everywhere in the US. Geez. Let different states propose their own experimental design and then assess the results of their different strokes. Supposedly that's what governance at the state level was for, ever since 1789.
Comparing the US to other countries using a single metric sounds like the least useful way to improve US students.
Otherwise you get grade inflation for example when I left school an Civil service ASO (assistant scientific officer) you needed 5 or 6 O levels (age 16 exams) now its a Degree entry.
But a bigger question is what do we consider effectiveness? Do higher scores on math tests mean a school is more effective than another? Because if we do we are saying that the "best" schools are the ones that provide the best test scores. Is that the measure we want for our schools, for our education and for learning in general?
What do you think about that?
What else would you put emphasis on that can't be easily skewed by rampant subjectivity / favoritism?
First you could use more evaluative criteria. This is how most knowledge workers are evaluated in most modern startups. Unless you work on sales, you are probably measured by the quality of your thinking, your attitudes, projects, and initiatives. You don't deliver a piece of code and get a grade.
Things like PISA were invented so that the people managing could, well, manage. So they could compare and get "concrete" numbers to get elected.
And in terms of your question, what is wrong with subjectivity? We are dealing with human beings here, not computers. There will always be subjectivity. Our desire to boil everything down to data is a clear reflection of the computer age where everything needs to be a 0 or a 1. The medium is indeed the message.
I think a broader question is what is the purpose of school? Is it an institution where we help people learn, or where we teach people how to memorize things? We are what we measure. If we are measuring how well students can memorize things and give "right" answers, this is the kind of society we will create.
This is the reason schooling continues to be more and more irrelevant today.
I'd be much happier if my code received a grade. My boss doesn't currently like me because I'm not a complete "yes man", while the other Project Managers here would 100% tell him the moon is made of cheese if that's what he wanted to hear. "Evaluative criteria" are ripe for abuse in a thousand different ways, including internalized bias and racism.
There will always be subjectivity, but we can and should try to minimize it. I can't find the article now, but I remember reading a study where classical musicians were graded with the reviewers watching them, versus only hearing them play behind a curtain, and the scores were substantially different, exposing considerable bias.
Here is something similar... https://www.theguardian.com/women-in-leadership/2013/oct/14/...
Hence, standardized testing. It's practically a miracle, as far as I'm concerned -- a truly fair way to quantify many types of intellect / skills.
>This is the reason schooling continues to be more and more irrelevant today.
Schooling is more irrelevant today as people are realizing that putting a child into a school doesn't make them smarter, just like putting them onto a basketball court doesn't make them taller.
It's just conditioning and babysitting (which most families need, to be fair.)
But here is where I think I differ.
If you are grading a performance, or evaluating the effectiveness of a drug, or which peanut butter brand you like more, then sure, go ahead and do a "standard" test.
But is that the point of school? Is the point of "learning" and "schooling" to test (in a standard way) how much students can do "math," or write "proper" english, or memorize "facts" about America?
I would say "no."
Does a score of 500 versus 400 mean that someone "learned more?" As soon as you have "tests" with "grades" you immediately optimize for providing the "correct answers" and to get "better grades". I think people jump to fast to correct answers = learning.
An example, in a similar way that if your code got graded, every coder would immediately optimize what they do to get that A. I am not questioning the method of how that A is evaluated, but I am questioning what are we optimizing for. And optimizing for giving correct answers is not a good idea, in my opinion, since it bears little resemblance to any of the important open-ended questions we are faced with on our day-to-day lives.
Schools should help people learn how to learn, cultivate their own interests, and develop skills and aptitudes for a changing world. Not "find" "correct answers" to problems that someone already knows the answer.
In regards to your points: > I'd be much happier if my code received a grade That's interesting and I'm sorry about the boss... I'm interested: what would be the criteria to determine the grade of the code your boss would give you? I'm assuming a few of them would be: does it run, does it follows proper syntax, is it simple and the best way to do this, correct? But what if you think there is a better way that goes against the criteria?
> School is more irrelevant... Agree 100%. Again, what is the point of putting people in school then. Isn't it to make them smarter? Is it to get them a job? Or is it to teach them how to conform? What do you think?
Thank your for the link, by the way! Let me know if you find that study. Sounds super interesting.
Around 2009, I was considering a change from software. I applied to both Teach for America (TfA) and the Boston Teacher Residency (BTR. Similar, but at least in principle involves more training and you end with a masters degree in education) and got in to both. I applied to teach high school math.
Brief digression: Why that route and not following the traditional route of college? Because I wasn't willing to sink a shit-load of my own money into taking a (massive) pay cut. Digression over.
I went to BTR and got my first firsthand view of how the education system works from the inside. The comments below are principally about the education system, not BTR.
shantly's comment [0] is spot on. There is a huge emphasis on testing, even in a school that is trying to emphasize broader learning. The school I was placed at had pretty minimal classes outside of the same shit we all suffer through. I believe the only music class available was choir. That may have been the only art class at all.
The math teachers I worked didn't ONCE meet with teachers of any other subject to do any discussion let alone planning about to alight subject matter. There are some pretty obvious ties between most math and most science classes. With a little effort, I'm pretty convinced you could find ties to non-STEM classes too, but everybody is focused on their own silo.
Outside the building, it's worth remembering that the standards/"best practices" change with a frequency that virtually ensures that every student will live through a shift in methodology. Remember No Child Left Behind? Me either; I was out of school for that, but whatever that was gave way to the seismic shift of Common Core some time in the late aughts/early teens.
Nobody (within rounding error) spends a lot of time reworking their entire curriculum/lesson plans every time a shift is made, because everybody knows that it's going to be changed again soon enough. In due time, the better teachers will have updated piecemeal as they have time and see what parts of the new methodology are good. They'll be done just about in time for the next big wave of change. Students, meanwhile get caught in the middle of these changes as everybody is trying to get their feet back under them.
If it wasn't bad enough that the politicians feel the need to change things up regularly, you have the well known problems of the morally bankrupt textbook and calculator industries. Unless of course, your school doesn't have enough money to have a book for every student (not a hypothetical; I never saw a math textbook during my time in BTR).
But wait, it gets worse: is your boss unskilled in your area of practice? Don't worry, if you're a teacher, you're almost guaranteed to be in that situation. Two types of teachers try to get into administration. A few good, idealistic ones, and a lot of crappy ones who don't much like teaching and would rather appreciate a pay raise.
The problem propagates upwards. At some point you leave the bureaucratic side of administration and enter the political side. Good luck finding a skilled teacher there. Unfortunately, since everybody has had the experience of going through school, a lot of people have strong opinions of how school should be done. Periodically, one of those people gets high enough to drive a change, and you end up with the aforementioned seismic shifts.
And then the press comes around a couple years after a shift and writes a whole bunch of articles along the lines of "We fixed it: Why isn't it better yet?". The answer is because your pipeline extends almost literally to the birth of the student. But nobody wants to face those problems and everybody wants a quick fix.
Full disclosure: I decided that while I didn't much want to be in software, I didn't much want to be a teacher either around February. I would have been lousy anyway, at least at first: I'm not much for multi-tasking, and I didn't want to spend several years trying to get good and letting my programming skills get out of date. All for the aforementioned big pay cut.
[0] https://news.ycombinator.com/item?id=21692026
[1] Most teachers would rather appreciate a pay raise, but most of them aren't interested in going admin to get it.
In middle school in Italy, all the classes were coordinated so you were learning similar material. Therefore, if in tech class we were learning about the steam engine, in History class we were learning about the Industrial Revolution, in Literature class (Anthology) we read chapters from 19th books, in Art class we studied 19th century art.
The end of middle school exam was in front of the whole body of teachers, and you had to present what you'd learned in 8th grade. You had to cover every subject, but you had to transition to each subject seamlessly.
It was a lot of work, but I haven't forgotten any of it 20 years later.
Then I moved to N. America. There were positives, but school wasn't one them.
I did a similar route for Arkansas and the Non-Traditional Licensure program. My first year teaching circa 2004 I had 18 math text books for Algebra I class that had 25-ish students. The kicker was it was the exact same book that I had when I was a freshman in high school in 1993.
Not only did I not have enough text books, I was using a text book that was at least one maybe two shifts in teaching standards away. Hell, we had a smart board given to the school as part of a grant but no one could use it for fear of tearing it up. A locked up resource does no good, nevermind that a Smart board is useful but at best is a nice to have. Textbooks are a must have in comparison.
It was an utter shit show and explains more about why education in Arkansas is horrible.
Take the Quadratic Expression forms
o Standard Form: ax^2 + bx + c
o Factored Form: a(x – r1)(x – r2)
o Vertex Form: a(x – h)^2 + k
I would have needed to recognize that shortcoming and figured out a way to cover it so that my students didn't appear to be lacking according to the ever-shifting standards. Not impossible but arbitrarily stupid to redesign the standards every couple of years.
That seems to be the real problem.
One approach is the big government approach: nationalized education standard and accountability system, which includes NCLB / ESSA and common core. This approach is heavily pushed by the Bush and Obama administrations, and some left-wing organizations (mainly Bill Gates Foundation). It is the one addressed in the article.
On the other hand, there is another approach is the decentralized approach: school choices. It was suppressed by the federal government and the education establishment (the teachers union and their allies), but has been pushed in some states and cities.