Unfortunately, all the excuses in the world don't change the fact that these tests hurt education and hurt children. The production of these things is a shameful act.
I thought the tests we had in the UK were bad, and they are, but the US obviously has a far greater problem. I hope this changes, because you are hurting your children and damaging your future.
The rub is that: so do terrible teachers.
It's inevitable in a world where bureaucrats take a top down approach to test taking and measurement that you end up with people who "teach the test". It sucks.
But, when left to their own devices, some teachers don't teach at all. Teachers follow the same distribution of personalities as any other occupation on this planet; some are good, some are bad, most are average. These standardized tests were an attempt at saying, "You MUST teach at least this well", with "well" being defined by factoids. So, on one hand you have a free-for-all, where great teachers influence and inspire, while poor teachers ruin lives. The other side is a middle ground where what is to be taught is quantified, and everyone gets a mediocre (at best) education.
What we need a hybrid system that allows good teachers to do what they want, while bad teachers are held accountable and forced to, at the least, teach the test.
"Factoids" is a laughable metric for learning. (Especially when you're looking for factoids in a long-form response. If you want factoids, you need to ask for factoids. This test asked for complex thought, the grading should require complex thought.)
How is it possible that a student is worse off having the knowledge necessary to pass a test than to have no knowledge?
This test asked for complex thought, the grading should require complex thought
Kids are dumb. Their "complex thought" is really quite simple. Further, the iPhone required a stunning amount of complex thought to create, yet I can asses its functionality with only simple observation. I doesn't necessarily require complex thought to asses the product of complex thought, particularly when that's complex thought at grade level X.
That's a totally false dichotomy. My point is that the test rewards students for simplistic thinking. It rewards the children who are not learning how to learn, but learning how to memorize.
>Kids are dumb. Their "complex thought" is really quite simple.
That's a terrible attitude to design a curriculum around. Curricula need to be designed around the concept that every child has an immense capacity for learning and complex thought. Curricula should be designed to reward complex thought. Rewarding rote memorization is not a bad thing, but it is at best a supplement to critical thinking.
Not really, since you replied to:
Bad teachers teaching bad tests is worse than bad teachers left to their own devices.
Bad teachers don't teach. They don't convey knowledge.
That's a terrible attitude
No it isn't, it's reality. Kids are dumb. That's the point of growing up: you become inherently more intelligent and schooling gives you progressive knowledge which is (hopefully) just beyond your full grasp at any given time.
every child has an immense capacity for learning and complex thought
Do you actually know any children? They only have an immense capacity for complex thought if your expectations are low enough.
By the way, I love kids and enjoy interacting with them. But I don't for a second think they're precious, brilliant little snowflakes. They're still dumb. They may be smart among their peers, but they're still dumb on an absolute scale.
Teaching is not a simple matter of shoveling some quantity of "knowledge" into children. Education requires purpose, focus, and motivation. Good teachers inspire children to learn on their own. Bad teachers actively harm students' achievement.
As for my comment on tests, tests that encourage students to write terrible essays loaded with factoids are bad because they encurage students to write terrible essays. This has nothing to do with the teacher - except that if used as a metric for evaluating teachers, it will cause us to devalue teachers who teach students to write better essays.
And I stand by my assertion that no one who calls children dumb should be involved in their education. Motivation is a primary piece of education, and calling students dumb is profoundly demotivating.
I don't know why you're telling me the obvious. Nor do I see what it has to do with the distinction between good and bad teachers. You're making a lot of proclamations without any supporting evidence.
if used as a metric for evaluating teachers, it will cause us to devalue teachers who teach students to write better essays.
Why? What is the basis for that claim? Teaching a testing strategy for a unified test has very little to do with week-to-week teaching and testing. That's sort of the point, it's a standardized test meant to establish baseline comprehension. Why does an essay that hits at least a couple of key points somehow qualify as 'bad'?
And I stand by my assertion that no one who calls children dumb should be involved in their education.
What are you talking about? Who was talking about people involved in education calling children dumb? Who was even talking about calling children dumb? I describe children as dumb, because that's the reality, I don't call them dumb.
Not if learning those factoids allows one to move up the chain, from what is probably a terrible teacher to one that is hopefully, better. Or, those factoids might get you a job where one can begin to learn outside of a scholastic setting which isn't ideal for every personality. I realize it's not ideal. That's why it's such a difficult problem.
I come from a country (and province) with an incredibly powerful teacher's union. So I've seen what happens when teacher's are not held accountable for anything.
>"This test asked for complex thought, the grading should require complex thought"
Do you trust the average teacher to be a good judge of "complex thought"?
Most of these blunt instruments will catch just as many good teachers as bad ones.
In my experience there are precious few teachers who don't teach at all. In fact, I never had a single one that bad. At worst, they hand you the text book and make you read it over the course of the year and give you some tests on it.
The best teachers don't teach much really. Rather they inspire. The worst teachers may teach a lot, but they make you despise the material.
It seems to me that having teachers be forced to teach to standardized tests will kill off the first type of teacher, while having little or no effect on the second kind of teacher.
The worst of all worlds.
Going to my 11th grade Biology class every day, I used to joke with a couple friends "Do you think we'll be learning today?", "Ha ha, Probably not." The class was fun, not very serious, with the bulk of the period spent listening to our teacher tell stories. All that and the homework was always easy.
But at the end of the year, while studying for the final, we realized that we had learned a huge amount of biology. Our teacher had spent the classes telling us stories that related to the material, and the homework was easy because she inspired us to be interested in the questions being asked. I didn't remember toiling over the Krebs[1] and Calvin[2] cycles, but I certainly knew them. More importantly though, I found them interesting.
Bad teachers being held accountable is a good idea. It is such a good idea that bad teachers should lose their jobs. In some cases they should even lose their certification.
I agree, but my goal was more to share what I've seen of the process than to judge it. I think folks here are percipient enough to make those judgments independently. On top of that, I've long been out of the field and am far too ignorant too suggest alternatives, and that's what we seem to need more than critiques.
I've long been out of the field
These observations, though amateur and unscientific, suggest a few solutions. Basically, go back to relative rankings, do a lot less testing, and start rewarding uniqueness. Teachers need to have time to actually teach outside hitting every checkbox on list for the end-of-the-year test. Teachers need to be able to teach to their own students, rather than to the lowest-common-denominator child in rural Alabama. Teachers need to be able to encourange and guide every student individually, developing artists, scientists, thinkers, and doers, rather than having to force every single child into the same standardized mold.
</rant>
If I were to guess, the types of questions you received were different, but the general process of how the tests were made remained the same. And that's the bureaucratic, lowest-common denominator mentality that pervades broadly given tests. And I agree with you that "[t]eachers need to be able to teach to their own students." I think that was a big part of the OP's point since that's something she does.
It's all part of the conundrum: Everyone wants good teachers and accountability in education, but there doesn't appear to be a high-level way of making such assessments that can pass the necessary political muster.
...lowest-common denominator mentality...
After reading your initial comment, I couldn't help noticing that you might have meant "lowest-common-denominator mentality." Consider this pedantism a subtle way of saying, "I read your comment completely; thanks for posting it."
They also help, by identifying bad teachers and bad schools.
Do you have any data suggesting they hurt more than they help?
There is a slowly growing realisation that the result may not be good for the children.
See the Wolf report
https://www.education.gov.uk/publications/standard/publicati...
and also a short OFSTED report into Maths teaching, see the newspaper summary below. The full report may be of interest to you as it explains the pedagogy of mathematics well.
http://www.telegraph.co.uk/education/2982483/Ofsted-testing-...
Teaching is a highly dimensional task. Assuming you could devise a metric space adequate to the task, my teaching at any point would be represented by a set of coordinates. The norm of the coordinates of my 'point' in the space might be higher or lower than the norm of another teacher. How do you decide which one of us is less 'bad'?
Your second link provides no data on whether testing is good for children. It merely shows that actual teaching methods do not conform to what the author's believe are the best teaching methods. No data is provided on student outcomes.
Teaching is a highly dimensional task...How do you decide which one of us is less 'bad'?
Any goal-oriented system is designed to maximize some arbitrary objective function. With standardized tests, you are forced to write down your objective function and admit it's an arbitrary choice.
What benefit do we receive from having an unspecified objective function and no uniform method of measurement?
Link 2: Outcomes are improving year on year, but OFSTED found that understanding of Maths is decreasing. Implication is that outcome measures are not appropriate
"What benefit do we receive from having an unspecified objective function and no uniform method of measurement?"
Children who can think for themselves.
I think we may have an example of paradigm incommensurability here. You are seeing some kind of Goals -> Measuring Instrument -> Optimisation system, I'm seeing a political system with a lot of stakeholders and children that need to learn. As a practitioner, I find it hard to abstract from the daily process of meeting the needs of a very diverse student group. We may be talking past each other.
I must be missing something here.
Children who can think for themselves.
How do you know you get this without uniform tests? And for that matter, what does it even mean?
As for me, I'm well aware of political realities. I just don't see how avoiding careful measurement, clearly stated objects and transparency helps a political system give better results.
"How do you know you get this without uniform tests?"
The health or otherwise of our small companies, and the creativeness of artists, musicians, mathematicians, and the vigorousness of our politics.
"And for that matter, what does it even mean?"
It means what it has always meant! It means young adults who can take responsibility for themselves and others and who can act in society. Seriously, there is a level where one has to simply point rather than define (Wittgensteinish argument).
"I just don't see how avoiding careful measurement, clearly stated objects and transparency helps a political system give better results."
Because there are different stakeholders, and each part of your process will be challenged, and interpreted differently by some, and others will 'collapse' the wider concept of education down to a narrow focus on measurable outcomes.
Not sure if I'm making sense here because I'm in a different place I suspect.
They are very simple metrics which completely disregard test scores, a metric I reject.
Gatto has some good things to say about this. So does Neil Postman, in his book "Technopoly."
How do they differ?
And once you have a definition of "being educated", why not simply measure this? Once you change the tests to a measurement of "being educated", "passing the test" and "being educated" would be identical.
Princeton Review was particular good at teaching metadata identification in helping to "guess" when you didn't know the answer.
This sounds like a job for random.shuffle.
Incidentally, independent (i.e., not funded by Kaplan) investigations of test prep centers show minimal effect on performance.
Both of those are standardized tests. The latter actually tells us whether the student understood the material but the former is much cheaper to grade and report on.
This is a genuine question.
I'm also a bit disappointed about the lack of rigorous evidence base with education. (Well, with everything, really.) In the UK the Department for Families and Schools has a lot of power over education. In theory that should help with an evidence base because they should be flowing good research down through to schools. But there's such a political atmosphere about teaching that interference from government is often seen as unwelcome. And, sometimes, that attitude is correct be government is suggesting something that's stupid.
There's also a problem that bad teachers are difficult to remove from teaching.
Test results are a measurement of how well educated the student is at a point in time. That's all.
You can measure the effect of a school by studying value added - how well a student performs after education compared to how well they performed before. Or similarly, how well a student performs compared to statistically similar students.
You can determine that some subsets of students are harder to teach by correlating observable characters (parental income, race, gender, etc) with outcomes.
How often are they carefully used, with an honest statistician doing the analysis, and without politicians / reporters / etc mangling the numbers?
Or maybe we can accept that nothing is perfect, and stop letting perfection be the enemy of the good.
Since there's very little research on the benefits or disadvantages of constant testing it's surprising that people are happy to spend so much time and money on it.
Even the value add comparisons are hard to draw correct conclusions from without more context: a student shows no improvement in math. Did they have a bad teacher, an attendance problem or was it something like being an ESL student who has never received the help needed to understand their classes? That's not the kind of analysis which happens when the tests are effectively a single number which determines careers.
They're useful for showing students are underperforming. But you can't say anything about teacher quality. In fact I suggest if you took the teachers from the "worst schools" and put them in the "best schools" and vice-versa the results the following year wouldn't look noticeably different.
Does that mean that teacher have no impact at all. I don't believe that. But I do think the large gaps in achievement are systemic problems more than the problem of teacher quality at certain schools.
A simple numerical example, where the correlation is guaranteed (i.e., I took y=x+noise):
http://i.imgur.com/rrmUI.png https://gist.github.com/2183927
Most of the correlations he expects to find are present. They are noisy, but present.
The fact that in one case, reality is "contrary to what every teacher in the world knows" just suggests maybe teachers don't have a great grip on reality.
...if you took the teachers from the "worst schools" and put them in the "best schools" and vice-versa the results the following year wouldn't look noticeably different...Does that mean that teacher have no impact at all?
Not quite, but almost. It means that variation among existing teachers impact less than the size of measurement, and the current crop of teachers are basically interchangeable cogs.
Or that the students - and their parents - are a very important factor with significant variation and classes a relatively small sample size. Ask any teacher and they can tell stories about how different classes can be - and how much a school having a weak discipline policy will ensure that many students will have poorer classroom experiences because of one or two difficult students.
It's not like we don't know how to do scientific measurements of complex systems but it's expensive and slow at a time when the political requirement is fast and cheap. It'd be awesome if school districts actually hired people with backgrounds doing serious statistical analysis or large-scale studies with human subjects but nobody is jumping to fund that.
Aside from the arguments posted in sibling comments about what's being measured, you also have a problem of unaligned incentives. In particular, your claim only has a chance of being true if the students are actually interested in scoring as high as they can on the exam. Even at the AP level (I teach at university level and interact with secondary teachers that teach smart, high-level students taking CS), there are students who have decided that they don't care about the subject, or maybe even like the subject but have no (perceived) benefit from a high score due to their chosen college not giving credit in that subject or whatever. Such students may leave their answer book blank, or doodle in it, or maybe just blast through for the easy points and finish early and not worry about thinking about it.
This isn't even necessarily a particularly irrational choice on their part!
But it's a strong argument why the exams shouldn't be used to evaluate the teacher or the school. In a lot of places the students aren't permitted to opt out of the exam, even if they don't care about it, but there's no penalty to the student for taking a dive on it (and any penalty you could try to assess would have false positives and false negatives and still not motivate many of the students with differently-aligned incentives).
All of these problems are going to be a million times worse on a general-education primary- or secondary-level assessment than they are on AP exams.
It's similarly difficult to prove that clinical trials are beneficial in medicine. You might try to compare medicine developed with clinical trials to medicine developed without it, but how would you actually make such a comparison? Not with a clinical trial, obviously...
The idea of tests sounds good, but the implementation is so bad they are effectively worthless. Again, you can argue that great tests could mean something but we don't have them we just have crap. So, if you want to defend tests you need to defend that crap because that's the reality.
GPA is closer to a useful metric. GPA and college admittance are basically the same thing as standardized tests. Except that unlike standardized tests, you can't compare any set of grades to any other set of grades. How does that help?
Incidentally, do you realize your biggest criticism of standardized tests is that they are not standardized enough? I.e., there is too much geographic and spatial variation in them?
You can also track where a school systems switch to abstinence only education and the rate increases. Now granted it's not supposed to be the most useful thing teenagers get from public schools, but it is a vary important part of their job and we have fairly good data on it as well.
Not really. Here in Chicago, the Noble charter high schools will often teach directly to the ACT. This results in some dramatic boosting of ACT scores, with some schools taking kids from the dysfunctional CPS System and getting a school average of 23. Unfortunately, once they reach college, they see little academic success despite being straight-A students with 25+ ACT scores.
This has a detrimental impact on the curriculum:
1) The English program structured heavily around basic reading comprehension, with little to no emphasis on writing composition. A students understanding of essay composition s roughly: "Organize the things you want to talk about into paragraphs... then write a conclusion." However, to their credit, they're really good at reading test questions.
2) Math is focused on teaching Pre-algebra fundamentals and then layering on test-specific Algebra, Geometric (with that goofy proof system), and basic Trig. It's a sad, narrow sample of our already sad & narrow HS math curriculum. It covers few "advanced" Algebra and Trig subjects. This means anyone who has to take "college math" will need courses in Trig and Precalc in college, with the possibility of an Algebra refresher course before proceeding.
3) Social Studies and Science? All rote-to-test. Students are drilled on step-by-step procedures on how to interpret graphs that'll score correctly on tests... without giving them actual knowledge on how to critically think about information - be it historical or scientific.
NCLB schools do the same song-and-dance, except with a much less rigorous test. If you've actually seen the questions on most NCLB tests, you'd be disappointed. Unfortunately, the composition of such tests is so political and messy it's impossible to provide any measure of quality.
You cannot assume testing will provide you accurate information or a better outcome for students. If you're going to implement a testing regiment, you need to be very mindful of the Observer Effect: You can very easily change the outcome by measuring it. This is not an easy problem to solve.
Similarly, if your manager writes a bunch of nonsensical unit tests, it's ridiculous to blame unit testing if you wind up building the wrong product.
Why do you feel that not holding teachers accountable for meeting the objectives of their school system will improve education?
I agree that some schools may have bad goals, but why do you believe teachers/admins have better ones?
The second two sentences have nothing to do with anything I said.