Hacker News Reader

> It doesn't really matter how good the model is, it's an impossible task politically. There will always be some people aggrieved by the new process and they will always have plausible ways to spin it as racist or classist or whatever else is necessary to fuel the outrage machine. As if the previous setup was some kind of examplar of egalitarianism.

So just because every system has flaws, we should throw in the towel and give up? Taking that line of thinking all the way means we can just replace university allocations with a lottery.

That's not how society works.

Even if every system have flaws, even if the current suffers from racism and classism, we have to strive towards making it better. The system the UK invented here does not even try. It very clearly favors people with nice postcodes.

Maybe we should replace it with a lottery, it would be equal. But I'm pretty sure the people with nice postcodes would like it much, and that's the people propping up the UK gov. Etonians, Oxford, silver spoon etc.

I for one believe there must be _some way_ to fix the situation and is better than what they came up with.

It's clear there's some abuse of power going on here. It doesn't sound like an entirely objective process. That the algorithm has been selectively applied, is definitely worth an independent review and some time in court.

That argument can be made without drawing emotional prejudices into it, but it wouldn't hurt to raise the question of prejudice while conducting this debate, because there has been evidence of a class based skew in the UK education system for years now.

And this appears to be the most overt attempt to rig the system yet seen.

> That argument can be made without drawing emotional prejudices into it, but it wouldn't hurt to raise the question of prejudice while conducting this debate,

There's a public sector equality duty under the Equality Act 2010 which is one reason people are looking at the disproportionate impact on some groups.

> both high- and low-achieving students were celebrating when the standardised exams got cancelled here in Ireland

They celebrated the message "Exams are cancelled"

If the message had instead been "Exams are cancelled, at this school A grade students will receive a C instead" I suspect you'd have seen a very different response.

Well, yeah, but they'd really be thrilled at the school where they announced C grade students would get an A.

There is some benefit to society to matching up the right students to the right courses, so it's not completely zero-sum. But the cost of getting it a bit wrong isn't terrible: some marginal students get to follow their dreams and study medicine, some others miss out on medicine and have to try their hand at accountancy.

This discounts the sense of disenfranchisement that may result, which might be consequential, or that we may end up with inferior doctors.

I would expect moderation to lower grades much more often than it would raise grades.

After all, it's understandable why a teacher would over-estimate grades (assumption everyone else is going to do the same, and their kids will be relatively disadvantaged if they don't) and hence get moderated down.

And it's understandable why a teacher would give spot-on estimated grades (honesty, valuing the credibility of their estimates) and hence not get moderated at all.

But it's difficult to see why a teacher would under-estimate grades.

So I suspect there are not many C grade students getting upgraded to an A.

There is good evidence that comprehensive school pupils were disadvantaged and independent school pupils were advantaged, when you dig beneath the overall averages.

Yeah, they decided to have a magic number of 6 when they started standardising. Given that lots of private (nee public) schools have smaller class sizes, this made the bias worse.

I believe it is 10-15 actually. If the class is more than 15 people then it uses historical grades. If it is less than 10 it just uses the prediction (which is obviously generous). In-between they are weighted.

A number of reports suggest that the algorithm has (and will be, in the case of the coming GCSE results) been selectively applied too. So your academy's history will only be taken into account, once it's determined your school is of a certain size.

Which most state run schools are, of course.

This is a nightmare, really. Like, no one can directly prove the malice here, but it certainly feels like there was an aspect of malevolence in the enterprise.

Who would stand to gain from that and what would they stand to gain? UK education policy over the last couple of decades has been squarely set on getting as many state school students to university as possible so it would be strange for them to suddenly switch track on the sly.

I'm no fan of the UK government but I'm pretty sure this is a textbook example of Hanlon's razor

> getting as many state school students to university

They want the state students to get to universities yes, but they want the private students to get to the good universities (oxbridge).

It would keep exclusive institutions exclusive.

Much of the UK educational system is like that. For example, I don't know how it works elsewhere, but in England, certain students will only be offered certain exam questions, in the same subject, if the teachers determine you can take them.

So immediately, you can see some aspect of these grades are based on a subjective standard. In this case, at the discretion of teachers. One could easily argue that this Covid exam fiasco is a supercharged manifestation of those very same principles.

In the above case, obviously, all students sitting the same exam, for the same subject, should see the same exam papers.

I mean, why would they not?

I suspect the obvious answer would be the cost of teaching resource. If that's the case, this then becomes a political argument, not a technical one. But the 'exam algo' debate is definitely a technical one, and it can and should be used as a lens to examine (pun) a number of questionable practices in the UK education system.

When I took exams in England (admittedly a decade and a half ago) there were indeed different questions based on what you had learned, but you were instructed to only answer one of them (if you answered more it didn't count), and they were all worth the exact same number of points. It's possible some schools taught two texts to let their students choose whatever was easier on the day, but I doubt this was widespread and I don't see this system as being unfair.

It does not need to be malice.

Good independent schools (and grammar schools) have very consistent results. Often 70%+ of A/A* year after year.

If previous grades and predictions say that most pupils should get A/A* that's not surprising. That's the way it is every year, and that probably does not indicate an inflation because of this year's strange circumstances.

But for a comprehensive school where results can vary much more from pupil to pupil, and for which past data are a mixed bag, it is trickier.

If some students would prefer to take exams and some not, it seems obvious to let only those students who want take the exams. Propose the model’s grades to the student. If they accept it, you’re done, if they refuse then let them do an exam. Likely 80% or more of students would accept the proposed grades and it would make it doable to organize socially distanced exams for the rest.

Exams are already "social distanced". That was never an issue! Has everybody forgotten what exams are like?

It wasn't just about the exams; students missed out on months of teaching, which would have put them at a huge disadvantage if they were then expected to sit the originally-planned exams.

So we'd need exams tailored to just the part of the curriculum that the students have actually covered -- which would be different for each school, making any kind of standardisation impossible.

Another thing people don't seem to be considering is the knock-on effect on universities, which -- if students who didn't complete the A-level course are given a hypothetical grade, whether based on teachers' predictions or modified exams or "mocks" or whatever -- will get an intake of students who are less well prepared than normal, and will need some kind of remedial support if they're not going to flounder in their degree courses.

Just giving students generous grades now, to calm the current storm, means storing up trouble for the future when university teaching staff are faced with a cohort that includes students who shouldn't be there and can't cope -- but their teachers gave over-generous estimates -- or who in principle should be OK but arrived woefully under-prepared because they missed a bunch of key A-level content.

I agree with most of the points you're making, but I do not think that the exams would need to be tailored to just the part of the curriculum that the students have actually covered.

If one student does not know key parts of the curriculum and the other does, then this should be reflected in the grade no matter what caused that difference, whether one of them got more schooling or studied on their own or took some out-of-school tutoring.

And if a diligent student who got all A's before missed out on key parts of the curriculum that's required for further study, then the final grade should reflect that they do not (yet) know the subject as expected and are going to need remedial support, i.e. their final grade should not be an A, the exam should ask for the expected content (no matter if it was not covered in school due to Covid) and the exam results and grade should illustrate whether the students have learned it otherwise or if there's a gap.

I'm inclined to agree, in principle, but this would mean that many students would end up with a lower grade than they could expect to have achieved normally. This would particularly apply to less-advantaged students whose schools and/or families have been less able to fill in the gap.

While that'd result in grades that would be far more meaningful than any of what's currently being done -- and universities, employers, etc would just have to take account of the special circumstances surrounding 2020 results when making admission/employment decisions -- I don't think it's politically tenable.

There are exams taking place in the Autumn, so this is kind of what is happening.

That’s too late for university admissions this cycle, and next year’s Oxbridge admissions cycle will be a nightmare given the large number of reapplicants. It’d have been necessary to use the time saved in not having to mark exams to run the algorithm early.

They will have missed this cycle of admissions, and not all students have the luxury of taking a gap year. The students have also missed a term of tuition, and so have probably not covered the course material necessary to pass or do well in autumn.

Children/adults in care (people without families that can support them) might not be eligible for support for another year, and will often be expected to go out and work to support themselves. Poor families often rely on the extra financial support that ends with "finishing" education.

There are a lot of stories out there of children/young adults in exactly this situtation that should be off to skilled work training or university that might end up having their entire education wrecked over this despite being able academically. And this is all with the context that private school students often have not been subject to any "standardisation" process due to their small class sizes, or have had their awards upgraded above predicted grades.

We just need to give this year a free pass in much the same way as workers have been given a free pass with the furlough scheme. We can live with one year of inflated results.

Its a particularly bad time for students not being able to take a gap year as well - traveling is pretty much of the cards for those who would have otherwise have done that, and for those who would have spent the year working they're being thrown into that amongst the highest unemployment rates in decades, and up against people who have extensive experience doing the sort of jobs they'd be applying for.

Students will only opt to take the exam if they think that doing so will improve their grade. The end result would be grade inflation.

An update: they're now going with a scheme whereby students get to keep the better of the two grades given by two different methods, with teachers' predictions on the one hand, and the 'standardisation' system on the other.

Presumably this will lead to grade inflation.

https://www.bbc.co.uk/news/uk-53810655

See also Wales seeming being the first to switch to this approach: https://www.bbc.co.uk/news/uk-wales-53807854

Yes, it will lead to grade inflation. The important question is: does that matter? Exams already have their flaws such as being far more likely to give a flawed bad grade to someone than a flawed good grade. How much grade inflation would you see purely from eliminating that effect? And that would largely be desirable.

Well, Covid definitely meant some drastic action had to be taken, but when the UK government originally discussed suspending high school 'finals' for all students, they strongly suggested the derived grades would be based on mock exams, and teachers predictions / appraisals. Which would have at least localised the appeal system.

Then any regulatory board could have worked 'backwards', so to speak... Only look at schools where there are blatant discrepancies across aggregated results.

There was never any mention of the application of an opaque post-process. You would think that would be the kind of detail you would have to discuss in a national assembly before the state commissions it's implementation, but clearly, it was never raised.

So while I understand your point (accusations of 'classism' are premature), I think the fact that we're only learning the details of this system now, is an outrageous abuse of trust and technology.

"okay, lets do it, refuse all evaluation of our work and just push something through" is not the appropriate response to "this task might be impossible". It being "zero-sum" in some sense is also an extremely low bar we don't generally accept as sufficient.

They would have been better off politically just giving them all their predicted grades and letting the universities sort out the mess. I think that's a bit cowardly but it would have taken the heat off.

Concerns over long-term grade inflation are totally overblown anyway since the majority of these exam results get used that year or the next for university admission and only for that. What does it matter whether my grades from 20 years ago are comparable with grades now? It's not like we'd be competing for the same university places.

The only problem with accepting the CAGs is that some centres will have been laxer with them but that could easily have been sorted by using similar techniques to the ones they used here to look for way out-there schools that clearly aren't taking the process seriously.

> There will always be some people aggrieved by the new process and they will always have plausible ways to spin it as racist or classist or whatever else is necessary

But if you have a transparent process and documented algorithms, it's much, much harder for them to do that. Of course, UKGOV has neither of those things in this instance...