Hacker News Reader

Comment by mjburgess | Hacker News Reader

> it's hard not to be sceptical that this wasn't intentional and class-motivated.

It's incredibly easy "not to be sceptical".

The teacher's predicted grades are, on average, unprecedented. And represent many times greater than any previous yoy increase. This is far less true at top-performing schools where teachers are predicting far more consistent grades (by the very nature of being a top performing school).

Teachers shouldn't have been asked to predict grades. That's the heart of the problem. Not some harebrained in-public conspiracy.

This whole thing is dumb. And the media are deliberately failing to explain basic facts about the education system to provoke this kind of nonesense outrage.

View on HN

I completely agree that the system put in place is poor and that it causing an accidental bias is very feasible.

But if you're going to put this system in place, with months of planning, you should make damn sure that it isn't massively favouring the children who are already advantaged in life. Which it statistically is, whether intentional or not.

They screwed it up so spectacularly that the education secretary had to make concessions to it two days before the results, then a day after the results Ofqual made a statement about who could appeal that contradicted the education secretary, before withdrawing that statement a few hours later saying they were still deciding.

So sure it could be entirely down to incompetence not malice. But given either is possible, and this government's track record of cronyism I don't see why it's unreasonable to be sceptical.

It doesnt "favour" any children.

Those children were already going to get the highest results.

There is no incompetence here. The predicted grades are a good estimate of what the results would have been.

The issue is that all teachers significantly over-predicted their students grades, but less so at high-performing schools. (Why? because the grades are more consistent at those schools, so there's less room for error).

Independant schools (private schools) received 4.7% more A/A* grades than they did in 2019. The most common type of non-private school, Secondary Comprehensive, received only 2% more A/A* grades than in 2019.

The algorithm didn't keep the gap between paid and unpaid education, it expanded it.

(Source for these numbers and more in my first comment.)

I was lucky enough to not only go to two private schools, but the second one was ranked #1 in England for exam ranks 2 of the 3 years I was there (and helped me get good grades despite being extremely unmotivated and lazy). But at least my privilege was to be given a good education that made learning / passing exams (the latter more) easy, not that the government decided my school deserved an artificial grades boost compared to state schools.

Right, and what was the previous yoy disparity?

If the previous YoY was also 5%/2%, then it is just repeating the trend.

It isnt the job of statisticians to "fix" social inequality by BSing some grades. Its their job to estimate likely exam results based on previous performance.

There's absolutely no grounds on which to assume that growth in A*/A across these demographics would be the same; and of course, very many reasons to presume they'd be different.

I can't find that data easily, if I'm bored tomorrow I might try to do the research myself - all the data is available but raw not top level overview at https://www.compare-school-performance.service.gov.uk/downlo...

It's only been a couple of days, we'll get more data journalists breaking it down better soon.

You know what would be really helpful? If the algorithm was released to be publicly reviewed. Oh, but the government is refusing to do that, and Ofqual made anyone consulting on this project sign a 5 year NDA or they couldn't get involved. Hmm.

Edit: and while I'm pretty sure the data will show schools weren't on track for that inflation, actually no I disagree with you that it would be fair to extrapolate previous cohort improvements to this year's anyway.

I agree. It should be published.

However more important than the algorithm is just the media reporting on what ofqual actually say. I've barely read any of their actual analysis in news on the issue.

Key facts: students are always ranked and graded on a curve. Student populations have a fixed number of grades awarded every year, etc.

The system is specifically designed in recognition that grades are positional goods, ie., they are valuable only to the degree that other people do not have them.

That's the view of the education system every year. And its the basic assumption of this process. They will never over-award A grades specifically because they want universities/business/etc. to index them to a percentile performance in the population.

> Key facts: students are always ranked and graded on a curve. Student populations have a fixed number of grades awarded every year, etc.

I'm pretty sure that's not correct, please find a source suggesting that. There have been major reforms (primarily the changing A Levels back from the modular version that Labour had previously introduced) and these were specifically aimed at reducing grade inflation. If there were a fixed number of grades each year there wouldn't be any inflation to counter.

Starting with the 2010 introduction of the A* grade, the proportion of grades awarded has been almost constant (25% A/A*, 25% B, 25% C). This is in contrast to the consistent 'grade inflation' visible for the previous twenty years. https://en.wikipedia.org/wiki/Grade_inflation#UK_A-Level_cla...

Isn't that due to the attempted curbing of grade inflation working well? As far as I know they haven't returned to actually grading on a curve ranking like they did decades ago, in theory if next year every student somehow handed in the perfect papers they'd all get top scores.

>Student populations have a fixed number of grades awarded every year, etc.

Yes, for the national cohort. This is different, the predicted results were weighted on a per-school basis. For example, a particular child taking, say, a history exam, in a school which had previously had no pupils that achieved A* in history over the past four years had probably no chance of getting an A*, even if that was their teachers' prediction. Purely on the basis of which school they went to. A weak cohort (compared to previous years) at a good school would be upweighted.

Previous YoY was -1.9%/-1.5%.

How could +5% possibly be an actual trend? What new magic teaching innovation have independent schools done for this year's cohort, that wasn't available in previous years or to teachers in the state sector?

Table 9-10: https://assets.publishing.service.gov.uk/government/uploads/...

You're making very strong statements over a huge assumption that you have no data to back up. This is not like previous years. The students sat no exam, they were ranked within their school, and then assigned grades based on the previous performance of their school. It is reverse affirmative action.

What are the strong statements you think he’s making?

They sat for mocks.

  > The issue is that all teachers significantly
  > over-predicted their students grades,
  > but less so at high-performing schools.

"All teachers" is a sweeping statement.

There are teachers in comprehensive schools that are tough-markers, too. My wife, who is a teacher in a London school, was annoyed in both directions - there were teachers that she felt had given away high grades, but there was also a teacher that had given the kids in her class grades lower than she expected they'd get (as her identity is tied to being overly careful and she liked to grade kids harder in mocks, etc).

What she felt was bad was that cohorts of kids were often very different year upon year. Last year there were particularly bright kids doing Physics in her school, but this year there was only a few kids and they were much worse. But the algorithm awarded them grades above what she expected.

There were also kids that she expects would have worked hard before their exams and gotten results above what they were given. They didn't get given the chance to do this.

I meant all teachers across school types, not all teachers individually.

In both state and private schools, teachers over-predicted. But higher-performing schools have higher and more uniform results, so there's less room to actually over predict -- so there's comparatively less reduction.

That's just an in-built concequence of what it means to be a high performing school

Sure. It makes sense.

I don't believe in the media conspiracy that the Tories created an algorithm that systematically awards higher grades to rich kids because of inherent classism or even by accident.

But I do really disagree with the insistence that all teachers in comprehensive schools were conspiring to give their kids higher grades than they deserved -- this view was frequently repeated on social media. Even if there were some that were doing this and it was apparent from the statistics, teachers are not a homogenous group and there are many teachers that take fair grading very seriously and even take enjoyment in being tough markers.

I don't know the data but my own worry is that this will have hurt the high variance or lazy kids that won't normally get high grades but would normally settle down and revise hard for such an important exam. I hope that these kids will be given the opportunity to resit the exam for free, since I doubt an algorithm could capture the variance that exists every year.

Just to be clear about what actually happened, the results were weighted on a per-school basis, based on results from the previous four years. The disparity in weighting is not solely down to effects at the top of the distribution.

https://thaines.com/post/alevels2020

I just read this article, and it was interesting.

> but less so at high-performing schools.

The bit you haven't understood is that teacher predicted grades were reassessed by the algorithm unless the class sizes were 5 children or fewer. This meant that the predicted grades that were not adjusted by the algo were overwhelmingly at independent fee-paying schools.

Independent schools got higher grades because of a quirk in the algorithm. If there weren't enough students taking a particular course at a particular centre for the rank/previous results method to be reasonably accurate, the algorithm defaulted to partially or wholly relying on predicted grades.

Independent schools generally have fewer students and a wider range of courses and predicted grades were inflated across the board, so they ended up with better grades than larger colleges and academies.

I'm not sure how you could have avoided this bias without creating a lot of really weird outcomes.

Well, wealthy students who opt out of government secondary schools could be given lowest priority at government colleges, and if that offends someone they can expand government funded colleges so that all could attend.

>if that offends someone they can expand government funded colleges so that all could attend.

Universities in England are funded primarily by tuition fees. There are consistently more university places than applicants, but not necessarily at universities you'd want to attend.

You are now about to witness the extent of my oxbridge[1] knowledge: https://www.youtube.com/watch?v=McMSHqWM3G8

[1] to understand this locution it may help to understand that there is also a breakaway technical college somewhere off in the fens...

You could have looked at past performance of those schools.

https://twitter.com/queenofswords6/status/129447148835198566...

> A levels...short thread

> If anyone doubts who the winners are this year, here's a random selection of independent schools and the % of A*-A grades this year.

> 2019 figures follow in brackets.

  Mayfield Girls'     66% (55%)
  Sherfield School    70% (37%)
  Leweston School     51% (37%)
  Farlington School   62% (48%)
  The Marist School   55% (52%)
  Godolphin School    50% (29.2%)
  St Mary's Cambridge 54% (51%)
  St George's Ascot   63% (35%)
  Rye St Anthony      48.1% (18.3%)

It's not obvious that predicted grades are categorically worse than the system they put in place though, is it? In some aggregate statistics we can see them perform worse but exams and grading aren't about averages at school level but individual achievement. A prediction systems that doesn't take students prior achievements into account is just obviously defective here. And that matters especially if it locks out a whole year of students from the top universities that are so life/career deciding in the UK, because getting a place at them by definition requires being an outlier.

The social and cultural factors at play are not well represented by just looking at the mean.

And finally, so what if 2020 had had outlier grade average? It's the outlier year.

> A prediction systems that doesn't take students prior achievements into account is just obviously defective here.

Doesn't it? What else would how students have been ranked in ability mean?

> And finally, so what if 2020 had had outlier grade average? It's the outlier year.

"Hey, we'll give everyone 40% better grades, because it was a tough year" won't do good. You'll get lots of people into university who will fail miserably, because the grades don't magically improve their skills. Unless, and that's a bad idea as well, you just tighten the criteria so the same number as before can go to uni. Grade inflation won't help anyone either.

We're going to have a demonstration of what happens if everyone just gets the predicted grades and 2020 becomes an outlier year, because the government did a u-turn and it happened. So far, top-tier universities don't have the space to admit everyone who meets their offer requirements, the lower-tier universities are likely to not have enough students to remain financially viable due to everyone going to the top-tier ones, it's likely that the former will be ameliorated somewhat by removing the cap on admissions but this will make the latter worse, and it's all a huge mess. Every part of this was predictable, but none of it was explained by the press - which had been pushing the idea that everyone should be able to get their predicted grades - until it actually happened, because they were more interested in blaming the Government than accurately describing the pros and cons of the different options. Now the BBC is pushing for an inquiry as their headline front-page story, as though there was some kind of secret malfeasance to be uncovered rather than a set of bad options with really obvious, unavoidable tradeoffs: https://www.bbc.co.uk/news/uk-53826305

What doesn't quite make sense, is the fact that these students sat 'mock' examinations earlier in the year. Presumably, these 'mock' exams are based on an empirical standard, similar to the one used for the 'final' exam. So the first question is, why weren't these 'mock' results the main anchor for these derived results?

The problem with mock exams is that they don't follow any national rules on how they're used. Some schools do them in November, some in March. Some teachers mark them having not been trained in A Level marking. Some schools may do mocks in exact exam conditions, others might not bother with one part of the test, or have them done without supervision. Some teachers grade them accurately, some intentionally grade them harshly to motivate their students to study. Etc.

I suppose teachers never had a duty to communicate that to the students before, but that's going to be a pretty important detail in the coming weeks.

I doubt many parents would have been made aware of the precise meaning of 'mock' examinations results either.

The next question then, was whether this emergency scheme was tested against results from previous years, rather than fashioning the algorithm to match results from previous school years.

Given the extent of some candidates differences and the impact to their life (denied entrance to offered Uni courses for that year) there’s some very serious outrage. If you were part of good class in a school with a poor record, then you’re stuffed. It’s a unprecedented situation and the only guide that you have are teacher grades.

>Given the extent of some candidates differences and the impact to their life (denied entrance to offered Uni courses for that year) there’s some very serious outrage.

It's not as bad as you might think. A large number of students are rejecting their offers with the intention of re-applying in 2021 and there has been a substantial reduction in the number of foreign students, so there are an unprecedented number of surplus places available through clearing. Even if your grades are substantially worse than you might have hoped, your chances of getting a place on a decent course are very good.

Doesn't that make next year even worse, as we'll have 1.5x applicants for the same seats?

That depends very much on whether foreign student enrolment rebounds.

I went to a private school on Australia, which is also pretty common.

What you realize is that the school's primary deliverable is acceptance into a university program.

It wouldn't surprise me at all if the private schools were actively lobbying to deliver the best result to their customers.

Teachers predict grades all the time. It's part of how student progress is tracked. We are constantly being told what level our kids are at and predicted level at exam time.

Part of the problem is that teachers often push the "prediction" slightly higher as a means of motivating the student. Now those predictions are being used as the real deal.

I am pretty convinced that most teachers with more than superficial experience will be able to predict grades reasonably accurately, after doing so for many tens of students 5-6 times a year .

Well as a point of fact, they didnt. Their requested, "accurate" grades represent a 20% increase in general attainment. That has never happened in the history of modern education, and occurs only when, eg., civil wars end.

As a comparative reference, we'd expect an at-most, 3% increase in IQ in the general population.

I have no problem with the idea of down-weighting the entire national cohort to correct for this. The problem is that they actually weighted the results on a per-school basis.

Even down-weighting the entire national cohort... some kids have seen grades go from C (or B/C in one case) to U! A fail grade for an exam they weren't allowed to take.

That would be an improvement if your luck of the draw was national rather than biased in favour of private schools, but it's still a pretty awful outcome.

There may not have been any solution available that would be 100% fair to both this year and other year cohorts, but at the very least they should avoid making any student feel actively screwed over.

This is desirable and it's easy to explain why. For any student there will be a range of grades they could expect to achieve. The fairest grade to assign is the best case grade in that range (not the midpoint), because there is no evidence that they would not have studied hard and achieved that grade in the exam. If every student is given their best case grade then you will indeed see an overall rise, this simply reflects a lack of knowledge of who would have studied hard and who would not have.

That's only fair in an environment where education is not a scarce good. But college education needs some way to artificially pick winners because the schools are underfunded and then economy is classist.

Nonsense. There has always been a divide in the UK between public schools that try and predict grades accurately and private schools that game the system to maximise their students' chances of acceptance.

I don't think teachers, or anyone else, can reliably predict results. I got the equivalent of an A* in most of my of my AS exams, but because I had bad GCSEs my results were ranked down for my predicted grades to AAB. Only after complaining did they change them to AAA. In the event I got two A*'s and an A - a whole grade above all of my original predictions.

The entire system is asinine because there's no reliable way of predicting future results from past performance, because of the unfair discretion it gives teachers and schools, and because it demotivates students to achieve above their prediction (because they cannot be accepted by any course above that prediction). It should have been abolished long ago.

The teachers don't play a part in downgrading based on past results, only on grade prediction based on current performance.

I'm glad you got the grades you wanted eventually, but the finger needs to be pointed in the correct direction.

Reply on news.ycombinator.com