A-Levels: The Model is not the Student
thaines.com
thaines.com
The entire entrance process pretty much uses grades as the first filter - changing that would be a huge amount of work not to mention a lot of people will have conditional offers that are expressed in terms of these grades.
So picking a course at random (Physics at Edinburgh) requires: SQA Highers: AAAA - AAAB or A Levels: AAA - ABB
https://www.ed.ac.uk/studying/undergraduate/degrees/index.ph...
However, the governing principal of Cambridge University, for example, is that all operations are done at a human scale, and I think when you factor in the large numbers of staff whose labour the Universities have access to, it becomes possible.
For a group of applicants for Computer Science at a Cambridge college, for example, you would have a Director of Studies and one or two other staff ranking around 20 applicants for half as many places. (The ratio of applicants to places is much higher prior to interview.) It is a tractable problem to solve, even if for other institutions the process is scaled by 2x/3x (4:1 / 6:1, up to Nottingham’s reputed 13:1) or maybe even more.
An idea we mooted over dinner last night was for these applicant rankings to be fed to UCAS — the University and Colleges Admissions Service — and for UCAS to combine these rankings with the applicant preferences to fill up each course from the top down. No need for grades, just use the desirability metric provided by the institutions.
Grades themselves are a strange beast. By their nature they are different to raw marks. A grade speaks to your ability as part of the entire cohort of examined pupils in that subject. Without the exam, taken nationally under examination conditions, it just doesn’t seem to make sense to refer to grades.
If we called them “exam grades” perhaps the folly of trying to assign them to pupils, who have not actually taken an exam, would be clearer.
(I speak with no real authority, but I am a high school teacher in England.)
There's still a substantial bias in offers towards public schools, but a lot of that is because such schools are selective like the universities, actual student ability does still matter and better schools still improve student ability more, and there's a strong self-selection of candidates for top universities (a lot of people who might get accepted for oxbridge don't apply because they don't think they have a chance) as opposed to old-school elitism in the application process. (Not that all of the above factors are actually fair from a grand socioeconomic viewpoint, but they are reasonably fair in as much as the university judging the merits of the applicants it gets).
It's also a single grade per subject, so the university /employer will usually have 3-5 grades to work with, as well as the broader picture built up from the personal statement/cover letter and interview.
You're right that grades are reductive, but the alterntive isn't more of the same information.
I watched the news broadcasts when the results were announced. There were TV journalists at schools in deprived areas interviewing students before the grades were released, with the journalists giving dour warnings about the impending problems they expected. When the results were announced the students were leaping about with joy, hugging and celebrating, it was like a world cup win. The journalists didn't seem to know what to say. back in the studio the best the presenter could do was mutter that "I'm sure we'll find some students that were disappointed later".
I'm sure they did.
Fundamentally it's a hard problem: the admissions require more information than is available. But...
> All this fiasco does is demonstrate how unnecessarily reductive the whole concept is.
...there never really was enough information. This is an important factor IMO. The system has always been unfair, now it's less fair and in a colder, more obvious , more systematic way that resonates with the current political climate around technology. Politically it's hard for this government to get away with standing by a policy that adversely affects poor students in the same way it would have been hard for a left-wing government to institute the current fiscal policy.
>kids
Nit: almost all of them are now legally adults. At university they can look forward to this twilight age where they are treated as an adult when it's about financial obligations and treated as a child when it's their institutions forcing policy upon them.
But much middle class [1] and some working class Tory support still believes in the social status dream, and this fiasco has directly undermined that narrative.
The party is getting huge heat from its base over this, not just because of their future prospects, but because some of them are now wondering if "Work hard and get ahead" has ever been true for them.
[1] Note for the US - in the UK "middle class" specifically means "Highly educated professionals with significant cultural and political capital" not just "People with a degree making decent money."
But it's not as simple as that, either. Coincidentally, I just read an obituary of John Hume, who was born into poverty in Derry and rose -- through education and hard work -- to become a leading politician and a Nobel prize winner.
https://www.irishtimes.com/life-and-style/people/john-hume-o...
> It's an outright lie, because the reality has always been "Go to the right school and have the right parents and get ahead."
There have been and will always be exceptions, cherry picking a few of these is a form of straw man.
The current system isn't coping well with COVID-19 but let's not throw the baby out with the bathwater here.
The current system is simple, and that means it can be seen to be fair. If studying X at university Y needs grades AAB that's easy to understand and you know it's not biased.
I much prefer that to the American system where every university has an opaque system that purports to look at a complete picture of the student then mysteriously picks white students over Asian students with better grades and can't explain why.
If only it were that simple. Universities often also take in to account the outcome of in-person interviews which are inherantly biased.
Some courses like Medicine do have interviews at institutions that usually don't interview.
I think it's a shame this was the exception rather than the norm.
It seems clear that if you just take a distribution from previous years and lay them on top of a ranking, you are creating a lot of problems. And you are hiding your incentives on the penalty function, which is something he mentioned. People have correctly figured out that a good kid at a bad school will not be found, and that the marginally second best kid in a year is going to get marked down.
Whatever you think of exams, and I'm no fan, they do give people that mix of noise and signal that society needs to put to rest the issue of who deserves what. They're not entirely a lottery, not entirely skill or hard work, but if we need a way to distribute X number of future doctors, that's the long established way to do it.
They wouldn't sign a five-year NDA:
https://news.sky.com/story/a-levels-exam-regulator-ignored-e...
The nicest interpretation of this is that Ofqual had no idea what the hell they were doing.
However, what they opted for was the same thing, but without the second step.
Why do we need the second set of testing at all if the predictions exist and if the marking is going to be judged against them to the extent we are trusting the prediction over the marking.
Because, fundamentally, the current UK government is not interested in scientific evidence or taking advice from experts in an area (c.f. Michael Gove's famous line "people of this country have had enough of experts")
"I think the people of this country have had enough of experts with organisations with acronyms saying that they know what is best and getting it consistently wrong."
He was using the example - nearly ten years on - of how "experts" didn't predict the 2008 crash. At best, a straw man argument (which experts; were any expert advisors aware and did they endorse the extreme lending that affected the UK, etc.)
The government is on record as wanting precisely more technically qualified people to enter the civil service, and restructuring the service toward this end. Whether or not the way they intend to go about it is sensible or reasonable is another topic for debate.
A very substantial number of the people who work at Ofqual are statisticians of course, it just seems that they have made some serious mistakes.
I don't understand why they couldn't have published at least the algorithm for peer review and discussion months ago. These issues could have been resolved with plenty of time left before results had to be out.
Another problem with this algorithm is that every year, a certain number of exam takers just fuck it up on an epic scale. They didn't get any sleep, they get a question that completely flummoxes them, whatever. So the distribution of grades, even at a good school with a pretty good intake, will have some very low grades on them. A big school will have a U or so. Now come to this year - noone had the opportunity to mess up their actual exam so some poor probably-C, B-on-a-good-day kid has to be stuffed into the "U" to make the distribution fit. Purely on the basis that the big school they went to usually has at least one. The difference is, if you fuck the exam up, you come out knowing that! So when you get your U, you know how it happened. This year someone has had it assigned to them without actually screwing up the exam which seems rather harsh.
It's worth noting that the predictions that were initially planned to be used as part of the algorithm would have resulted in 38% higher grades than any other year, ever. It's clear that teacher predictions are ALSO "wrong".
https://twitter.com/lewis_goodall/status/1295069110829752323...
And this describes the advantages that private schools got: https://twitter.com/hugorifkind/status/1295292925463724032?s...
I think it should be possible to create ‘virtual classes’ above the threshold by combining smaller like classes together.
Such a model would be fairly ham fisted.
> but I think the bigger issue is
The tricky thing about this scandal is that it is actually about 8 different catastrophes for students all enmeshed together. This makes it hard to know where to start critisising the government.
Were you predicted worse grades than you achieved, and as a result not offered places where you would have liked?
( * they may have, significantly, it's been a long time)
Motivation for working hard during 6th year can be difficult if you get an unconditional offer based on results you already have....
I effectively got an unconditional offer in England way back when, was predicted something like 3 As and a B, and was given an offer to my preferred place at two Cs. Still brought in two As and two Bs though.
If you have a conditional offer then you face the prospect of being like everyone in the rUK.
It's pretty nice if you get unconditional offers!
Edit:
"38% of applicants (97,045 applicants) received at least one offer with an unconditional component in 2019, increasing from 34% (87,540 applicants) in 2018, and continuing the year-on-year growth in offers with an unconditional component since 2013." [1]
This year at least one Oxford College has already decided to honour all offers without looking at actual A-Level results , whereby effectively turning all offers into unconditional offers [1]. Granted, this is not a normal year.
Let's be honest, though, if you got an offer from Oxbridge you're very good so they're not exactly taking a risk and it gets them brownie points.
[1] (PDF) https://www.ucas.com/file/250931/download?token=R8Nn7uoI#:~:....
[2] https://www.theguardian.com/education/2020/aug/15/a-levels-r...
I believe it's a way for them to better predict how many students they'll have next year, otherwise it can fluctuate pretty wildly based on exam day results.
>Were you predicted worse grades than you achieved, and as a result not offered places where you would have liked?
Yes. Story time but skip to the next quote if it's boring :)
I got 4 A's at AS level back in 2001 (I did my Alevels in 2002, the first year of the AS/A2 split). My subject teachers all agreed I'd get the same at A Level. My School decided that "to avoid over estimating" they would reduce all the predictions by 3 levels. So I was predicted BBB. What the Fuck? My friend was friends with the head of VI form (he did theater with her after school, they got drunk together at after show parties, he was very pretty and charming and she was 50+, super weird and dodgy but whatever). Somehow he was predicted AAB. The only one in the school year of ~90 students over BBB. Some people even cried over this as they wanted to do medicine and that's difficult with AAA predicted...
My friend was called for interview at Imperial, where we both wanted to go and study Physics. He went and was offered AAA. I was not called for interview.
Results came in and I got my AAA. He got BBB. Imperial didn't care and admitted him anyway.
I think I am sort of a perfect example of what's wrong with the current system: I was poor and badly turned out and a bit brutish but ferociously bright. He was sort of the opposite. I got good grades, he was better at getting predictions. He was the one that got the Imperial place.
>Those students who did not achieve the requirements and those universities who still have places to fill then enter the "clearing" process to match up.
I think this is the issue because this rarely happens even though it's meant to. There are 2 sides to this:
* If you miss the required grades, you generally keep your place. Universities routinely make an offer at X and then take you whether you get the grades. I understand why: when you reject a candidate with BBB, you're not guaranteed one with AAA, your going to end up with whatever clearing gives you. And not many AAA candidates are in clearing are they? This becomes a self fulfilling prophecy of sorts.
* If you blow through your predicted grades (As instead of Bs) you have 3 options: 1. Stick with the school you were offered at (I did this). 2. Take a whole year out and re-apply next year with your grades. That's a pretty huge step for an 18 years old to take. 3. Enter clearing, but you know that Imperial, Oxbridge etc don't really do clearing do that?
Back in 2002 there was actually discussion of forcing all uni's to keep X% of places for clearing for exactly this reason. It's also worth noting that these days something like 20% of offers are unconditional.
I think I did more than ok in the end. I had a better time than my friend (he had to live at home in first year and never really got the "experience" etc). But people should be aware of how the system works: you're actual grades make very little difference, predicted grades are KEY to getting into (not just an offer, actually into) a top uni, and the best way to get good grades is only about 50% about ability and hard work compared to looking nice and sucking up to the right people.
Maybe this is a great lesson for how life isn't fair :)
Imperial figures in my story too. I was predicted very highly, either 4 As or 3A+B, I already had an A at Mathematics under my belt, taken a year early. I approached Cambridge and Imperial. The Imperial open day was first and they offered at 2Cs, because they knew I was interested in Cambridge and figured that if I got a high offer from there I'd keep them as a backup.
The Cambridge interview day went badly. I came away feeling like I'd just been a complete dumbass, missed some really obvious stuff because I was stressed, and no offer was made. So I had Imperial on offer at 2Cs and no backup. And indeed I went there.
And that's when I discovered London nightlife, drugs, sex and dropping out. Failed the first year of my engineering course miserably and ended up studying CompSci at Southampton a year later. Which has turned out great, so no complaints :)
(I'm also lucky enough that there was nothing to pay for either course. I was in the last year through the system when there were no fees, what's happened on that front since is pretty awful, IMHO)
It's too much to hope for but... I actually think part of the answer is forcing everyone to wait a year to go to uni. Then they can get themselves together a bit more. They can get being 18 out of their systems. They can apply to what they want, not what mum and dad say. Forcing people to do national service would give people the experience of living away from home before they have to go and fail a degree over it or get a year in and realise they want to be somewhere else doing something else.
I'm so glad it turned out that way. I had no idea who I was, and needed that time to discover myself. I shouldn't even have been applying for Uni, but that's the thing you do, isn't it?
I wonder how many people from this debacle will look back and say they're grateful it happened because it gave them a chance to take a different path.
But using grades pupils have achieved during the previous years(s) seems fine to me and can potentially erase the issue of having a good/bad exam day.
In some countries, either they use exam results and/or they look at report cards for the previous years.
I am a big "final exams not coursework please" guy. But they'd be better having end of term exams for the 6 terms in A Levels and using the results from the first 5 for most of admissions IMHO.
In the previous system, there was more coursework in some subjects (almost always still <50%), thus recognising the abilities of those who don't do so well in exams, and still recognising the abilities of those who are better at exams than coursework. Exams were split over two years, with most of the material studied in year 1 examined in one go in year 1, and material studied in year 2 (building on year 1 material )examined in one go in year 2.
Now there is an almost total reliance on exams, all in one go at the end of the second year.
Hence (partly) the current storm.
That explains more than anything what the problem is with the British Civil Service. The epitome of "nobody every got fired for buying IBM".
Mock exams may be useful because, arguably, the results should already be pretty normalised, but I'm not sure if these took place this year.
So what is left is to work with is the grades the students have had throughout the year and previous years and, importantly, also what school they are in.
The latter is heavily charged politically so they are wary of communicating on how that impacts their model.
It seems to me, though, that pupils from top private schools are not complaining too much about having their grades downgraded, so it seems that it did not happen too much for them.
It's still going to prove controversial, but sounds much better than the apparently woolly 40 - 75% accuracy for the OFQUAL algorithm mentioned in the article.
"The Algorithm" must exist. Can we see it?
https://assets.publishing.service.gov.uk/government/uploads/...
It suggests that their dataset was either way to narrow or their regressions far too simple
> Obviously it has gone catastrophically wrong
I have no idea what this is referring to. The author is obviously (!) assuming some knowledge on the part of the reader that I do not possess. Can someone please fill me in?
The problem is that due to lockdown, no one sat the exams. So they kind of guessed what the grades should be based on predicted grades from teachers or the grades assigned by mock exams. However, those awarding the grades felt that predicted grades were too optimistic and there is no standardisation for mock exams between subjects at the same school, let alone multiple schools across the country.
What the exam boards seem to have done (and I don't think they've admitted to this) is moderate the grades based on past results from the school. However, schools in poorer areas traditionally do worse, so students in those areas lost anywhere from 1 to 3 grades, e.g. an A becomes a D. This includes students that never had a D at any point in the past. So then the government said students could appeal. Then they said exam grades would be adjusted back to no lower than their predicted grades. Or they could resit them once lockdown ended.
Meanwhile university places are filling up based on the first set of grades. Some universities are doing better, notably one of the Oxford colleges who decided to honour all offers made on the basis that they have no new information to change their offer on. But those that didn't apply to these universities have to go through clearing but still don't know what their grades are. Universities need to confirm places for accomodation which is also filling up fast.
To call it a catastrophe isn't far off the mark.
Twice in the opening text it talks about teacher predictions being dropped, but they aren't, the teachers ranked the students and this is used as part of the input data. Just not the entire input data.
If it was the entire input data then we know that, miraculously, this would have been the smartest set of kids to ever come through the system. So clearly just going by teacher predictions is also wrong.
I'm sure there are other valid objections to what has happened, but this isn't really one.
Arguably, a prediction depends on a ranking but contains less and more data. Less data in that only part of the intrastudent rank is maintained, more in that the ranks are tied to grades.
It would seem that you agree that teachers cannot be expected to produce fair absolute grades, but will produce fairer rankings - which is supported by the articles references.
In terms of the objection being obvious - it is obvious in a political sense that people would quarrel with the prediction being dropped as it both favours them and is more personal. Mathematically its presented as obvious that this is preferable [1].
[1] personaly I object in principal to us estimating grades full stop. I'd have preferred spending the last five months rewriting exams based on covered material and then a mammoth effort at getting kids to sit them safely. However that requires foresight and funding.
It's clearly a massive kludge - those rankings are only estimates, lots of kids would have performed better or worse than estimated, and effectively they are being granted grades based on a whole load of factors other than their own actual performance. I feel very sorry for them.
They almost certainly...wouldn't? The exams are stressful and hard work: both high- and low-achieving students were celebrating when the standardised exams got cancelled here in Ireland and I'm sure it was little different in the UK.
At best, those who think they'd get better grades in the exam would prefer it. The ultimate use of these grades is zero-sum (there are only so many university places) so every decision that hurts one student is helping someone else.
It doesn't really matter how good the model is, it's an impossible task politically. There will always be some people aggrieved by the new process and they will always have plausible ways to spin it as racist or classist or whatever else is necessary to fuel the outrage machine. As if the previous setup was some kind of examplar of egalitarianism.
Which most state run schools are, of course.
This is a nightmare, really. Like, no one can directly prove the malice here, but it certainly feels like there was an aspect of malevolence in the enterprise.
I'm no fan of the UK government but I'm pretty sure this is a textbook example of Hanlon's razor
They want the state students to get to universities yes, but they want the private students to get to the good universities (oxbridge).
Much of the UK educational system is like that. For example, I don't know how it works elsewhere, but in England, certain students will only be offered certain exam questions, in the same subject, if the teachers determine you can take them.
So immediately, you can see some aspect of these grades are based on a subjective standard. In this case, at the discretion of teachers. One could easily argue that this Covid exam fiasco is a supercharged manifestation of those very same principles.
In the above case, obviously, all students sitting the same exam, for the same subject, should see the same exam papers.
I mean, why would they not?
I suspect the obvious answer would be the cost of teaching resource. If that's the case, this then becomes a political argument, not a technical one. But the 'exam algo' debate is definitely a technical one, and it can and should be used as a lens to examine (pun) a number of questionable practices in the UK education system.
Good independent schools (and grammar schools) have very consistent results. Often 70%+ of A/A* year after year.
If previous grades and predictions say that most pupils should get A/A* that's not surprising. That's the way it is every year, and that probably does not indicate an inflation because of this year's strange circumstances.
But for a comprehensive school where results can vary much more from pupil to pupil, and for which past data are a mixed bag, it is trickier.
They celebrated the message "Exams are cancelled"
If the message had instead been "Exams are cancelled, at this school A grade students will receive a C instead" I suspect you'd have seen a very different response.
There is some benefit to society to matching up the right students to the right courses, so it's not completely zero-sum. But the cost of getting it a bit wrong isn't terrible: some marginal students get to follow their dreams and study medicine, some others miss out on medicine and have to try their hand at accountancy.
After all, it's understandable why a teacher would over-estimate grades (assumption everyone else is going to do the same, and their kids will be relatively disadvantaged if they don't) and hence get moderated down.
And it's understandable why a teacher would give spot-on estimated grades (honesty, valuing the credibility of their estimates) and hence not get moderated at all.
But it's difficult to see why a teacher would under-estimate grades.
So I suspect there are not many C grade students getting upgraded to an A.
So we'd need exams tailored to just the part of the curriculum that the students have actually covered -- which would be different for each school, making any kind of standardisation impossible.
Another thing people don't seem to be considering is the knock-on effect on universities, which -- if students who didn't complete the A-level course are given a hypothetical grade, whether based on teachers' predictions or modified exams or "mocks" or whatever -- will get an intake of students who are less well prepared than normal, and will need some kind of remedial support if they're not going to flounder in their degree courses.
Just giving students generous grades now, to calm the current storm, means storing up trouble for the future when university teaching staff are faced with a cohort that includes students who shouldn't be there and can't cope -- but their teachers gave over-generous estimates -- or who in principle should be OK but arrived woefully under-prepared because they missed a bunch of key A-level content.
If one student does not know key parts of the curriculum and the other does, then this should be reflected in the grade no matter what caused that difference, whether one of them got more schooling or studied on their own or took some out-of-school tutoring.
And if a diligent student who got all A's before missed out on key parts of the curriculum that's required for further study, then the final grade should reflect that they do not (yet) know the subject as expected and are going to need remedial support, i.e. their final grade should not be an A, the exam should ask for the expected content (no matter if it was not covered in school due to Covid) and the exam results and grade should illustrate whether the students have learned it otherwise or if there's a gap.
While that'd result in grades that would be far more meaningful than any of what's currently being done -- and universities, employers, etc would just have to take account of the special circumstances surrounding 2020 results when making admission/employment decisions -- I don't think it's politically tenable.
Children/adults in care (people without families that can support them) might not be eligible for support for another year, and will often be expected to go out and work to support themselves. Poor families often rely on the extra financial support that ends with "finishing" education.
There are a lot of stories out there of children/young adults in exactly this situtation that should be off to skilled work training or university that might end up having their entire education wrecked over this despite being able academically. And this is all with the context that private school students often have not been subject to any "standardisation" process due to their small class sizes, or have had their awards upgraded above predicted grades.
We just need to give this year a free pass in much the same way as workers have been given a free pass with the furlough scheme. We can live with one year of inflated results.
Presumably this will lead to grade inflation.
https://www.bbc.co.uk/news/uk-53810655
See also Wales seeming being the first to switch to this approach: https://www.bbc.co.uk/news/uk-wales-53807854
So just because every system has flaws, we should throw in the towel and give up? Taking that line of thinking all the way means we can just replace university allocations with a lottery.
That's not how society works.
Even if every system have flaws, even if the current suffers from racism and classism, we have to strive towards making it better. The system the UK invented here does not even try. It very clearly favors people with nice postcodes.
Maybe we should replace it with a lottery, it would be equal. But I'm pretty sure the people with nice postcodes would like it much, and that's the people propping up the UK gov. Etonians, Oxford, silver spoon etc.
I for one believe there must be _some way_ to fix the situation and is better than what they came up with.
That argument can be made without drawing emotional prejudices into it, but it wouldn't hurt to raise the question of prejudice while conducting this debate, because there has been evidence of a class based skew in the UK education system for years now.
And this appears to be the most overt attempt to rig the system yet seen.
There's a public sector equality duty under the Equality Act 2010 which is one reason people are looking at the disproportionate impact on some groups.
But if you have a transparent process and documented algorithms, it's much, much harder for them to do that. Of course, UKGOV has neither of those things in this instance...
Then any regulatory board could have worked 'backwards', so to speak... Only look at schools where there are blatant discrepancies across aggregated results.
There was never any mention of the application of an opaque post-process. You would think that would be the kind of detail you would have to discuss in a national assembly before the state commissions it's implementation, but clearly, it was never raised.
So while I understand your point (accusations of 'classism' are premature), I think the fact that we're only learning the details of this system now, is an outrageous abuse of trust and technology.
Concerns over long-term grade inflation are totally overblown anyway since the majority of these exam results get used that year or the next for university admission and only for that. What does it matter whether my grades from 20 years ago are comparable with grades now? It's not like we'd be competing for the same university places.
The only problem with accepting the CAGs is that some centres will have been laxer with them but that could easily have been sorted by using similar techniques to the ones they used here to look for way out-there schools that clearly aren't taking the process seriously.
Guessing one of five outcomes correctly 50% of the time is pretty good.
I've always thought those things were better than exams before remote learning regardless and many school boards weigh them similarly in aggregate. Testing was always a bit of a flawed model pandemic or not.
I'm currently studying a part-time undergrad in CS at the Open Uni and even they have failed to provide an alternative to exams this year. You would think if any educational body would be prepared, it would be the Open Uni. If an institution that prides itself on distance learning can't make it happen, lord knows what's happening at traditional schools.
However, teachers naturally want to motivate their best performers (what teacher would grade so that the best student in their class would get a B- ?) and school administrators have a practical incentive to inflate the grades. This means that subjective evaluation of speeches and projects (especially if you're comparing them within a single institution, not to what projects and speeches are considered good and bad in other institutions) and continuous real-time questioning won't lead to an evaluation that's useful to determine where a student stands in comparison to other students in other institutions, and furthermore, they are going to be optimistic. A relevant headline is "40% of A-level results were downgraded from teachers' predictions." - however, that seems normal and expected to me; do we have the equivalent numbers from last year, comparing the teacher's prodictions with the actual exam results?
Technically you might consider taking the "local evaluation" and make an adjustment for the institution, but this is exactly what they attempted to do this year in UK, and the whole original article is about the limitations of this approach.
Also, fraud and cheating is a problem. There are many approaches that have proven to result in somewhat effective remote teaching and learning, however, we don't have good solutions for effective remote evaluation (this is a problem that I personally experienced this spring in my work). For the pre-COVID remote education, the only thing that IMHO worked was a network of in-person proctoring centers supervising the testing process and identity of the people who are taking the test (otherwise people will get others to take the test for them, it's not a hypothetical issue) or, in certain cases, a privacy invasive and labor intensive online proctoring.
IMHO it makes all sense to separate instruction and teaching from final evaluation and certification, due to the unavoidable conflicts of interest and misleading results that happen otherwise. This is also what UK has chosen as its policy - but as they found out, this means that they can't have a meaningful evaluation if they skip the actual evaluation part this year.
It's about 39%. The problem is that in a normal year, everyone comes out an exam with a rough sense of how they've done. If you come out feeling like you bombed it then seeing your B become a C is not going to be a surprise.
The issue is that "predicted grades" are not expectation values in a statistical sense, if they were then teachers would be pretty consistently doing a bad job of forecasting them.
A predicted grade is based on an assessment of ability: "What do I think little Johnny is capable of when he takes his exam in however months time". Now in theory, results can be both better and worse than that prediction. In practice, the fact that the scale is closed ended (and many more are near the top than the bottom) means that it is more common for the variability of exam performance to lead to slightly worse performance. In other words, more pupils underperform their ability based prediction on exam day than outperform it.
So there is nothing inherently wrong with reducing grades from the centre assessed grades, many newspaper headlines are treating this like the CAG is the "real" grade that is somehow being taken away.
If you look at this table: https://twitter.com/Samfr/status/1293979033458417668/photo/1 you can see that the overall grade distribution produced by the model for this year is similar to that for previous years. Not surprising since that is by design.
The problem here is that Ofqual is treating grade inflation like it's the worth possible thing that could happen and has essentially sacrificed individual fairness on the altar of aggregate consistency.
Ranking students might be roughly correct, but many of the rankings in the middle where students are very similar are basically random.
Imagine there are two schools: A typical small private school, which normally achieves high grades, and a larger state school which normally achieves a broader spread. Normally the highest individual grade is achieved at the private school.
The historic average of the private school might be A-star, A, B, but predictions were A-star, A, A. Rankings are correct to the individuals' abilities. "Actual" or "deserved" standard could be A, A, A or A, A, B.
The state school's top 3 grades are normally A, A, B. This year you have a student who's capable (and likely) to achieve an A*. Teacher predictions for the top three grades are therefore A-star, A, A. Rankings are still correct according to ability.
Across the country the top 6 grades are normally A-star, A, A, A, B, B.
The private school uses the teacher predictions, because it has a small class: A-star, A, A.
We're now left with only A, B, B available to give to the top students at the state school, so the A-star, A, A assessed students are awarded A, B, B. Every one is a downgrade.
But it's OK, because across the country there's no grade inflation.
Thankfully the UK government have now decided to revert to the teacher assessed grades.
The whole reason we have a controversy over this is that the teachers were asked to give predicted grade to their pupils, and in aggregate their predictions were improbably high compared to other recent year groups.
Also these days I'd heavily debate your premise- i think exams are more of a test of memory and being able to game the system especially in the UK where you are given a test bank of past papers which are almost identical to exam papers. I was a terrible student until 2 weeks prior to the final exam where I grinded maybe 6 or 7 past papers and at least 3 out of 10 questions were carbon copied from previous papers. I would not be able to approach a problem that fit outside of the structure of the problems in the exam papers in the real world.
Which sounds like continuous, standardised, externally applied examination. Leaves rather little room for individuality in teaching techniques or pace...
> How it currently works is that these types of assessments are treated as second class citizens which i guess was really my frustration.
In as much as these things exist at all, they are part of the teaching method and aren't in any way standardised.
I'm not saying your proposal is wrong, but it would lead to a much more rigid system.
You're trying to work out who is a good fit from very limited data. You can run some practical tests - whiteboard, paper exam - but that doesn't take into account daily variations in performance, and it disadvantages outliers who may have very useful skills or character traits that don't tick the usual boxes.
The problem may be more that education is seen an industrial production line with authoritarian control of outcomes than as a personal development tool. Early apprenticeships and interning opportunities with real tasks to solve and real challenges to face might be better at developing a broader range of talents in context.
Projects sounds like another word for coursework.
Judging on prepared speeches doesn't sound fair at all, unless it's a course on public speaking.
There's a reason things are as they are.
> Maybe continuous in-class real-time questioning to see if the student is keeping up?
So when a student asks a question in class, they have to do so in the knowledge that they may be negatively judged for it? There's a reason learning and assessment are at least somewhat separated.