Alongside lots of sectors the English exam system has been thrown into turmoil by the impact of the Covid-19 pandemic. For the first time ever exams were cancelled in the summer of 2020, and now they have been cancelled again for 2021. All of the changes made as part of these cancellations have been done under the direction and leadership of the Department for Education, with Gavin Williamson, the Secretary of State for Education having overall responsibility.
With the cancellation of the 2021 summer exam series for A-levels and GCSEs, schools and colleges responsible for exams are once again in a state of confusion and another cohort of students are currently uncertain about how their qualifications will be awarded. The exams system has now been in turmoil and uncertainty for 10 months and that is set to continue at least until the GCSE and A-level results are released in August 2021.
Of course there have been impacts in many sectors, and the perhaps over used but appropriate labelling of 2020 and 2021 as “unprecedented times” may excuse some aspects of this uncertainty. However, when the changes to exams for 2020 and 2021 are looked at as a whole there is a pattern that emerges of missed opportunities and warnings that have been ignored, let’s start in March 2020…
Closure of schools in March 2020
When Gavin Williamson announced on the 18th March that schools would close to the majority of pupils from 20th March it instantly raised the question about what would happen for those students with exams scheduled at the end of the academic year. (announcement)
What the DfE could have done is place exams “under review” at least for a few weeks while the impact of school closures were evaluated and consultations could be progressed on possible routes for providing grades. Keeping exams under review would have given students in exam years a clear reason to stay engaged with schools and complete their courses. Through review and consultation with schools and colleges, alongside Ofqual and Exam boards, it may have been possible to establish some form of structured, standardised moderated assessments that could be conducted later in the academic year.
What the DfE actually did was immediately state on 18th March that all exams are cancelled for summer 2020. With schools closed and no exams it became clear that large numbers of Year 11 and 13 students were disengaging from their school and college courses altogether.
In April 2020 Ofqual issued guidance making it clear that any work done by students after 20th March must be treated with extreme caution (see here). This instantly made any further work set or done by examined students pointless, and therefore most schools and colleges stopped setting any at all for exam years.
The impact these decisions had is that many Year 11 and 13 students disengaged from school or college altogether, and as a result many of them will not actually have completed the courses that they now have GCSE or A-Level qualifications in.
Keeping exams under review in the early days of school closures would certainly have increased pressures on schools who would have had to include provision for exam years as part of their hurried moves to remote learning. However ruling out exams altogether at such an early stage certainly harmed the integrity of the qualifications that those students now have.
Method of awarding grades in 2020
What the DfE could have done is use Centre Assessed Grades (CAGs) alongside statistical models, and in fact this is actually what they said they were going to do (see below). This should have ensured that there was a process to compare CAGs with the grades expected from statistical models. As part of this build in allowable variances to recognise that professional judgements of the 2020 cohort may not conform perfectly to historical statistics. Where CAGs were clearly out of line with statistical models seek supporting justifications from schools and colleges (in fact the Ofqual guidance issued in April 2020 [here] stated that centres should retain records of supporting evidence “… in case exam boards have queries about the data”). If justifications are accepted then the CAGs could stand, where justifications were insufficient the statistical model may have overruled, or perhaps where there were large changes there could be an element of splitting the difference.
This would of course require discussion of contentious grades between exam boards and exam centres before results days, however there was plenty of time to plan in the processes for this. Alternatively if the results could not have been discussed in advance a fuller appeals process could have been established that would allow schools and students to submit convincing evidence where an individual was badly served by the statistical model.
Included in all of this could have been the general principle that the statistical model would not raise grades awarded by centres. While it is logical, natural even, for teachers to err on the side of optimism when awarding grades, if a professional has stated that a student should receive a B grade at A-level it is likely because they feel the student genuinely isn’t of sufficient ability to secure an A grade.
What the DfE actually did was design an algorithm that only applied statistics to the rank orders provided by centres and allowed no substantive scope for appeal.
While it was stated in the Ofqual documentation that CAGs would be considered the algorithm only considered the rank order of students with the CAGs completely ignored, even when there were multiple grade differences between the CAG and the statistical expectation.
Despite stating that schools should keep suitable evidence to support grading decisions, there were no processes in place to use this evidence as part of any appeal, and no proactive action taken to query discrepancies between CAGs and the algorithm. Where the algorithm assigned a grade that was different to the grade given by teachers the algorithm was taken as correct without question.
The algorithm artificially raised some grades when it calculated that the school was entitled to more high grades, even if the teachers knew for a fact that the students do not deserve those grades. The algorithm also lowered other grades from CAGs when the statistics didn’t fit the individuals and completely ignored the grades that teachers had assigned, with no scope for hard evidence to be submitted to challenge this.
The process even forced schools to rank students differently even if their performance was academically identical on all school measures. When the algorithm was applied students on the same CAG but different rank were treated very differently, meaning students on the same CAG were awarded different grades, sometimes 2 or 3 grades different.
Early warning of concerns of an algorithmic approach emerged with the issue of results in Scotland, and the devolved Scottish government rapidly U turned and reverted to awarding CAGs. This was ignored and dismissed by the DfE for England and A-Level results were issued via algorithm on 16th August regardless.
The appeals process established was only based on challenging factual errors in the application of the algorithm, or when there were large scale statistical differences that the algorithm failed to account for. There was no scope to appeal small scale local departures from statistics or clearly unfair outcomes at an individual level.
On 17th August, just 24 hours after results issued the decision in England was overturned, reverting to CAGs and also applying any grade uplifts from the algorithm. This meant that not only was there an element of grade inflation from CAGs, but this was actually increased via the algorithmically uplifted grades.
On 27th August 2020 Boris Johnson placed the blame for the resulting fiasco on a “mutant algorithm”(here). As an A-Level Further Mathematics teacher who teaches a unit on “Algorithms” I can assure you that algorithms do not mutate. The key point of an algorithm is that they do specifically and precisely what they are designed to do and nothing else. The fact that the outcome did not deliver the desired results is entirely the responsibility of those that designed, tested and quality controlled the algorithm.
The fundamental flaw was that the algorithm and all the related quality assurance processes were only focused at a headline statistical level. Nationally the algorithm worked as it produced broadly the right pass rates, the same can even be said at a whole school level. However because the algorithm paid absolutely no notice to the CAGs as judged by professionals, and ignored the impact at an individual student level it completely failed to build in year to year or student to student variation.
The impacts these decisions had are massive grade inflation across GCSE and A-Levels for 2020, combined with, public uproar and loss of faith in the exam system. Delays to the decision meant that A-Level students with grades that changed missed out on university places in 2020 because courses had already been filled by other students.
Start of the school year in September 2020
What the DfE could have done is state that the intention was to run exams in the summer of 2021 but ensure that there was a parallel system in place to guide schools in building a strong evidence base to support potential CAG use in case of disruption.
Possible examples of a parallel system could be clear national standards for the timing and nature of mock exams, perhaps even specify common exam papers to use nationally so that there is a central reference point across schools. Alongside or alternatively exam boards for each subject could have established some core common assessments for each subject that can be done in schools at key points during the year to get a common base of comparison.
Schools could have been given clear and early direction on “Special Consideration” and how this applies to disruption due to the ongoing Covid-19 pandemic (Special consideration is the process through which schools make requests to exam boards to account for pupil illness on the day of an exam or other significant disruption to their exam or preparation). This guidance for example may have laid out how many days of isolation or remote learning would qualify for special consideration. Perhaps give clarity on whether it matters when in the year isolation or remote learning occurs, or whether specialist teacher absence should or could be accounted for. If the student themselves is made unwell due to Covid-19 at any point in the year should a consideration to be made? If a student is impacted by Covid-19 related bereavements of family or close friends, or of teachers should there be special consideration given? Or should account be made for student household impacts such as illness, loss of income, or furlough of parents?
The above guidance could have been put in place at any point during the autumn term, or at least plans established to put this guidance in place with a clear process of consultation and defined point where a decision would be made. Particularly as Covid-19 cases rose nationally from September through December and the fact that attendance at schools was being substantially impacted there should have been some response along these lines.
What the DfE actually did was repeatedly state intention to run exams.
On 12th October Gavin Williamson announced a delay to the exam season to allow students more time to prepare (here). As part of this he committed that the exams would be “underpinned by contingencies for all possible scenarios”, with a promise to consult further with stakeholders and publish more detail later in the autumn term.
On 3rd December Gavin Williamson announced some further measures to support exams in 2021 (here). These extra measures included a suggestion of students being told some exam topics in advance and more help in terms of reference materials in exams, however there was no actual detail given at this point so schools and students could not respond in any way other than preparing for exams as normal. He also announced that grading would be more generous than normal, which simply means that the grade inflation impact of 2020 would be carried across into 2021. While he announced an expert group to monitor the variation in learning across the country he gave no actual detail that schools could work with and plan on. There was no confirmation of any form of contingency planning.
When Scotland cancelled their National 5 exams in October (here) and later cancelled their Higher exams in December (here), and other home nations announced moves away from exams the DfE resisted all calls to consider alternatives and put contingencies in place for England. This position continued to be publicly reinforced, including the adherence to taking exams right up to the end of December (here). On 30th December Gavin Williamson emphasised that Exam years would be at the “head of the queue” for rapid covid testing as part of schools reopening in January (here). This position remained the case right up to the point when schools were closed as part of the January 2021 lockdown (here).
The January 4th lockdown announcement (here) stated that “it is not possible for exams in the summer to go ahead as planned.” This also stated that the DfE would “be working with Ofqual to consult rapidly to put in place alternative arrangements that will allow students to progress fairly.”
On 6th January Gavin Williamson changed direction further by announcing that all exams “will not go ahead this summer” (here). This is despite schools being expected to reopen and welcome exam years back well before the end of the academic year. Stating in January that all summer exams are cancelled closes the door to formal standardised testing across any subject, instantly reducing the potential for reliable comparisons across exam centres. This statement included the claim that “The department and Ofqual had already worked up a range of contingency options.” However the statement gave proper details on none of these options, despite apparently working on all possible contingencies since 12th October. The only fragment of detail given was to confirm that he wishes “to use a form of teacher-assessed grades, with training and support provided to ensure these are awarded fairly and consistently.” So on the face of it the plan for this year is to do the same as last year, overseen by the exact same people and regulators as last year. The only difference apparently is that this year they intend to train teachers better to make sure that they don’t make the mistakes that the algorithm designers made last year. In 2020 they made a similar statement that teachers and schools would be given guidance on how to award CAGs fairly… We know how that went.
Even with lockdown in place the government also allowed January vocational exams to go ahead. When questioned on this they advised schools that they can run the exams “where they judge it right to do so” (here) but gave no guidance at all on how to judge if it was right or not. There are no definitions of what would constitute “safe” or “unsafe” conditions for running an exam in a lockdown and the DfE have resisted all calls to lay them down.
Throughout this no guidance at all has been given on how “special consideration” may relate to Covid, and no timeline given for when guidance might be shared. In fact there has been no indication that any account would be taken for the November series of exams, the January vocational exams that do run, or any other assessments that have or still will happen this year.
The impacts of these actions are that despite making statements that contingency plans were being made the exams have been cancelled again at short notice with no detail at all on what those contingencies will be. At every stage the DfE, Ofqual and Exam boards have been completely unprepared for disruption to the year, meaning schools are without guidance from these vital organisations. While it has been obvious since the first school had to isolate the first child in September that there would be some form of disruption to the 2021 exams, the lack of available detail or action to safeguard comparable assessments during the year shows a complete lack of foresight.
Schools, colleges, and students in exam years have once again been left with no guidance on what is to become of their qualifications. There are now genuine fears that we will have a repeat of the grades debacle that happened last time as the exact same people are at the helm and absolutely nothing has been changed in the meantime to provide more robust preparations for CAGs. Schools again are left unclear on the guidance to give to parents and students.
Overall then….
At every decision point through the pandemic there was a route that would have given a more strategic response, clearer guidance and better outcomes for students and the wider education sector. These more strategic responses do not require hindsight, they simply require foresight and understanding of the education and exams sector. If staff at the DfE and at Ofqual do not have this foresight then they need to acknowledge this and consult those that do as a matter of routine, ideally educating themselves to the point that they can provide this too.
Despite there being opportunities to find a better path at every stage the DfE has chosen to take actions that actively reduce guidance, limit the opportunities for robust assessments and harm outcomes for students. Every decision has been taken too late, and right up to point of the U turn the DfE and wider government have continued to actively brief and reinforce the previous line, even sometimes up to the point at which a different decision is announced. This is so consistently the case it could almost be viewed as a systematic attack on the education and examinations system, with last minute changes enacted specifically to destabilise the entire sector.
To make one or two mistakes or errors of judgement during the unprecedented times of Covid I could understand. To make mistakes and errors of judgement at EVERY stage however requires active incompetence. To ignore the likelihood of disruption to the 2020-21 academic year and make no contingency plans at all is a sign of complete negligence and lack of leadership. From the point of cancellation of exams in 2020 there should have been plans made to make sure we had better information to work with for 2021 and clear parallel processes in place.
Let us remember here, this is Gavin Williamson’s job, alongside all of those that work for him in the Department for Education and Ofqual (Ofqual annual budget of the order of £17.5million). Williamson is paid a ministerial salary of £67,500 per year (in addition to his basic MP’s salary of £74,962) to do this role. Given the lack of action and negligence evident in the above I need to ask what has he, DfE and Ofqual have actually been doing to improve the exam and qualification system since March 2020 (or for since he assumed office in July 2019 for that matter)?
They say “to err is human”… To repeatedly and systematically ignore advice and fail to make adequate contingency plans is at best incompetent, certainly negligent and at worst deliberately undermining. As a country we simply cannot continue treating students in exam years, their parents, and their schools like this. The Department for Education has been complicit in the undermining of qualifications for two years in a row now, in Ofsted parlance they wouldn’t just “require improvement”, they would be “inadequate”. A school this badly run would have been placed in special measures and the leadership team replaced.
Despite this catalogue of failings (and that doesn’t even scratch the surface of the turmoil in the Primary, Early Years and University sectors that he is also responsible for) Gavin Williamson then has the temerity to suggest that parents should complain to Ofsted if their remote learning isn’t up to scratch just two days after the government made the U turn to close schools (here).
This isn’t party political – it’s about leadership, or more specifically the lack of leadership of the DfE, and that is true regardless of the political affiliations of those involved.
On reflection of all of the above I make you this offer – pay me £67,500, it doesn’t even need to be in addition to my teaching salary… in fact just pay me a basic teacher’s salary and I guarantee that I will make substantially more impact on improving education in the first 3 months than Gavin Williamson has done in the last 17 months that he has been in post…
We cannot continue with an inadequate DfE...
