Is there still an appetite for ‘mastery’ qualifications? - Paul Newton

By Paul Newton, research chair at Ofqual

The ‘mastery’ qualifications at the heart of this blog are ones in which:

1. unit content is specified via learning outcomes
2. the unit standard is specified via assessment criteria for each learning outcome
3. to pass each unit, a learner must acquire all specified learning outcomes, which we refer to as the mastery requirement

Since the 1980s, many qualifications have adopted this approach to qualification design, from National Vocational Qualifications (NVQs) to BTECs, and all of the qualifications that were accredited into the Qualifications and Credit Framework (QCF). For want of a name for this broad family of qualifications, we recently christened them CASLO qualifications, because they all ask assessors to “confirm the acquisition of specified learning outcomes”.

Although the CASLO approach came to dominate the landscape of regulated qualifications in England, it has always been controversial. NVQs were heavily criticised by educationists from the outset. More recently, characteristics of QCF qualifications – including the mastery requirement – have been challenged in a number of high-profile policy reviews, including the Wolf report. I’m currently leading a research programme at Ofqual into the past, present, and possible future of the CASLO approach, and one of my questions is whether the appetite for mastery qualifications has receded in recent years.

Back in the day, when I mainly researched GCSEs and A levels, if you’d have asked me what gave rise to the CASLO approach, I’d probably have said that it’s just the logical approach to designing occupational qualifications (and, by extension, vocational ones too). After all, if you need to certify that a prospective pilot can fly an aeroplane, you first need to figure out the elements that collectively comprise flying an aeroplane (the outcomes), then you need to determine what it means to achieve each of these outcomes (the criteria), and finally you need to be sure that each of the prospective pilots has actually achieved each outcome (mastery). Critically, it’s no good if a pilot can take off but not land – they need to be able to do both, and everything in between.

Having spent some time studying the evolution of the CASLO approach in England, I now appreciate that it was introduced for more subtle reasons linked to North American educational movements from the middle of the twentieth century. In other words, although the CASLO approach came to prominence in England via occupational qualifications – NVQs, in particular, during the late 1980s – it was introduced for reasons that were fundamentally educational.

The idea of specifying qualifications in terms of learning outcomes and assessment criteria can be traced back to the Objectives Movement, pioneered most prominently by Ralph Tyler. He is best known for his 1949 book Basic Principles of Curriculum and Instruction, in which he criticised approaches to planning curriculum, pedagogy, and assessment that launch from a foundation of syllabus content alone. Too often, he claimed, it was entirely unclear what students needed to “do” with that content. Recall it? Understand it? Apply it? Evaluate it? Furthermore, because the more complex of these cognitive “behaviours” also tend to be the hardest to assess – particularly via traditional written exams – this can result in assessment focusing unduly on less complex ones, with a consequent negative backwash impact on teaching and learning, and without this necessarily even being noticed.

The solution, Tyler believed, was to start, not from teaching inputs (syllabus content), but from learning outcomes (genuine educational objectives). Agreement over a comprehensive set of learning outcomes provides a blueprint for alignment between curriculum, pedagogy, and assessment. Everyone sings from the same hymn sheet, and everyone sings all of the lines. Where complex cognitive behaviours are specified as valued learning outcomes, we do our best to find a way to assess them.

This is exactly how Gilbert Jessup, the principal architect of the NVQ system, envisaged the impact of the CASLO approach within NVQs. NVQ design would start from a comprehensive specification of outcomes, which would drive a comprehensive assessment process, which would drive comprehensive learning. Gone were the days of placing too much reliance upon traditional written exams. Continuous authentic assessment would be the way forward.

But what about mastery? In the notes section of his 1991 book – Outcomes: NVQs and the Emerging Model of Education and Training – Jessup quoted Benjamin Bloom, a student of Tyler, and pioneer of the Mastery Movement: “The most wasteful and destructive aspect of our present educational system is the set of expectations about student learning each teacher brings to the beginning of a new course or term. The instructor expects a third of his pupils to learn well what is taught, a third to learn less well, and a third to fail or just “get by.” These expectations are transmitted to the pupils through school grading policies and practices and through the methods and materials of instruction. … A pernicious self-fulfilling prophecy has been created.” (page 147, quoting Bloom, 1971, p.47)

Bloom wished to disrupt this tradition of low expectation, claiming that most students (perhaps more than 90%) were capable of mastering the learning outcomes that were specified for them. It was the job of the teacher to facilitate this, using assessment formatively to diagnose learning gaps and to identify appropriate instructional interventions. The new goal was to create a self-fulfilling prophecy of success, reinforced by repeated experience of successful learning. As Jessup was later to put it: the concept of failure was no longer relevant; learners would simply continue learning until they reached the required standard of competence.

The CASLO approach generalised from NVQs to many other qualifications, particularly with the introduction of the Qualifications and Credit Framework, which mandated all three of its core characteristics for accredited qualifications. By the end of 2014, most of the qualifications that Ofqual regulated were accredited to the QCF and were therefore obliged to adopt the CASLO approach.

Fairly soon after its introduction, the QCF came in for considerable criticism, and this included criticism of the CASLO approach. The Review of Vocational Qualifications was particularly critical. Alison Wolf argued that QCF qualifications have various characteristics which are “ill-suited to delivery within education and training institutions” (page 88). This included the mastery requirement, which she argued exerted downward pressure on standards. If we require all learners to achieve all learning outcomes, then we are prohibited from specifying difficult learning outcomes, else the lowest-attaining students might fail in droves. This may be less of a challenge for apprentices who learn on the job with no fixed time limits, but it is particularly problematic for college-based students following sessional courses.

There may well be ways of mitigating concerns of this sort, for instance, by building grading into the CASLO design template. Yet, it is fair to say that the CASLO approach has faced a lot of criticism over the years, and the idea of mastering learning outcomes has not always been operationalised as well as Jessup would have hoped.

The QCF was withdrawn in 2015. Ofqual no longer requires any type of qualification to adopt the CASLO approach, and there are fewer CASLO qualifications on our register nowadays. So, has the appetite for mastery receded in recent years?

On the one hand, there’s been a resurgence of interest in mastery teaching and learning in certain contexts, particularly in relation to mathematics. On the other hand, we can embrace mastery teaching and learning without necessarily also embracing mastery qualifications. Although, having said that, we also know how forcefully the assessment tail wags the dog of curriculum and pedagogy. Where a qualification stipulates mastery as a requirement, we can certainly be more confident that students and teachers will aspire to it.

In certain assessment and qualification contexts, the concept of mastery has evolved over time. In response to the Richard Review of Apprenticeships, the Future of Apprenticeships in England Implementation Plan continued to embrace mastery as the certificated goal of an apprenticeship, but recast the concept far more holistically, as the ability to transfer learnt skills from one setting to another to enable workers to adapt to change within a sector. A new perspective on mastery is also built into the T level model, through the concept of threshold competence within an occupational specialism, which is defined as being as close to full occupational competence as can reasonably be expected of classroom-based learners.

Certificating mastery is no mean feat. The fundamental challenge is how to confirm competence across a full domain of learning without having a negative backwash impact on the quality of teaching and learning, and without the assessment process becoming unbearably burdensome. It is quite possible that conceptions of mastery may continue to evolve as we continue to grapple with this challenging conundrum.

If you feel strongly about building mastery into qualification design, or about the CASLO approach more generally, then we’d love to hear your thoughts.

You can contact us on strategicrelationships@ofqual.gov.uk

The views expressed in Think Further publications do not necessarily reflect those of AoC or NCFE.