Comparative Judgement - did it live up to expectations?

My previous post described my introduction to comparative judgement as a way of assessing writing.

As a federation, we embarked on our trial of this approach, using No More Marking(NMM), and facilitated by our regional consortium, in September 2022. We were using NMM's 'Assessing Primary Writing' product.

For each year group from 1 to 6, there was an assessment 'window' and these were spread across the autumn and spring terms. A set prompt for writing was supplied just before the assessment period began and the children had to write on individual, pre-printed lined sheets. This meant that there was a certain amount of administration on my part, entering names, dates of birth, classes, gender and entitlement to free school meals (the category on the spreadsheet is 'Pupil Premium' as it's designed for the English system), and then printing and distributing the sheets for the children to write on. 

The teacher guidance is clear that after explaining the task, the teacher must not support the children with their writing. For some children, who rely on lots of reassurance, this proved challenging. Since September 2021, we have introduced regular opportunities for 'Writing for Pleasure' to develop confidence, independence, creativity and stamina. This has proved very successful in itself, but our curriculum planning for writing can still be rather rigid, almost always tied to a list of success criteria. The fact that some children struggled to write independently suggests the need for more opportunities for free expression within curriculum planning.

For most year groups, the prompt was a picture, and they were invited to write in a narrative or descriptive style. However, for Years 4 and 5, the prompt was for non-fiction writing and both staff and children had reservations about the topics. Year 4 had to imagine the government was going to ban mobile phones for children under 13, and write about whether they thought this was a good idea or not. Year 5 had to write about a famous person that they admire and why. The first of these was not age-appropriate in terms of subject matter and genre, and the second required prior research in order for the children to feel that they had sufficient information to write about. These were the only two year groups in which a few children were in tears, simply because they did not know what to write about. 

We gave feedback to NMM, as did other schools, and they have announced that all of next year's prompts will be for fiction writing, with optional non-fiction prompts that schools can use if they so wish.

Once completed, the children's writing sheets were scanned and uploaded to the NMM site, where they were anonymised and prepared for judging.

The judging took place a few weeks after each assessment window. It is good practice for as many staff members as possible to take part. Each judge is allocated a series of judgements (pairs of scripts) to complete. Most of the scripts are from your own school, with around 5% from other schools (all scripts are anonymous), to provide moderation. The judging doesn't have to be completed in one go, as long as it is completed within the judging window. For the first time, we met together to complete the judging, allowing for support to be given if necessary. For subsequent judging sessions, staff chose when to complete their allocation. As we became more familiar with the process, we took less time to complete the judging.

After each year group's judging was completed, NMM prepared a detailed report for the school, including scaled scores, writing ages, comparisons according to gender and entitlement to free school meals, and the position of the school in relation to all schools taking part. 

It was an interesting and worthwhile experience, and certainly proved the usefulness of comparative judgement as a method of assessment, particularly in the context of Curriculum for Wales.  As well as being useful for individual schools, it could be used by clusters or wider groups of schools for standardisation and moderation purposes. The process was straightforward and didn't have an adverse impact on workload, beyond the administrative elements. 

Being able to determine 'writing ages' is something unique, as far as I am aware, to NMM, and this has proved most useful for us as a federation. Following the two periods of online learning, we knew that, for many children, writing development had taken a big hit. While learning gaps in many areas have been narrowed over the time since, standards in writing are taking longer to return to pre-pandemic levels. Our results from the trial backed up what we already knew but also gave us better insight into the numbers of children in each year group whose writing is at a standard significantly below others of the same age. We were also able to create a portfolio of examples of writing from each year group illustrating the range of standards. This also clearly shows where there is greater variation within than between year groups.

So what now? In my next post, I will share what we have already done to support the development of writing, and what we need to develop further.