RAG123 user survey - the results!

I posted the RAG123 survey a few weeks ago and have now collected enough responses for it to be meaningful.

Don't know what RAG123 is? see here and here.

In total 40 people responded, which I know is fewer than the number that are actually using RAG123, but it represents those that saw the tweets about the survey and found the time to complete it - for which I am grateful as I know time is precious. 40 isn't a massive number, but it's enough to draw some conclusions on...

A quick health warning - these are the results of 40 responses - any statements made refer to the views of this sample only and shouldn't be extrapolated to wider populations. Also this was a USER survey - I've not got data from non-users, that wasn't part of the exercise. I'll also be clear that one of the responses is me - I'm a RAG123 user after all.

Profile of users
I was worried at one point that lots of people I communicate with about RAG123 appear to be departmental leadership of SLT. It made me wonder whether it is truly sustainable for a mainscale classroom teacher, however 50% of respondents are classroom teachers...

Profile of RAG123 users by job function

85% of respondents first heard about RAG123 via twitter, 5% heard from others in their school, and the final 5% are Rob Williams and I, who did the first trial at our school in November.

Subject coverage

The majority of respondents teach maths (60% if you include those who also teach another subject), and the next big group is science (25% if you include all who mention science). Humanities had 5% and the remaining few are individual teachers of other subjects.

Respondent's subjects

I wouldn't want anyone to draw a conclusion on this that RAG123 only really works for Maths and Science. Notably the two blogs/twitter feeds that have been pushing this idea are mine (Maths based) and Damian Benney's (here - Science based), as such it's hardly surprising that there is a bias here. What I am pleased by though is the fact that other subjects are represented, including the notable "Whole School." I do know from twitter exchanges that RAG123 has been used in MFL, Music, and a some other subjects too - it just happens that they didn't complete the survey.

Part of doing this survey was to collect some info on who was using RAG123 so we could share tips and best practice more directly. Users that included their twitter ID have been sorted by subject and can be found in these lists.

Impact on pupils

That's what we do it for after all!!

An overwhelming 82.5% report an improvement in either effort or attainment, or improvements in both. 10% are new RAG123 users and felt it was too soon to detect any changes, and the remaining 7.5% report no change on the part of the pupils.

Reported impact on pupils following RAG123 introduction

Impact on workload

One of the things I found when I started RAG123 was it improved my workload, I was interested to see what others thought...

Impact RAG123 has had on perceived workload

55% state their workload has decreased, 32.5% say there has been no change, and 12.5% state an increase.

Notably of those reporting increased workload all but one recognise improved pupil effort and attainment (the remaining one response is a "too soon to tell"). In the write in comments all of those with an increased workload are still very positive about RAG123. For example:

"Although I am not perfect at RAG123 and still have to do STAR, it has made marking so much quicker and actually I like doing it."

"My dept and I were sceptical and only did it cos u kept tweeting no negative feedback yet!!!! But we are sold!!"

In terms of where it has had biggest impact, 68% mention marking, 53% mention Dialogue and 60% mention planning (as these don't add to 100% you'll realise that many mention more than one of these!)

Best things about RAG123

This was a free text bit of the survey and the responses ranged from a few words to much more detailed. I could try to pick and choose best bits, but in all honesty it's best just to see the full text cut & pasted in here:

Picking up on misconceptions at source and the value pupils place on such regular marking. Also the way that informs your planning. Impossible to say just one.

Dialogue and relationships with students

Although my work load has increased as I am now taking books in every lesson for checking RAG123, it is a positive increase. I am able to judge how well my lesson has gone straight away. I can use the RAG123 to set targets more effectively and cater better for the individual. As a result DIRT happens every lesson now which I hope will pay off with regard to progress over time and stickability. My students are responding positively to me monitoring their progress so closely and a better dialogue has been established. If I find a smarter way of recording targets in exercise books so there is clear signposting of what is going on for observers, my workload should decrease in the future.

Informs future planning

The simplicity of it and the ease of use.

There are many best things, and the only downside is setting aside the time every lesson to make sure students do their part in it.

Marking is very quick. Pupils marking their own work a real game changer.

I know where everyone is after every lesson and can therefore plan for this in the next lesson. No one slips through the net; misconceptions identified readily (with more certainty than other AFL techniques) - there's nowhere to hide! No chance for bad habits to develop.

Communication with students

You can fully track progress of the whole class. I can identity misconceptions earlier and check students motivation.

Improved dialogue with students

Simple self assessment

It helps me identify what is not making a piece of work G1 and able to then identify where to improve.

Enables frequent marking and formative feedback.

Ability to plan effectively the next lesson and show progress.

quick whole class overview of progress and understanding

The students know that I am looking at the books very regularly and can write me messages that I will read.

I also get them to hand in their books in RAG123 piles so I can start with/spend more time on the students who need more help.

Students are getting used to assessing their own understanding, which I think will help with their revision.

I feel like I know my students better and what they have understood

Makes sure the kids complete their work - being able to keep on top of exactly what they are doing

No more marking guilt and amazing dialogue with students.

The ease, and the fact you know where the kids are after each lesson

quick feedback

Much more informed about planning. Kids love it and whilst I'm not sure any improved attainment is down to this I am convinced improved effort is. Combining it with pupils marking their own/each others work with green pens. We mark in red.

RAG123 is quick and extremely effective. I have mini dialogues with students in their books and can see patterns in behaviours as well and spot misconceptions quickly.

The best thing and what has helped me the most is being able to manage my marking load better. We have weekly book checks, one week yr7 and yr8 and the next week, yr 9 and 10. Before I was always in a mad panic about these. Now I know that although I may not be doing great at following the latest marking policy, my books are marked and feedback is there. (Especially those which I have still kept on top off!!) I know I'm not RAGging properly as I'm trying to squeeze too much feedback in, but it's much better than the paragraphs I wrote before! 

planning for next lesson, allows me to monitor how the students are doing on a regular basis. They self-regulate their effort often.

student focus on their own progress and effort

Checking work after each lesson and before the next one!

The opportunity to have dialogue with pupils. They enjoy doing it too..

Pupils are excited to read and respond to my feedback each day. The impact it's having is worth the extra effort!

It's instant and instantly useful. Supports using LO/SC in all lessons.

I know how everyone is doing and what they need to do to improve or correct misunderstandings.

Quick, easy for both learner and teacher. gives you indication on how class doing, useful for ensuring tailor made lessons.

Regular monitoring, review link to lesson objectives, planning response better informed.

Simplicity, focus on student involvement and it's evidence based system.


The easiness of marking. It helps me to keep on top of it. I now feel a lot more knowledgable about all of my classes.

The worst things about RAG123

For balance I also need to include all of the negatives - this is an unedited cut and paste of "the worst things" - I'll try to address some of the comments in another post:

It can be a pinch if you have parents' evening/meetings after school. Can be overcome though!!

That more people aren't using it!

Nothing! What I want to do is print the RAG123 criteria on a sticker and have it at the front of exercise books. That way I can then have the success criteria on the board linked to 123 which is something I have not yet been doing. I also need to get students to improve their justifications which will come once I have linked the RAG123 criteria to the success criteria.

Colleagues' reactions when you say you mark after every lesson.

Have to remember to mark after every lesson for it to be effective

Nothing

The students who don't mark work or RAG it. 

Those who just go for A2 every time to save actually thinking. Part of the reason that we are going to try red/orange/yellow/green so you are above or below half way.

Having so many books around at school, because I am not tidy and when I lose 1 book (no doubt student put in wrong pile) I have to look for it as I know that they handed it in. (Student then says oh yeah I forgot I have it because I didn't hand it in - grrrrr)

Not sure yet

It doesn't encourage students to make subject specific comments

Needs to be done very regularly.

Sometimes hard to summarise effectively into RAG123 categories

effort grades, personally I don't use them. Grading effort is unreliable.

Not found one yet.

not too sure yet

We also have to give SWANS feedback at least every three weeks, so I have to do that as well as RAG123.

It is sometimes difficult to get the books marked before the next lesson but it's worth it

Some students really don't like it"

Marking the books everyday

Making sure you do it before you go home...

Not sure yet

Still struggle to find time to mark after every lesson - but I put that down to pastoral responsibilities - those pesky kids and parents stop me doing most things when I plan to!!!

If I set homework then I miss a lesson or two with some classes. Maybe I could think about giving separately homework books, but my experience with these has not been good.

The pressure I feel when I've had a bad week and fallen behind. Have five lessons out of six most days, and sometimes struggle to RAG everyday before the next lesson. This is because I've not got it right yet, but I get very stressed and then fall even more behind!!

Need to make more of a glance and RAG123 thing. Have started timing myself now!!!

Doesn't work if you can't keep up checking every lesson - I've fallen victim to this.

students overestimating their understanding

Probably feeling the pressure to check after every lesson, especially when there are after school commitments like parents evenings etc

I have found that it doesn't lend itself to every lesson. 

I sometimes struggle to find the time for the students to do it properly so it ends up being rushed.

Daily expectation! We have probably gone a step too far with it! Setting individual questions, activities even card sorts etc. Viewed as an investment in next lesson rather than a quick response to previous. We are worried about sustainability though.

Sometimes harder to use in English where you're not always working on something as discrete as maths.

Can't think of any. It's easily the best thing I've done in my teaching career.

No real.negatives. some year 11 boys just sat A1 as it relates to breaking bad!

It doesn't record the volume of quality verbal feedback given in maths - but then neither do other written systems, the major issue with ofsted's version of marking and feedback monitoring.

Apathy of some to look at the benefits. Mainly, "you got this from twitter!" What do Ofsted think, well now I know.

I sometimes struggle to mark books every day. Especially on one week where I teach 4 days without a free period or lunchtime or after school.

So there you have it...

There is a bit more analysis to do, and I still need to sort out the top tips bit - there are some gems in there. However I wanted to get this post published this weekend...

Notably for me all of the worst bits are things people struggle with, not reasons to stop. Yes it can be difficult to do every day, yes it takes time for students to respond to it - we need to train them in how to use it and learn how to use it ourselves.

Still sceptical of RAG123? Give it a try!!!

How do you do RAG123 so quickly?

Whenever people start off with RAG123 they take too long marking. It's not their fault - they're used to taking longer and struggle to do it more quickly. When I say a full set takes 15 minutes I am often greeted with incredulity. I thought I'd prove it... (video run time 3 mins - all the rest is explained...)

(In case of trouble streaming the video you can download a copy here http://bit.ly/RAG123video)

While the set of 26 books marked in the video took me 15 minutes 30 seconds you can see that I write an extra comment/response in almost all of them - for me this is the longest that a RAG123 set ever takes. Sometimes a full set can take less than 10 minutes if I'm not writing extra comments. As part of recording this video I actually filmed myself marking 3 full sets of books - that's 86 books and they were all reviewed in 38 minutes.

Not just about whizzing through books
I really want to emphasise though that RAG123 is actually a whole teaching approach, not just about blasting through a set of books in 10-15mins. The real strength comes from responding to what you see to shape how you approach your next lesson. It informs planning, it makes differentiation better, it helps you to get to know your students better.

Some people may say that "proper AFL in lesson is better than picking things up from reviewing books." To some extent I agree, but this gives an extra method of AFL, and one in which students have nowhere to hide.

Before RAG123 I thought I was quite good at AFL in lesson. I thought I had a good handle on what each student could and couldn't do, and what each student had actually done in lesson. When I started using RAG123 every lesson I found that I was wrong. I had a partial understanding at best, and RAG123 helps me to complete this picture. The insight it gives me helps me to meet the needs of my classes much more effectively than I ever have before. I now dislike planning a lesson until I've reviewed the output of the last one - otherwise it's too much of a guessing game.

Importantly RAG123 shouldn't replace any AFL, or other in class strategy. It also shouldn't be the only form of feedback the students receive.

As always - comments are welcome, please let me know your thoughts...

RAG123 user survey

I was asked the other day if there was a list of people using RAG123, in order to share best practice and develop it further. There currently isn't such a list... but here's our chance to create one!

If you use RAG123 please could you take 5 mins to complete this form. I'll look to share the results in due course - intention is that we can connect people using RAG123 in the same subject, same phase, etc. And just plain old share good practice...

The form is found here: http://bit.ly/RAG123survey

thanks in advance for taking the time - please share this survey with anyone you know that uses RAG123 - the more input the better...

Blogging birthday post!

My first ever post was published a year ago!! I remember being nervous about putting my thoughts out there for the world to see...

1 year later and this will be my 45th published post. More astounding to me though is that people have actually read any of them! This little blog has now clocked up almost 25,000 visits! I'm flattered that anyone takes time to read this collection of ideas that rattle round my head, so if you're reading this - THANK YOU - I know you don't have to, and this has never been about self promotion.

I was asked recently how many words I'd written as part of this blog - I worked it out the other day and was shocked to find that even excluding this post it's well over 46,000 words! (Note to self - must work on brevity!) When I discovered this massive word count I wondered which words I'd used, so I used Wordle to summarise it - I was quite pleased with the result...

Alongside this I've found myself with almost 800 followers on twitter - which also amazes me!

So what is it all about?
During the year I've blogged about various topics, from leadership to homework, to SOLO, to marking - you can find links to these topics on the right of the blog page. More recently I've been a bit preoccupied with RAG123 marking, which has dominated posts since November, but frankly it's such a powerful bit of practice I think it as deserved the priority I've given to it. (by the way - if you've not heard of RAG123 then have a look at my posts - I promise you won't regret it!)

As an indirect result of twitter and blogging, and the reflective, innovative practice that it encourages my department and I have:

• Developed feedback and marking processes
• Innovated RAG123
• Created departmental CPD open days
• Written on just about every surface in the classroom with magic whiteboard, chalkpens, etc.
• Analysed summative tests to make them formative using various tools.
• Created graffiti walls for revision
• Made question grenades
• Developed the use of QR codes, bitly links and videos for flipped learning in our department
• Created maths plasters
• Developed seating plan formats to show student data easily
• Introduced SOLO to our school
• Used formats to share learning objectives with KS5

And there are loads more things - I'm amazed how long that list is already.

Has it been worth it?
Without a doubt yes! 

Of course it takes time to write posts - sometimes more than others, but it's actually never felt like a drag. I've also surprised myself in that I've never once struggled to find a topic to write about - stuff just happens and I think "that'll make a good blog post." I wrote a post (here) in June considering why I had bothered getting involved in blogging and twitter and whether it was worth it. Having read back over that post I don't think I can improve on it, I've honestly never looked back since starting.

It has been a bit of a journey this year; discovering twitter and blogging as a CPD tool has been a massive eye opener. It's transformed my practice in many ways, and helped me to lead my department with new ideas. Comments and feedback from the blog and from tweets have helped me to improve my classroom practice, and writing about it has helped me to be more reflective about it too. Furthermore being on twitter has helped me to be more connected to the wider world of education than ever before. I'll continue blogging and using twitter because it makes me better at my job.

A new phase begins...
This week has been fairly momentous for me. In addition to the anniversary of blogging I was successful in an interview on Monday, and will now be joining my current school's SLT as an Assistant Headteacher from September.

I can honestly say that the combination of blogging and twitter helped me to be a much more credible candidate for a role on the school's SLT than I would have been without these connections. It really came home to me when I sat down to prepare for the interview and realised that I was already fully up to speed with latest thinking about lesson observations, assessment structures, learning theories, inspection frameworks, etc. In fact in some aspects I was more in touch than the majority of the existing SLT - all through keeping up to date with my twitter feed and reading other people's blogs!

I was also able to ask for advice and tips on my application for this SLT post from contacts in my personal learning network who I didn't know at all last year! There were a couple in particular who helped by proof reading my application form and offered encouragement and suggestions. I'll not name you in case you get inundated by requests from others, but you know who you are and thank you!

I fully intend to keep blogging as I make the transition from middle to senior leadership, and hope that it continues to shape my practice, and contribute to the future of my school and its students.

So after all that I think this post is really a thank you to anyone who has ever taken the time to read, comment, tweet or follow as part of my journey in the last year. I've certainly enjoyed it and think I'm a better teacher and leader as a result... I hope you've found some of it useful!

Here's to another year - I wonder what will develop in the next 12 months...!

A RAG123 epiphany

Our PGCE student had a real lightbulb moment with RAG123 this week... It emphasised to me that it is way more than a quick way to get books marked...

(as a respect for anonymity I will refer to the PGCE student  as "PG" for the rest of this post).

First lesson blues
It was PG's first lesson teaching a particular group, having only observed with them up to now. The group has a few characters in it who will always test out a new face in front of them. I wasn't observing the lesson myself but afterwards PG came to see me, clearly quite disappointed in what had happened.

In PG's view the group had not bought into the topic that had been taught. It was intended to be a lesson that picked up from earlier knowledge and then moved them into more advanced ideas. However PG felt that the students had just decided that their earlier knowledge was enough and they had rejected the more advanced method, and really not engaged with the lesson at all. This view was based on the verbal responses of a number of students during the lesson which shaped the general feel of the group (this "feel" was also backed up by the teacher who was observing PG teach).

I talked with PG about why they felt this way and what they could do to make sure that the next lesson was more successful. Now obviously part of this is a planning/experience thing for PG - at times like this it is important that the students can appreciate the need for the more advanced technique, so the earlier plan probably needed to include a way to demonstrate a flaw with the existing knowledge, creating a conflict that can be resolved with the new methods. We discussed that this might be where to start the next lesson. However I also encouraged PG to go and have a look at the books to get a wider view than just those more vocal students. (This group have been using RAG123 regularly since November so I was hopeful they would have given some comments/reflections) 20 minutes later PG came back to see me...

Perception from the few, reality from the many
Having reviewed the whole class's books and RAG123 self assessments PG's view of the class, and of the lesson was vastly changed. It was clear from looking at the work actually done by the class that the majority of the students had engaged with the lesson much more successfully than either PG or the observer had thought.

I emphasise - the observer was an experienced teacher who had been present for the full lesson, and they had also formed the perception thought that the majority of the class had failed to engage with the learning. However the work done by the students proved completely the reverse - the majority HAD engaged; the perception was dominated by the vocal few who had been more resistant.

Vitally with this information PG is now able to plan activities for the next lesson that align with where the class actually ended the last one. This starting point is MUCH more advanced than would have been the case if the lesson plan is based solely on the perceptions they ended the lesson with. It also allows a more effective demonstration of the gap between the students who were resistant than those who actually engaged but were less vocal.

Fundamentally PG has moved from feeling really disappointed about the progress of the whole class to being much more focussed on taking action to secure better engagement and progress for the relatively few individuals who shaped the overall perception. It's gone from a whole class issue to an issue involving only a few students.

More than marking, RAG123 is TEACHING
I've seen occasional sceptical comments about RAG123 (from those not using it), suggesting that teachers should use more than just looking in books to judge pupil's understanding or detect misconceptions. I completely agree with this perspective - I've never suggested that RAG123 should be used to the exclusion of other assessment methodologies. Abandoning any one method to the exclusion of others is rarely a good strategy for anything.

What this tale really highlights though is that the perception left at the end of a lesson for both the teacher and the observer was VASTLY different to the reality as shown by the student's work. Clearly there are questions about use of AFL strategies during the lesson - perhaps the right plenary or assessment method would have shown this up; then again it might not. However there is no doubt that without RAG123 the next lesson for this class would have been MUCH less effective.

This is a demonstration that RAG123 isn't just marking. The strongest bit is that it gives a way to connect with the class that is based on fact (e.g. what have they actually DONE) rather than perception (e.g. how did the you think the lesson went?). Shaping planning in this way means differentiation is more effective, less time is lost in either catching up or waiting for the perceived progress of the class to reach the desired level.

I know that PG is now completely sold on RAG123 as a way to make sure planning is effective....

All comments welcome as always

Why does RAG123 work?

I've been pondering for some time why RAG123 has such impact. I know I like it because I get a better understanding of my students, and can plan more accurately as a result. But to have such a big impact on progress (e.g. to have an effect size of 0.6 or better), there must be a number of reasons that it works. Having thought for a while I think it hits many fronts...

The cognitive science view
In his book "Why don't students like school?" Daniel Willingham (@DTWillingham) says:

"You want to encourage your students to think of their intelligence as under their control, and especially that they can develop their intelligence through hard work. Therefore you should praise process rather than ability." (page 183)

(I should point out that Willingham was referring to "slow learners" in that passage, but there is no logical reason that this would only apply to those below average - it's just that they are less likely to be praised for ability so have more need of an alternative.)

The effort gradings in RAG123 do exactly this - We acknowledge effort, even if the learning itself isn't successful. If the student is applying a good level of effort then it is the responsibility of the professional teacher to ensure that the learning experiences the student encounters are effective. Conversely without effort from the student even an otherwise perfect series of lessons will result in ineffective learning.

As such using RAG123 to raise the visibility/prominence of effort in the student's mind will encourage them to improve. They have full control over effort, they may not have full control over learning.

Similarly as a teacher, if you become clearly aware of a student that is trying hard but not making much progress you will be prompted to make a change, it forces you to be more reflective.

The visible learning view

I mentioned effect sizes above and this is most closely linked to the work of John Hattie, e.g. here.

If you look at the top end of Hattie's ordered list of effect sizes a further possible explanation of the power of RAG123 starts to emerge...

Source: http://visible-learning.org/hattie-ranking-influences-effect-sizes-learning-achievement/ (highlighting added by me)

The highlighted line items, for me, have some kind of link to what RAG123 does. Self reporting grades is relevant because the students are self rating their understanding of each topic/lesson. Formative evaluation is done as the RAG123 rating shapes what both the student and teacher do in the next lesson, that also links to Feedback. RAG123 also impacts on Teacher-student relationships as it gives a much closer and immediate dialogue on the work done, the level of understanding shown, or the barriers to learning that have been found. Meta-cognitive strategies are about students having an appreciation of how they are learning, and therefore about what makes them successful learners. With the lesson by lesson feedback enabling students to link actions from a particular lesson directly to the learning outcome RAG123 encourages this link. Finally the feedback that the teacher gets from RAG123 means that there is a clearer link between what students have done and the teaching strategies they have used. As such it increases the chance of selecting more effective teaching strategies in the future.

Of course I have simplified this a fair bit - just taking the headings from the Hattie rankings and making links to RAG123, which may be a little tenuous at times, but I think the basic logic is sound though.

The teacher reflection view (i.e. it forces you to be a better teacher)

Using RAG123 forces a teacher to be more reflective. If a student is persistently putting in poor effort it becomes more obvious if you record that every day - it will prompt some kind of action. Similarly if a student simply isn't progressing despite effort then it's the teacher's job to fix it. If a lesson was really successful you have immediate feedback on it and can use that to roll into the next lesson, and the converse is true.

Just recently I have been trying to rate my own teaching with RAG123. (RAG for quality of planning - note this is distinct from quantity! 123 for success of the lesson) - not sure I've got the criteria nailed yet, and I do struggle to both be 100% reliable in the assessment of planning quality. Also it's hard to separate lesson success. However the reflection itself is powerful. 

I've not given myself any R1 ratings, but I have occasionally given G3, which then prompts me to consider if G3 is actually possible as surely a high quality plan must be successful? Otherwise it's not a high quality plan, or is it something else? I've also rated myself a few A1s, and again I start to wonder if the planning was really that average if the lesson was a success, but then how much better might it have been with better planning? Overall doing this has spurred me on to do more G level planning rather than A, simply because it prompts me to be more conscious of it.

Still no negatives

Finally I just want to observe that I've still NEVER found anyone who has tried RAG123 that doesn't like it and doesn't want to continue. But I do find loads of people who haven't tried it that are unsure that they could make it work. If you've never tried it then what's stopping you?

All feedback & comments welcome.

What can education learn from Quality Management?

Bit of a big post this - it's been evolving in my drafts file for a while now...

I learnt a lot about management and quality during 10 years working in the automotive industry. Due to the complexity of the products this is an industry that has been at the forefront of quality management since the 1950s. Cars are the most complicated consumer product in the world. They are built in vast numbers, require tens of thousands of components to work together when operated by relatively untrained drivers in a massive array of conditions. What's more failures have the potential to be both catastrophic and fatal. Also they are almost all built to a very tight budget, meaning that waste needs to be eliminated from all parts of the business to allow a car company to turn a profit.

The reliability of modern cars is truly gobsmacking, and it is due to the fact that the automotive industry are global leaders in the field of quality management. Versions of the approaches pioneered in the automotive sector are now deployed across manufacturing, and are even being used as models for business management in many non-manufacturing settings. However even the most forward thinking of these alternative applications are usually about 5-10 years behind the latest in the automotive world.

I'm sure having got this far you're now thinking "Kev's lost it, he's gibbering about cars, I thought he was a teacher and this was a blog about education?" Well you might be right! But I really do think there is much to learn about the idea of quality in education.

Don't panic, I'm not about to insist that schools are like production lines, propose time & motion studies or some hideous Fordian uniformity to classrooms; I guess I need to explain where this is coming from in terms of how quality management evolved... (but if you want to cut to the chase then scroll down to the "lessons for education" heading)

An evolution of practice

All quality assurance or management systems currently in place in any industry today have a lineage that traces back to the automotive sector in post war Japan. Before this point it was all about "quality control" - essentially inspection at the end of the production line to check that things were correct. This was ok, but it was only partially effective at catching all potential problems. You physically can't check for everything. As a result some things slipped through the net and caused issues in the field.

From control to assurance

To improve on the partially effective "control" system the emphasis shifted to "quality assurance". The premise was that rather than inspecting at the end of the line where tests were limited and rectifying errors was expensive and took a long time, why not do that inspection earlier in the process? Perhaps even before the parts are fitted? Or even delivered? The automotive firms pushed chunks of their inspection processes back up the production line all the way into their supplier's factory. The idea being that if all the parts arrived certified as "good" then the resulting car must be "assured" to be good. They still inspected a smaller number at the end of the line, but quality had improved as faulty parts were often detected before they were fitted.

However there was still a level of variability inherent in the design. Humans build it for a start, and we make mistakes! Even certified parts have to be supplied to geometric tolerances that cause variations, and other physical, chemical or even biological variations can creep in when you make large numbers of components or large numbers of vehicles. (Biological - really? Well for example loads of car components are made from rubber, which is a natural product. It literally grows on trees! As such, until relatively recently when it has become better understood and controlled, rubber components on cars were subject to variability depending on the season in which the natural rubber was harvested! As another example the quality of some car paint finishes can be affected by the type and quantity of deodorant used by the operators working in the paintshop!)

Error proofing

The next development in quality was to start to manage it from the outset. To do things with the design that prevented errors, or make the performance of the completed vehicle tolerant to variability of its components. Japanese terms like "Poka-yoke" are now commonplace in car design - it means "mistake proofing" and helps to remove human errors on the production line. For example, if an operator has to connect 3 different electrical plugs in the same area of the car each plug should be designed such that it only connects to it's correct socket, leaving no room for human error.

Assurance becomes management

By taking the focus away from inspection/control, wherever it is in the process, and looking in more detail to the systems and processes quality becomes managed rather than assured. This means designing OUT variability, designing IN error proofing, planning for quality from the very start of the process rather than applying it as an inspection at some point.

Continual improvement requires empowerment.

However perhaps the most powerful thing to come out of the Japanese automotive industry was the concept of continuous improvement, and with it the empowerment of everyone in the business to make suggestions to improve the product. This is often referred to by the Japanese term "Kaizen" (literal translation "improvement" or "act of making bad points better"). Toyota used this word to brand improvement activities in its factories and visiting western engineers and managers adopted the word.

At the heart of Kaizen is a philosophy that improvement must be lead from the top, but not directed from the top. Every worker in the factory has their part to play in the quality of the end product, and as such every worker has the right to make suggestions about how to improve it.

Vitally this includes the idea that the person that fits the brakes all day becomes an expert in fitting brakes. As such this person is very well placed to make suggestions about how to minimise errors when fitting brakes. This applies across the whole vehicle and as a result the shop floor assembly workers have a big voice in improving designs and optimising the processes.

LESSONS FOR EDUCATION
Firstly, we're currently broadly applying the "quality control" type of management. We inspect at the end of the process, both by assessing student's progress through final high stakes exams, and by the use of increasingly high stakes observations/assessments for teachers (I say increasingly high stakes due to imminent explicit linkage to pay structures in the UK), and high stakes inspections for schools.

The best schools will use more of a quality assurance model. "Good" practice will be embedded in school and departmental policies to reduce variability in practice between staff. However too often these are policed and enforced through inspection (e.g. observations, work scrutiny, learning walks). To take this on to the next level the structures need to be put in place to make good practice, and therefore success of the students, inevitable.

Developing systems in which good performance becomes inevitable can only come if the people doing the processes are inspecting it themselves. It becomes less about doing it well because you are being watched, and much more about not needing to be watched because you are watching yourself.

Ok that last paragraph sounds like a load of idealism, but if we pick up the Kaizen model in education and truly empower teachers to improve their practice they will feel a much greater ownership of it. By encouraging this ownership we make it much more likely that they will do it.

For example, who is best placed to formulate a marking policy that is workable for a teacher with a full mainscale timetable? It certainly isn't best done as a decision by someone that only teaches a partial timetable. Setting out a basic framework that includes the key characteristics of good marking and then asking teams of all staff to develop a way that this can be done in a manageable way would create a policy much more likely to be adhered to.

Just this week I was told about someone who wants to try #RAG123 marking (not heard of RAG123? It's awesome - see here!). They aren't allowed because it doesn't conform to their school's policy. Under their school's policy this person recently spent over 2 hours marking just 7 books, and they are saying that the quality of their planning is suffering due to the volume of marking they have to do! Nobody on a full timetable could possibly sustain that type of marking alongside teaching, planning and having a life. This is a clear example of a solution being imposed on people without thought to actual delivery.

Similarly it is really important that there is a route and process for all staff to highlight where the school is not working efficiently. For example are there flaws in the school sanctions and rewards system meaning that a group of teachers are struggling to use them effectively? Feedback loops are important in industry, and should be in education too. Vitally though if feedback is sought and given there MUST then be action with support from the top to address the concerns and improve the situation.

Where is the value added?
It is recognised in the automotive industry that the only people actually adding value are those building the cars. They take the components and combine them into something that can be sold at a profit. Everyone else and every other process in the organisation is an overhead that chips away at profits. They may be absolutely needed as part of the long term business, but they still cost money. As such these other processes need to be as efficient as possible, and mustn't interfere with the effectiveness of the production line.

In schools it would be too simplistic to suggest that the only people that add value are the teachers, as it's the total experience at school that's important, and not always just the lessons or exam results. However systems in the school absolutely mustn't make the jobs of those that interact with students harder to do well.

Consider how easy it is to get accurate data about a specific student... Attainment, targets, behaviour, attendance, SEN, FSM, IEP, etc, etc, Is it all in one place or in lots of different places? Is it easy to download a class list of information in a usable format? I know I've worked in schools where each bit of data is stored in a different place, and in different formats.

I've heard regularly that schools do well by prioritising on Learning and Teaching. The question "If it's not improving Learning and Teaching then why are we doing it?" pops up as part of this kind of thing. However how often is that really applied across ALL systems in a school? It may well be applied to guiding CPD, or some directed time activities and meeting agendas, but is the attendance system actually optimised to stop it interfering with learning and teaching? Is the behaviour system an add on administrative activity or an integral part of learning and teaching?

Loads of education practice, particularly on the administrative side, is based on finding a system that basically works, and then iterating as needs change. Sometimes this creates a real monster of a system with add on bits and extra files all over the place. For example all schools I've been in have slightly different ways of managing student data, sometimes this is based on the specific skills or preferences of the staff involved in creating them, or just on how it's been done for years. Sometimes these systems are brilliant, other times they are ineffective. New staff come in and have to learn the foibles of a particular system, and then all the various "work around" methods to get key bits of data in the right format.

It is incredibly rare to find a system that has been completely designed from the ground up to do its job in a way that is completely aligned with the needs of all of the users in the organisation. Process mapping and optimisation of processes are effectively alien terms in education, but they really shouldn't be.

In summary - we need to start actively managing quality
Basically what I'm trying to illustrate is that any push for improving "quality" within a school needs to aim far more at the quality management end, which is the cutting edge of quality practice; as opposed to the quality control end which is a blunt and inefficient instrument.

We need less direct inspection to enforce systems from the outside, and more design of systems to make good performance inevitable. We mustn't invent extra processes to fix problems; instead we should develop systems that simplify the job rather than making it more complicated.

Like choosing to walk across the grass or around the path, people only deviate from policy because there is a shortcut. We need to seek out and use the expertise of the people that will actually work with the policies the most to help redesign them to eliminate the shortcuts! If we make it hard to do the right thing then we can't be surprised if someone does it wrong. The purpose of good leadership and management must be to design and environment where we make it as easy as possible for our staff and students to do it right. That way success becomes inevitable.

As always I'd welcome any feedback and comments... :-)