RAG123 user survey - the results!

I posted the RAG123 survey a few weeks ago and have now collected enough responses for it to be meaningful.

Don't know what RAG123 is? see here and here.

In total 40 people responded, which I know is fewer than the number that are actually using RAG123, but it represents those that saw the tweets about the survey and found the time to complete it - for which I am grateful as I know time is precious. 40 isn't a massive number, but it's enough to draw some conclusions on...

A quick health warning - these are the results of 40 responses - any statements made refer to the views of this sample only and shouldn't be extrapolated to wider populations. Also this was a USER survey - I've not got data from non-users, that wasn't part of the exercise. I'll also be clear that one of the responses is me - I'm a RAG123 user after all.

Profile of users
I was worried at one point that lots of people I communicate with about RAG123 appear to be departmental leadership of SLT. It made me wonder whether it is truly sustainable for a mainscale classroom teacher, however 50% of respondents are classroom teachers...

Profile of RAG123 users by job function

85% of respondents first heard about RAG123 via twitter, 5% heard from others in their school, and the final 5% are Rob Williams and I, who did the first trial at our school in November.

Subject coverage

The majority of respondents teach maths (60% if you include those who also teach another subject), and the next big group is science (25% if you include all who mention science). Humanities had 5% and the remaining few are individual teachers of other subjects.

Respondent's subjects

I wouldn't want anyone to draw a conclusion on this that RAG123 only really works for Maths and Science. Notably the two blogs/twitter feeds that have been pushing this idea are mine (Maths based) and Damian Benney's (here - Science based), as such it's hardly surprising that there is a bias here. What I am pleased by though is the fact that other subjects are represented, including the notable "Whole School." I do know from twitter exchanges that RAG123 has been used in MFL, Music, and a some other subjects too - it just happens that they didn't complete the survey.

Part of doing this survey was to collect some info on who was using RAG123 so we could share tips and best practice more directly. Users that included their twitter ID have been sorted by subject and can be found in these lists.

Impact on pupils

That's what we do it for after all!!

An overwhelming 82.5% report an improvement in either effort or attainment, or improvements in both. 10% are new RAG123 users and felt it was too soon to detect any changes, and the remaining 7.5% report no change on the part of the pupils.

Reported impact on pupils following RAG123 introduction

Impact on workload

One of the things I found when I started RAG123 was it improved my workload, I was interested to see what others thought...

Impact RAG123 has had on perceived workload

55% state their workload has decreased, 32.5% say there has been no change, and 12.5% state an increase.

Notably of those reporting increased workload all but one recognise improved pupil effort and attainment (the remaining one response is a "too soon to tell"). In the write in comments all of those with an increased workload are still very positive about RAG123. For example:

"Although I am not perfect at RAG123 and still have to do STAR, it has made marking so much quicker and actually I like doing it."

"My dept and I were sceptical and only did it cos u kept tweeting no negative feedback yet!!!! But we are sold!!"

In terms of where it has had biggest impact, 68% mention marking, 53% mention Dialogue and 60% mention planning (as these don't add to 100% you'll realise that many mention more than one of these!)

Best things about RAG123

This was a free text bit of the survey and the responses ranged from a few words to much more detailed. I could try to pick and choose best bits, but in all honesty it's best just to see the full text cut & pasted in here:

Picking up on misconceptions at source and the value pupils place on such regular marking. Also the way that informs your planning. Impossible to say just one.

Dialogue and relationships with students

Although my work load has increased as I am now taking books in every lesson for checking RAG123, it is a positive increase. I am able to judge how well my lesson has gone straight away. I can use the RAG123 to set targets more effectively and cater better for the individual. As a result DIRT happens every lesson now which I hope will pay off with regard to progress over time and stickability. My students are responding positively to me monitoring their progress so closely and a better dialogue has been established. If I find a smarter way of recording targets in exercise books so there is clear signposting of what is going on for observers, my workload should decrease in the future.

Informs future planning

The simplicity of it and the ease of use.

There are many best things, and the only downside is setting aside the time every lesson to make sure students do their part in it.

Marking is very quick. Pupils marking their own work a real game changer.

I know where everyone is after every lesson and can therefore plan for this in the next lesson. No one slips through the net; misconceptions identified readily (with more certainty than other AFL techniques) - there's nowhere to hide! No chance for bad habits to develop.

Communication with students

You can fully track progress of the whole class. I can identity misconceptions earlier and check students motivation.

Improved dialogue with students

Simple self assessment

It helps me identify what is not making a piece of work G1 and able to then identify where to improve.

Enables frequent marking and formative feedback.

Ability to plan effectively the next lesson and show progress.

quick whole class overview of progress and understanding

The students know that I am looking at the books very regularly and can write me messages that I will read.

I also get them to hand in their books in RAG123 piles so I can start with/spend more time on the students who need more help.

Students are getting used to assessing their own understanding, which I think will help with their revision.

I feel like I know my students better and what they have understood

Makes sure the kids complete their work - being able to keep on top of exactly what they are doing

No more marking guilt and amazing dialogue with students.

The ease, and the fact you know where the kids are after each lesson

quick feedback

Much more informed about planning. Kids love it and whilst I'm not sure any improved attainment is down to this I am convinced improved effort is. Combining it with pupils marking their own/each others work with green pens. We mark in red.

RAG123 is quick and extremely effective. I have mini dialogues with students in their books and can see patterns in behaviours as well and spot misconceptions quickly.

The best thing and what has helped me the most is being able to manage my marking load better. We have weekly book checks, one week yr7 and yr8 and the next week, yr 9 and 10. Before I was always in a mad panic about these. Now I know that although I may not be doing great at following the latest marking policy, my books are marked and feedback is there. (Especially those which I have still kept on top off!!) I know I'm not RAGging properly as I'm trying to squeeze too much feedback in, but it's much better than the paragraphs I wrote before! 

planning for next lesson, allows me to monitor how the students are doing on a regular basis. They self-regulate their effort often.

student focus on their own progress and effort

Checking work after each lesson and before the next one!

The opportunity to have dialogue with pupils. They enjoy doing it too..

Pupils are excited to read and respond to my feedback each day. The impact it's having is worth the extra effort!

It's instant and instantly useful. Supports using LO/SC in all lessons.

I know how everyone is doing and what they need to do to improve or correct misunderstandings.

Quick, easy for both learner and teacher. gives you indication on how class doing, useful for ensuring tailor made lessons.

Regular monitoring, review link to lesson objectives, planning response better informed.

Simplicity, focus on student involvement and it's evidence based system.


The easiness of marking. It helps me to keep on top of it. I now feel a lot more knowledgable about all of my classes.

The worst things about RAG123

For balance I also need to include all of the negatives - this is an unedited cut and paste of "the worst things" - I'll try to address some of the comments in another post:

It can be a pinch if you have parents' evening/meetings after school. Can be overcome though!!

That more people aren't using it!

Nothing! What I want to do is print the RAG123 criteria on a sticker and have it at the front of exercise books. That way I can then have the success criteria on the board linked to 123 which is something I have not yet been doing. I also need to get students to improve their justifications which will come once I have linked the RAG123 criteria to the success criteria.

Colleagues' reactions when you say you mark after every lesson.

Have to remember to mark after every lesson for it to be effective

Nothing

The students who don't mark work or RAG it. 

Those who just go for A2 every time to save actually thinking. Part of the reason that we are going to try red/orange/yellow/green so you are above or below half way.

Having so many books around at school, because I am not tidy and when I lose 1 book (no doubt student put in wrong pile) I have to look for it as I know that they handed it in. (Student then says oh yeah I forgot I have it because I didn't hand it in - grrrrr)

Not sure yet

It doesn't encourage students to make subject specific comments

Needs to be done very regularly.

Sometimes hard to summarise effectively into RAG123 categories

effort grades, personally I don't use them. Grading effort is unreliable.

Not found one yet.

not too sure yet

We also have to give SWANS feedback at least every three weeks, so I have to do that as well as RAG123.

It is sometimes difficult to get the books marked before the next lesson but it's worth it

Some students really don't like it"

Marking the books everyday

Making sure you do it before you go home...

Not sure yet

Still struggle to find time to mark after every lesson - but I put that down to pastoral responsibilities - those pesky kids and parents stop me doing most things when I plan to!!!

If I set homework then I miss a lesson or two with some classes. Maybe I could think about giving separately homework books, but my experience with these has not been good.

The pressure I feel when I've had a bad week and fallen behind. Have five lessons out of six most days, and sometimes struggle to RAG everyday before the next lesson. This is because I've not got it right yet, but I get very stressed and then fall even more behind!!

Need to make more of a glance and RAG123 thing. Have started timing myself now!!!

Doesn't work if you can't keep up checking every lesson - I've fallen victim to this.

students overestimating their understanding

Probably feeling the pressure to check after every lesson, especially when there are after school commitments like parents evenings etc

I have found that it doesn't lend itself to every lesson. 

I sometimes struggle to find the time for the students to do it properly so it ends up being rushed.

Daily expectation! We have probably gone a step too far with it! Setting individual questions, activities even card sorts etc. Viewed as an investment in next lesson rather than a quick response to previous. We are worried about sustainability though.

Sometimes harder to use in English where you're not always working on something as discrete as maths.

Can't think of any. It's easily the best thing I've done in my teaching career.

No real.negatives. some year 11 boys just sat A1 as it relates to breaking bad!

It doesn't record the volume of quality verbal feedback given in maths - but then neither do other written systems, the major issue with ofsted's version of marking and feedback monitoring.

Apathy of some to look at the benefits. Mainly, "you got this from twitter!" What do Ofsted think, well now I know.

I sometimes struggle to mark books every day. Especially on one week where I teach 4 days without a free period or lunchtime or after school.

So there you have it...

There is a bit more analysis to do, and I still need to sort out the top tips bit - there are some gems in there. However I wanted to get this post published this weekend...

Notably for me all of the worst bits are things people struggle with, not reasons to stop. Yes it can be difficult to do every day, yes it takes time for students to respond to it - we need to train them in how to use it and learn how to use it ourselves.

Still sceptical of RAG123? Give it a try!!!