Imagine that we have got a SMART aim, worked up our driver diagram, developed our set of measures (outcome, process and balance) and collected some data to plot on a series of charts on Life QI.
We may have also gathered sufficient data to be able to establish a baseline of how our system is working before we start to change things. Baseline data is extremely important to allow us to gauge the effect of our improvement work, not forgetting that we need a minimum of 10 data points of baseline data to be able to establish a statistically valid baseline mean!
So what next? We want to answer the question ’How will we know that a change is an improvement?’ in the Model for Improvement, so we need to deploy our knowledge of how to interpret or ‘read’ our data when it is plotted out on a run chart.
To do this, we need to know how to spot when special cause variation is occurring in our data. All data will randomly vary over time. This is a naturally occurring phenomenon where we see a ‘saw tooth’ pattern to data when we plot it on a chart over time. Special cause variation occurs when we have made some effect on the data, such as when we have introduced a change idea and hence changed our system. It is the special cause variation that we need to look out for. Indeed, if we are trying to improve something, we want to see some change in our data to show that our changes are having an effect!
There are four common ‘rules’ that tell us that we are seeing special cause variation; the good news is that when plotted on Life QI, the system provides us with alerts to show when a rule has triggered and special cause variation is present. But it is necessary to appreciate what the rules are and why each is significant, so let us look at each of the rules in turn.
Why is the median value so important?
To interpret our run charts we need to draw a median line on the chart. The median is the middle value of all our data; importantly it is not the average (known as the mean). This is because the average value of our data is more sensitive to ‘outliers’ or unusually big or small data points and this would skew our interpretation of the chart and what the data is doing. The median value is less sensitive and therefore we get more reliable identification of that special cause variation that we are looking for in our data. The median is important because the rules are based on statistical probability that our data is or is not behaving in a random way. There are lots of statistics behind this based on normal distributions of data, but thankfully we don’t actually need to know what all that statistical calculation is because it is done for us in Life QI. We do not have to become statisticians to be good at using data in our QI work!
Rule 1: A Trend
5 or more consecutive points increasing or decreasing
Probably the easiest to think about, if the data is ‘going in one direction’, then it is definitely being affected. We need to see a minimum of 5 data points in an ascending or descending order for this to be statistically indicating special cause variation, but we then need to ask ourselves which is the ‘direction of good’. If the data is going in the right direction, then our changes are helpful and we should do more or continue with them, but if the data is moving against the ‘direction of good’, we need to stop or abandon our changes.
Rule 2: A Shift
6 or more consecutive points above or below the median
A shift in your data is when the data points consistently move or ‘shift’ above or below the median line. We need to see at least 6 data points that are all in sequence on one side of the median, so the data is not crossing the median during a run that is classed as a shift. If a shift happens in our data after we have started a PDSA test of a change, we can get very excited because it is highly likely that the changes we make are causing the shift.
Rule 3: Astronomical Point
Data points that are obviously, or even blatantly different from all or most of the other data values
Astronomical points are data points that do not seem to fit with all the other data we have plotted on a run chart. They are literally astronomically or greatly different (higher or lower) than the other data points. In this sense, astronomical points are most easily spotted by just looking at your run chart; they can ‘stand out like a sore thumb’! Of course, there is a statistical reason that we can identify a data point as an astronomical one and Life QI identifies them for us. When we see one, we need to think about what might have happened to cause it and we can usually explain something that happened ‘out of the ordinary’, for example, ‘we had three staff off sick that week’ or ‘we had 10 extra referrals to deal with on that day’. Most often, astronomical points should not concern us unless they keep happening, which would indicate that there is instability in our system and we might want to investigate the reason behind that.
One question that is asked frequently is “should we replot our data without the astronomical point(s)?’ The answer to this is definitely ‘No’; the astronomical points are part of your data, and are part of the description of how your data and system are behaving, so you need to keep them in your run chart.
Rule 4: Too few or too many runs
A non-random pattern or signal of change is indicated by too few runs* or too many runs
*number of times the data line crosses the median and add one
A run is when the data points cross the median line, so data that is acting in a normal random way can be expected to cross the median frequently and for a certain total number of data points, there are statistical ‘norms’ of how many runs you should expect to see. Too few or too many runs are therefore another indication that your data is behaving in a non-random way and you are seeing special cause variation. So, once again, if your run chart is showing this rule of the number of runs being triggered, something is acting on your data to make it abnormal and you need to think about what that is causing the change. We would hope that the changes we are making in our PDSA cycles are the explanation to seeing too few or too many runs in our data.
So, we have four rules that when triggered show us that we are seeing special cause variation in our data. And any of the rules triggered should prompt us to think about what we have done or what has happened in our system to make the data behave differently. Bearing in mind what is the ‘direction of good’ for our data, we can see the effects of our change ideas as we plot our data.
Working with these rules can be tricky or befuddling at first, but the more you look for them and think about them, the easier and more comfortable they get. If you have data that you are wanting to understand and interpret, don’t forget that one of the quickest ways to get help and support is to either talk to a QI coach or to book into a QI Clinic slot and talk through your run chart with one of the QI Team. The team will be more than happy to help you. Contact us at cnw-tr.improvementsupport@nhs.net to book in to a QI Clinic slot.
Contact Information
For support or further information on any aspect of improvement work in CNWL, please contact the QI Team in the Improvement Academy at:
cnw-tr.improvementsupport@nhs.net