Chapter 3 Looking for Signals on SPC charts – Beyond the 3-Sigma Rule
So far, we have focused primarily on Shewhart’s 3-sigma rule, which identifies special cause variation when one or more data points fall outside the control limits.
The 3-sigma rule is particularly effective at detecting large shifts in data. To illustrate this, consider a process where the data follow a normal distribution.
If the process suddenly shifts upward by three standard deviations (SDs), the previous upper control limit (UCL) effectively becomes the new centre line (CL). In this situation, approximately half of the subsequent data points will fall above the old UCL, providing a clear signal that the process has changed.
If the shift is smaller – for example, one standard deviation – the previous UCL corresponds roughly to the new 2-sigma limit. In this case, only about 2.5% of the data points will exceed the old UCL. The shift is still detectable, but signals occur much less frequently.
The strength of the 3-sigma rule lies in its ability to detect large and clearly distinguishable changes in process behaviour. However, smaller and more gradual shifts may remain undetected for extended periods. To improve sensitivity to such changes, additional rules and techniques are often used (Figure 3.1).
Figure 3.1: Control chart with progressive shifts in data
The performance of the 3-sigma rule has been studied extensively (see, for instance, Montgomery (2020) or Anhøj and Wentzel-Larsen (2018)). In general, it is most effective for detecting shifts of approximately 1.5 to 2 standard deviations or larger. Smaller shifts may go undetected for some time. Before introducing additional rules, it is helpful to consider the types of patterns that often signal the presence of special causes.
3.1 Patterns of non-random variation in time series data
The Western Electric Handbook (Western Electric Company 1956) describes a range of patterns that can assist in the interpretation of control charts. These patterns are based on the observation that certain data behaviours are often associated with specific causes.
In our experience, the most common patterns encountered in healthcare data are freaks, shifts, and trends.
3.1.1 Freaks
A freak is a data point or a small number of data points that are distinctly different from the rest of the data (Figure 3.2). By definition, freaks are transient – they appear suddenly and then disappear.
Figure 3.2: Control chart with a large (2 SD) transient shift in data
Freaks are often caused by data or sampling errors, but they may also arise from temporary external influences on the process, such as changes in patient case mix during holiday periods. Occasionally, a freak may simply represent a false alarm, which is to be expected from time to time.
3.1.2 Shifts
Shifts represent sudden and sustained changes in process behaviour (Figure 3.3).
Figure 3.3: Control chart with a minor (1 SD) sustained shift in data introduced at data point #16
In healthcare improvement, the aim is often to create shifts in the desired direction by changing the underlying structures and processes that generate the data.
3.1.3 Trends
A trend is a gradual change in the process level over time (Figure 3.4).
Figure 3.4: Control chart with a trend in data
While some definitions restrict trends to sequences of continuously increasing or decreasing points, we use the term more broadly to describe any gradual movement in one direction.
Trends typically arise when external influences act consistently over time. This is often observed when improvements are introduced incrementally. For example, small changes at the department level may accumulate into a gradual system-wide improvement.
3.1.4 Other unusual patterns
Many other types of patterns may occur in time series data. However, in healthcare settings, changes in performance are most often associated with freaks, shifts, or trends. Recognising these patterns is an important step in understanding process behaviour. In some situations, other patterns may also be relevant. For example, cyclic or seasonal variation may produce recurring patterns linked to time of day, week, or year (Jeppegaard et al. 2023). Such patterns may not appear as freaks, shifts, or trends, but they can still be important for understanding and improving processes.
3.2 SPC rules
If patterns of special cause variation were always as clear as in the figures 3.2-3.4, there would be little need for formal SPC rules. In practice, however, special causes are often less obvious. For this reason, we use statistical tests – commonly referred to as SPC rules – to help identify patterns that are unlikely to occur by chance.
A large number of such rules have been proposed. While it may be tempting to apply many rules simultaneously, doing so increases the likelihood of false alarms, where common cause variation is mistakenly interpreted as special cause variation.
It is therefore important to strike a balance: using as few rules as necessary to detect meaningful signals while limiting the number of false alarms. This issue is discussed further in Appendix C.
In this book, we focus on two sets of rules that have been well studied and shown to perform effectively.
3.2.1 Tests based on sigma limits
One of the best-known sets of rules is the Western Electric (WE) rules (Western Electric Company 1956). These consist of four tests based on the position of data points relative to the centre line and control limits:
- One or more points outside the 3-sigma limits
- Two out of three successive points beyond a 2-sigma limit
- Four out of five successive points beyond a 1-sigma limit
- Eight successive points on the same side of the centre line
The first rule corresponds to Shewhart’s original 3-sigma rule. The remaining rules increase sensitivity to smaller shifts.
The WE rules have been widely used for decades. However, their performance depends on the number of data points: with fewer observations they may miss signals, and with many observations they may produce more false alarms.
The fourth rule is independent of sigma limits and is based on the concept of runs, which we consider next.
3.2.2 Runs analysis – tests based on the distribution of data points around the centre line
Runs analysis is based on the distribution of data points around the centre line. If the centre line divides the data into two equal halves, the probability that a point lies above or below the centre is equal.
A run is a sequence of consecutive points on the same side of the centre line. A crossing occurs when two consecutive points lie on opposite sides.
In a random process, the number and length of runs follow predictable patterns. When a process shifts or trends, runs tend to become longer and fewer. This observation forms the basis for runs tests.
What constitutes an “unusually” long run depends on the total number of observations or subgroups. The more subgroups we have in our SPC chart, the longer the longest runs are likely to be.
Based on theoretical work and practical experience (Anhøj and Olesen 2014; Anhøj 2015; Anhøj and Wentzel-Larsen 2018), we recommend two run-based tests:
Unusually long runs: The longest run exceeds approximately log2(n) + 3, where n is the number of useful data points (Schilling 2012).
Unusually few crossings: The number of crossings is below the lower 5% limit of a binomial distribution (Chen 2010).
These two measures are closely related: as runs become longer, crossings become fewer.
Critical values for longest runs and number of crossings for 10-100 data points are given in Appendix E. For example, in a run chart with 24–26 useful observations (i.e. data points not on the centre line), a longest run of more than eight points or fewer than eight crossings signals a shift in the data.
In Figure 3.3 the longest run exceeds the expected limit, indicating a shift. In Figure 3.4, long runs and few crossings similarly suggest non-random behaviour.
It is important to note that these tests indicate the presence of non-random variation but do not explain its cause. Interpretation requires knowledge of the process and context.
3.3 SPC charts without borders – using run charts
Some SPC rules rely only on a centre line. If we remove the control limits and use the median as the centre, we obtain a run chart.
Figure 3.5: Run chart
Figure 3.5 shows a run chart of the data from Figure 2.2. Because the median is used, the data are evenly divided around the centre line. The runs analysis shows no unusually long runs or unusually few crossings, indicating no evidence of persistent shifts.
Figure 3.6 shows a run chart with a trend. The runs analysis confirms the presence of unusually long runs and few crossings.
Figure 3.6: Run chart with a trend
Run charts are simple to construct and do not rely on distributional assumptions. They are particularly useful for detecting moderate and persistent changes in process behaviour.
Control charts, however, provide additional information. They are effective at identifying large, isolated deviations and define the expected range of common cause variation.
Run charts and control charts therefore serve complementary purposes.
3.4 A practical approach to SPC analysis
SPC charts are used for two related purposes: process improvement and process monitoring. In process improvement, the goal is to achieve sustained changes in performance. In this context, shifts are expected, and runs analysis is particularly useful for detecting them.
In process monitoring, the aim is to maintain stability. Here, the focus is on detecting deterioration as early as possible. The 3-sigma rule is effective for this purpose, and additional rules may be used to increase sensitivity.
Regardless of the purpose, it is often useful to begin with a run chart using the median as the centre line. If the data show evidence of non-random variation, the focus should be on understanding the causes of change. If the data appear stable and the outcome is satisfactory, control limits may be added to support ongoing monitoring.
3.5 SPC rules in summary
SPC rules are statistical tools used to detect non-random variation in time series data. While many rules exist, using too many increases the risk of false alarms. A small number of well-chosen rules can provide an effective balance between sensitivity and specificity.
In this chapter, we have introduced the 3-sigma rule, the Western Electric rules, and runs-based tests. Together, these provide a practical framework for identifying signals of special cause variation.