Chapter 22 The Forgotten Art of Rational Subgrouping
A subgroup consists of the data elements that make up a single data point, for example, the waiting times used to calculate the average waiting time for a particular period.
Rational subgrouping is the intentional and intelligent sampling and grouping of data into data points for SPC charts with the aim of maximising the chances of detecting special cause variation while minimising the risk of false alarms. In other words, rational subgrouping is all about maximising the signal-to-noise ratio.
A rational subgroup consists of a set of measurements or counts that are:
- produced under conditions that are as similar as possible,
- taken close together in time and space, and
- likely to show only common cause variation.
The underlying logic is that when subgroups are rationally formed, variation within subgroups reflects common cause variation, whereas variation between subgroups that exceeds what is expected from within-subgroup variation indicates the presence of special causes.
In our experience – particularly in healthcare – rational subgrouping is something of a forgotten art. We often rely on whatever data are readily available, which are frequently pre-aggregated into monthly, quarterly, or even yearly time periods for administrative purposes rather than for quality improvement. This practice often leads to suboptimal use of data for driving improvement.
22.1 Too large subgroups — masking meaningful signals
A common mistake is forming subgroups across overly broad spans of time or space (e.g. organisational units), which blends common and special cause variation and effectively obscures the latter.
As an example, figures 22.1 and 22.2 display the number of C. diff. infections subgrouped by monthly and two-monthly periods respectively.
qic(month, infections,
data = cdiff,
chart = 'c',
title = 'C. diff. infections',
ylab = 'Count',
xlab = 'Months')
Figure 22.1: Monthly C. diff. infections.
qic(month, infections,
data = cdiff,
chart = 'c',
x.period = '2 months',
title = 'C. diff. infections',
ylab = 'Count',
xlab = 'Two-months')
Figure 22.2: Two-monthly C. diff. infections.
By using larger subgoups, as shown in Figure 22.2, we effectively mask the two signals – a freak value and a sustained shift – that suggest the process is trending downwards. Clearly, if the trend continues, it will eventually become apparent even with two-monthly data, but it will inevitably take longer to detect delaying any potential learning or intervention.
To avoid over-aggregating data by using excessively large subgroups, we need to record data at a temporal resolution that, at minimum, matches the expected rate of change we are trying to detect. High-resolution data can always be aggregated to a lower resolution if suitable (or necessary), but low-resolution data cannot be further resolved.
Note that the qic() function from qicharts2 includes an argument, x.period, which allows us to aggregate data into larger subgroups as demonstrated in the code producing Figure 22.2.
22.2 Too small subgroups — revealing unimportant noise
A less common – but equally important – mistake is to form subgroups that are too small, which fail to capture natural process noise and thus misinterpret it as special cause variation..
Imagine stepping on a scale three times each morning for a couple of weeks and plotting the results on an X-bar chart, using the within-subgroup (within-day) variation as the basis for calculating control limits.
Figure 22.3 illustrates this with simulated data. As expected, the variation within each daily subgroup is very low, while the variation between subgroups is noticeably higher suggesting that our body weight is highly unstable from day to day. However, this difference between within- and between-subgroup variation merely reflects natural physiological fluctuations and is irrelevant to our goal of monitoring long-term body weight trends.
# lock random number generator for reproducibility
set.seed(5)
# subgroups, 12 subgroups of three values
x <- rep(1:12, each = 3)
# random values, each repeated three times
y <- rep(rnorm(12, mean = 80, sd = 0.5), each = 3)
# add a bit of random noise within subgroups
y <- jitter(y, amount = 0.2)
# plot Xbar-chart
qic(x, y, chart = 'xbar')
Figure 22.3: Xbar-chart from too small subgroups.
A better strategy would be to plot data using an I-chart of the daily averages. The I-chart uses the moving range between successive subgroups as a basis for calculating control limits, which is – in this case – a much better representation of the day-to-day variation, which is really what we want to monitor.
## Subgroup size > 1. Data have been aggregated using mean().
Figure 22.4: I-chart of average daily body weight.
Alternatively, we could devise a different sampling plan that produces subgroups better representing the process of interest – for example, using weekly averages of daily measurements as subgroups.
Depending on our goal, different subgrouping strategies may be appropriate. If the aim is to monitor and maintain a stable body weight, weekly subgroups may be sufficient. However, if the objective is to support a weight-reduction plan expecting significant weekly decreases, daily measurements could prove more useful.
22.3 Striking the balance
As the chapter title suggests, rational subgrouping is as much an art as it is a science. In fact, it remains one of the more complex and unresolved challenges in statistical process control: to understand a process, we need rational subgroups – but to form rational subgroups, we need to understand the process.
In practice, we frequently need to try multiple approaches to subgrouping our data – experimenting with different timeframes, organisational units or other dimensions – before finding an approach that meaningfully reflects the underlying process behaviour.
That said, in healthcare – in contrast to the production industry – there is often far less flexibility to tailor data collection to the requirements of SPC. In a production setting, where outputs are counted in the thousands and inputs are largely under control, it is possible to design highly specific sampling plans aligned to precise time intervals and structured across machinery, personnel and organisational units.
In healthcare, by contrast, the smallest meaningful unit is typically the individual patient or procedure, situated within a specific organisational context or clinical pathway. Patient outcomes are shaped by a complex interplay of factors – including clinical complexity, social determinants, people, and system-level variation. Unlike components in a production process, this variability cannot be standardised or controlled with the same degree of precision.
However, that does not mean that we should not care about rational subgrouping. Although we may have limited control over input variation we can still design sampling strategies that aim to minimise the impact of these sources of variation by carefully selecting patient groups, clinical pathways, organisational units and time periods that reliably reflect the processes we aim to understand. Doing so helps preserve an optimal signal-to-noise ratio, enhancing our ability to detect meaningful changes.
22.4 A practical approach for rational subgrouping
There is no single ideal procedure for designing rational subgroups. As noted previously, this often requires an iterative process involving trial and error: Plan - Do - Study - Act.
Here are a few suggestions for points to consider when designing rational subgroups:
Consult individuals with expert knowledge of the process in question – rational subgrouping is a collaborative effort, not merely a statistical exercise.
Develop a detailed model of the process – including inputs, outputs, and factors that may influence either.
Consider whether your objective is monitoring or improving the process.
Consider the level of organisational granularity at which the data should be analysed, and whether aggregation or stratification is appropriate.
Consider the rate of change you aim to detect.
Choose a timeframe that is shorter than the expected rate of change.
For count data, select a timeframe expected to capture at least five events or cases (numerator), and, for proportion data, ensure there are at least five additional opportunities (denominator).
Remain flexible and open to revising your sampling strategy as new insights emerge.
Note that decisions about temporal and organisational granularity (#4–6) may conflict with the requirement for sufficiently large subgroups (#7) to ensure meaningful analysis. In such cases, it may be necessary to compromise or explore alternative methods to monitor the processes of interest – for example using different indicators or charts better suited for rare events.
22.5 Rational subgrouping “in short”
Despite the section header, there is no truly concise way to explain rational subgrouping. It is one of the most challenging aspects of SPC to understand and apply correctly – not least because there is no single recipe for getting it right.
Still, there is no excuse – getting rational subgrouping right is what separates meaningful SPC charts from those that mislead or fall short.