Data on hospital acquired infections in Danish hospitals have recently been published as part of the HAIBA programme. HAIBA collects, aggregates, and presents infection data from all hospital departments in Denmark. Data are updated daily.

In this post I use data on hospital acquired bacteremia (HABAC) from the period 2013-2014 aggregated by month and region to demonstrate how different data visualisation techniques may lead to more or less useful conclusions.

The classic solution – simple, obvious, and wrong

Figure 1 displays HABAC rates by region in a “classic” bar chart sorted from high to low with 95% confidence intervals represented by whiskers. The vertical line represents the national average of 8.3 HABACs per 10,000 risk days.

Figure 1: Simple, obvious, and wrong

Figure 1: Simple, obvious, and wrong

Figure 1 clearly shows that HABAC rates in three regions, Syddanmark, Nordjylland and Sjælland, differ statistically significant from the national average. One may easily draw the false conclusion that these regions represent something special.

What is wrong with this conclusion? Basically, it is the right answer to the wrong question. The estimates of regional means may well be significantly different from the national average as shown by narrow confidence intervals. But for our purpose, that is not important. Remember that confidence intervals are measures of the uncertainty of estimates, and that confidence intervals approach zero as sample sizes approach the population size. However, our concern is not whether estimates of regional HABAC rates differ but whether these rates may come from the same process.

Two kinds of problems – enumerative and analytic

The distinction between sample variation and process variation was discussed at length by W Edwards Deming who used the terms enumerative and analytic to describe two very different approaches to statistical analyses:

Enumerative: in which action will be taken on the material in the frame studied. The action to be taken on the frame depends purely on estimates or complete counts of one or more specific populations of the frame.

[…]

Analytic: in which action will be taken on the process or cause-system that produced the frame studied, the aim being to improve the practice in the future.

– W Edwards Deming (1975). On Probability As a Basis For Action. The American Statistician, 29(4).

The distinction between enumerative and analytic uses of data is subtle but of vital importance.

Briefly, the enumerative question is how many? The analytic question is why? is there any difference between the two classes, and if so, how big are the differences?

– W Edwards Deming (1953). On the Destinction Between Enumerative And Analytic Surveys. Journal of The American Statistical Association, 48.

The right solution – plotting the dots

In order to understand the why-question and to predict future outcomes we need to study, not single estimates of population characteristics, but the processes that produced these estimates.

A process is a series of related activities that take input and produce output. For example, the HABAC rate is an output from a highly complex process involving (but not limited to) patients, bacteria, staff, physical environment, procedures, and use of antibiotics.

Understanding (and predicting) a process always begins with plotting the dots. By placing repeated measures from a process in time order in a run or control chart allows us to decide whether the process is stable, i.e. predictable, or not. When a process is stable, the data points will be randomly distributed around a centre value and varying within limits that characterises the common cause (random) variation, which is inherent in all processes. If, on the other hand, the process is unstable we may observe non-random patterns in the distribution of data points around the centre line or that one or more data points are outside the control limits.

Figure 2 displays HABAC rates from the five Danish regions in control charts.

Figure 2: Stable HABAC rates in Danish regions

Figure 2: Stable HABAC rates in Danish regions

Each chart represents a stable process – the data points are randomly distributed around the centre line (mean), and no data points fall outside the control limits.

Since HABAC rates in all five Danish regions come from statistically stable processes, we may compare the means of these processes with a control chart using regions (rather than time) as subgroups. In case one or more regions had unstable HABAC rates, it would not have made sense to compare these to the rest.

The control chart in figure 3 confirms what our eyes already concluded from figure 2: Since all data points are within the control limits, the variation between regions is well explained by common causes, and we have no reason to believe that any one region represents anything special (with regards to HABAC rates).

Figure 3: Comparable HABAC rates between regions

Figure 3: Comparable HABAC rates between regions