There is ample evidence to suggest that trauma, particularly early in life causes a wide range of bad things later on, from mental illness to smoking or physical inactivity.
One important study by Felliti and colleagues asked people whether they had experienced any of a range of adverse childhood experiences (ACEs), such as childhood sexual abuse or witnessing domestic violence. They found that the more adverse childhood experiences people reported, the greater their risk of depression, smoking, physical inactivity, and obesity.
This has led to calls for screening for adverse childhood experiences. For example, the DocsForTots website explicitly suggests that those with four or more ACEs should be referred to mental health services whether or not they have symptoms.
But just because an exposure, like ACEs, increases the risk of some Bad Thing, like depression, doesn’t mean it will be a good basis for a screening tool.
There are a number statistics that we can use to understand how good a screening tool is:
Let’s look at how the ACEs questionnaire does on these measures.
We’ll use the numbers from the Felliti et al (1998) paper I mentioned earlier. That paper looked at a range of outcomes. We’ll use the data for depression (two weeks of depressed mood in the last year). These numbers can be found in table 4 on page 252 of the print paper. I’ve reproduced the important numbers in the table below:
| ACEs | n | prevalence |
|---|---|---|
| 0 | 3799 | 0.142 |
| 1 | 1984 | 0.214 |
| 2 | 1036 | 0.315 |
| 3 | 584 | 0.362 |
| 4+ | 542 | 0.507 |
This table clearly shows that increasing numbers of ACEs are associated with a higher prevalence of depression in the past year: 14% of people who reported zero ACEs had experienced an episode of depression in the past year, whereas about 50% of people with four or more ACEs had depression in the last year.
But does this mean that the ACEs questionnaire is a good screening tool for depression? Right away we should have doubts: only about half of those with four or more ACEs report an episode of depression in the last year.
To calculate the statistics mentioned above, we need the actual numbers in each group with and without depression. We can calculate this by multiplying the numbers in each group by the proportions reported:
| ACEs | n | prevalence | depression | no_depression |
|---|---|---|---|---|
| 0 | 3799 | 0.142 | 539 | 3260 |
| 1 | 1984 | 0.214 | 425 | 1559 |
| 2 | 1036 | 0.315 | 326 | 710 |
| 3 | 584 | 0.362 | 211 | 373 |
| 4+ | 542 | 0.507 | 275 | 267 |
A quick look at this table illustrates an idea called the ‘prevention paradox’ nicely. This is when, even though the prevalance of depression increases with the number of ACEs, almost third of the cases of depression report no ACEs and 85% of cases of depression in this study reported fewer than four ACEs. This doesn’t bode well.
We can calculate the main statistics using the formluae below (where ‘true positives’ are people with 4+ ACEs who report depression, ‘true negatives’ are people with 0-3 ACEs who report no depression and so on):
\(sensitivity = \frac{true \: positives}{true \: positives \:+\: false \: negatives} = \frac{275}{275 + 1501} = 0.155\)
\(specificity = \frac{true \: negatives}{true \: negatives \:+\: false \: positives} = \frac{5902}{5902 + 267} = 0.957\)
\(PPV = \frac{true \: positives}{true \: positives \:+ \:false \: positives} = \frac{275}{275 + 267} = 0.507\)
\(NPV = \frac{true \: negatives}{true \: negatives \:+\: false \: negatives} = \frac{5902}{5902 + 1501} = 0.797\)
So using 4+ ACEs as a screening tool for depression in this sample only detected 15% of people who had experienced depression in the last year. As we noted earlier, only about half of those screening ‘positive’ on our test actually had depression, while one in five of those screening negative had experienced depression. If this tool were used for screening for depression a lot of people would be treated unnecessarily and quite a few who needed treatment would miss out.
So how bad is this? The original ACEs questionnaire, in the population studied by Felliti et al, performed worse than 2-3 question ultra-short depression screening tools (these appear to be more sensitive but have lower PPV). The Canadian Task Force on Preventative Health recommends not screening for depression in primary care. Given that, it would be very hard to recommend using the ACEs questionnaire to screen for depression either.
There are some caveats: this analysis is based on the original ACEs questionnaire, and subsequent authors have added items to it, which will probably affect its performance as a screening tool. It’s also a specific population, and the tool may perform differently in a different population. I may repeat this analysis using data from more recent studies if I’ve got the time.
None of this is to say that preventing childhood trauma and adversity isn’t really important. Of course it is. Or that we shouldn’t be sensitive to the way trauma affects people and design services that work for people who have suffered trauma at any time in their lives. We definitely should. It’s just that we need to be cautious about using ACEs questionnaire’s as a screening tool without doing the sums first.