Bayes and Damay’s

A new experimental treatment has developed for Damay’s disease. The cure is about 70% successful, but 30% of the patients die unpleasantly within a week after starting treatment. Autopsies show that people without Damay’s also die at a 30% rate from the treatment. Ordinarily, someone with untreated Damays disease will last about a year from the time of diagnosis. The last days of Damay’s are also unpleasant.

Until recently, there had been no blood test for Damay’ s. The diagnosis was based mostly on its symptoms: a rash on the left earlobe coupled with a serious cough. But a lot of people without Damay’s also have these symptoms. Indeed, it is believed that only 2% of patients with these symptoms have Damay’s. Others may have a respiratory infection and a localized skin rash. Now, thanks to the work of the Damay Foundation, a new and expensive blood test has been developed. It is believed that with people Damay’s test positive about 97% of the time whereas people who do not have Damay’s test negative 95% of the time.¹ Is it worth running the test?

Analysis

Here’s how I would analyze this in R. First I would figure out how many people tested positive on the new blood test. This will be the denominator of a fraction.

testPositive<-0.02*0.97+0.98*(1-0.95)

Then I would figure out how many people tested positive and were actually positive. This will be the numerator of the same fraction.

actuallyPositive<-0.02*0.97

And then I would just divide the numerator by the denominator to get the proportion of people who test positive who actually are positive.

actuallyPositive/testPositive

## [1] 0.2836

As you can see, testing positive yields only about a 28% probability that one really is positive. The computation I just engaged in was based on Bayes law.

The meaning of a negative result on the test

We can also compute the probability that one does not have Damay’s if one tests negative. It’s the same process. The denominator of our fraction are the false negatives plus the true negatives.

testNegative<-0.02*(1-0.97)+ 0.98*0.97

And the denominator of our fraction is …

actuallyNegative<-0.98*0.97

So, a negative test result creates a 0.9506/0.9512 probability that one is negative: 0.9994. The test may have some value in giving people with rashy left ear lobes and serious coughs a 99.94% likelihood that they do not have Damay’s compared to a 98% chance beforehand.

Further analysis

So, beyond giving the truly negative some security, is it worth doing the test? Presumably, we won’t treat anyone who tests negative. So, for those people, the test was perhaps a waste of money. Presumably, we would treat everyone who tested positive (unless there were other contraindications). So, for every 100 persons getting tested, we will save about 70% x 28 lives, or 19.85 lives. But we will kill 30% of everyone we treat, regardless of whether they have the disease. So, we will kill 30 lives. On balance, then, then, we kill a net 10.15 lives.

Extra credit

We could use the uniroot command do a little numerical analysis in R to see how successful the treatment would have to be before it would make some sense to give it to those who test positive.

uniroot(function(success) -success*100*actuallyPositive/testPositive+
          (1-success)*100,c(0,1))$root

## [1] 0.779

Alternatively, we could see how specific the new blood test has to be before treating those who test positive would make sense.

positiveConditional<-function(specificity) {
  denom<-0.02*0.97+0.98*(1-specificity)
  0.02*0.97/denom
  }
  uniroot(function(specificity) -0.7*100*positiveConditional(specificity)+
          (1-0.7)*100,c(0,1))$root

## [1] 0.9736

So, a small improvement in the specificity of the test might make a positive result on the test a more productive finding.