Information bias

Epidemiology course, ITMO PHS

Artemiy Okhotin

10/20/23

Measurement

Real things (substances), constructs

Measurements

Gold standard

Measurement

Measurement error

Measurement errors (binary)

Measured positive Measured negative
Actually positive (P) True Positive (TP)

False Negative (TN)

(Type II error)
Actually negative (N)

False Positive (FP)

(Type I error)

 True Negative (FN)

Specificity and sensitivity

Sensitivity = True Positive Rate (TPR) = True positive / Actually positive

Specificity = True Negative Rate (TNR) = True Negative / Actually Negative

Positive predictive value = True Positive / Measured Positive

Negative predictive value = True Negative / Measured Negative

Sensitivity vs. specificity

Sensitivity vs. specificity

Gold standard

100% sensitivity, 100% specificity

No measurement error

Drawback:

doesn’t exist

Examples

100% sensitive test

100% specific test

50% sensitive, 50% specific

0% sensitive, 0% specific

Implication for screening

Low sensitivity – many cases are missed, false reassurement

Low specificity – many cases are harmed by misdiagnosed and mistreatment

PPV depends on prevalence

PPV and low prevalence

Sensitivity 0,9, specificity 0.9

Number Positive tests Negative tests
With disease 100
Without disease 10 000
Total 10 100

PPV =

PPV and low prevalence

Sensitivity 0,9, specificity 0.9

Number Positive tests Negative tests
With disease 100 90 10
Without disease 10 000 1 000 9 000
Total 10 100 1 090 9 010

90 of 1090 are truly positive, PPV = 90 / 1090 = 0.08 (8%)

PPV and mid prevalence

Sensitivity 0,9, specificity 0.9

Number Positive tests Negative tests
With disease 1 000
Without disease 10 000
Total 11 000

PPV =

PPV and prevalence

Sensitivity 0,9, specificity 0.9

Number Positive tests Negative tests
With disease 1 000 900 100
Without disease 10 000 1 000 9 000
Total 11 000 1 900 9 100

900 of 1900 are truly positive, PPV = 900 / 1900 = 0.47 (47%)

Screening

Cheap, easy tests are non-specific

Specific test are costly, invasive

Sequential testing:

0) perform screening in high risk population

1) use more sensitive, less specific tests to screen

2) use more specific tests to confirm

Screening harms and benefits

“All screening programmes do harm; some do good as well, and, of these, some do more good than harm at reasonable cost.”

Gray, Patnick, and Blanks (2008)

Bayes Theorem and Likelihood ratio (LR)

\[ O(A|B) = O(A) \times \Lambda(A|B) \]

Positive likelihood ratio = TPR / FPR

Negative likelihood ratio = TNR / FNR

Posterior odds = Prior odds \(\times\) Positive LR

Misclassification and associations

Non-dependent misclassification

Never creates spurious associations (only by chance)

https://okhotin.shinyapps.io/Misclassification/

Dependent misclassification

Dependent misclassification (example)

Calorimetric method of glucose and cholesterol measurement

Diabetes – glucose > 7 mmol/l

Hypercholesterolemia – cholesterol > 6 mmol/l

Error is introduced by non-calibrated dilution of the sample, affecting both glucose and cholesterol measurement (in the same direction)

Without misclassification

show code 
df <- data.frame(glucose = rnorm(1000, 5.5, 1), 
           cholesterol = rnorm(1000, 4.5, 1),
           addition_1 = rnorm(1000, 0, 1),
           addition_2 = rnorm(1000, 0, 1))

# exclude negative values
df <- df[df$glucose>0 & df$cholesterol>0,]

g <- df %>% ggplot(aes(x=glucose, y=cholesterol)) +
  geom_point(color="blue", alpha=0.5) + 
#  geom_point(aes(x=glu1, y=chol1), color="blue", alpha=0.5) + 
  xlim(0,10) + ylim(0,10) +
  geom_hline(yintercept=6, linetype="dashed") +
  geom_vline(xintercept=7, linetype="dashed") +
  theme_bw()

g

Independent misclassification

show code 
# add random but independent error
df$glu1 <- df$glucose + df$addition_1
df$chol1 <- df$cholesterol + df$addition_2

g1 <- df %>% ggplot(aes(x=glu1, y=chol1)) +
  geom_point(color="blue", alpha=0.5) + xlim(0,10) + ylim(0,10) +
  geom_hline(yintercept=6, linetype="dashed") +
  geom_vline(xintercept=7, linetype="dashed") +
  labs(x="glucose, misclassified", y="cholesterol, misclassified",
       title="Independent misclassification") +
  theme_bw()

g1

Dependent misclassification

show code 
# add random but independent error
df$glu2 <- df$glucose + df$addition_1
df$chol2 <- df$cholesterol + df$addition_1

g2 <- df %>% ggplot(aes(x=glu2, y=chol2)) +
  geom_point(color="blue", alpha=0.5) + xlim(0,10) + ylim(0,10) +
  geom_hline(yintercept=6, linetype="dashed") +
  geom_vline(xintercept=7, linetype="dashed") +
  labs(x="glucose, misclassified", y="cholesterol, misclassified",
       title="Dependent misclassification") +
  theme_bw()

g2

Tables and ORs

        hypercholesterolemia
diabetes FALSE TRUE
   TRUE    125   16
   FALSE   731  128

Odds ratio

estimate    lower    upper 
    0.74     0.41     1.25 

        hypercholesterolemia
diabetes FALSE TRUE
   TRUE     89   52
   FALSE   764   95

Odds ratio

estimate    lower    upper 
    4.69     3.12     7.02

Differential vs non-differential misclassification

Deiffrential misclassification:

  • misclassification of exposure depends on outcome

  • misclassification of outcome depends on exposure

  • (or both)

Misclassification (Sp/Sn) is unequal accross rows or coulmns of contingency table

Exposure-dependent misclassification of outcome

Outcome-dependent misclassification of exposure

Differential misclassification

Outcome No outcome
Exposure A \(\rightarrow\)
\(\leftarrow\)		 B
\(\downarrow \uparrow\) \(\uparrow \downarrow\)
No exposure C \(\leftarrow\)
\(\rightarrow\)		 D

\[ \frac{A / B}{C / D} = \frac{A \times D}{B \times C} \]

Implication for screening

Park et al. (2016)

Lead-time bias

Sticky diagnosis bias

Implication for case-control trials

Recall bias

RCTs: EXCEL Trial

EXCEL Trial Stone et al. (2019)

CABG vs. PCI for left main artery disease

Long rival between surgeons and interventional cardiologists

Left main artery disease was the last frontier for PCI (percutaneous intervention)

EXCEL Trial have shown non-inferiority of PCI for left main disease

EXCEL Trial

EXCEL trial

EXCEL Trial

Former EXCEL Investigator Alleges Trial Manipulation, Prompting Vehement Denials

Surgeon David Taggart set the EACTS meeting ablaze when he accused EXCEL researchers of stacking the deck in PCI’s favor.

https://www.tctmd.com/news/former-excel-investigator-alleges-trial-manipulation-prompting-vehement-denials

EACTS Pulls Out of Left Main Guidelines After BBC Bombshell Alleging EXCEL Trial Cover-up

https://www.tctmd.com/news/eacts-pulls-out-left-main-guidelines-after-bbc-bombshell-alleging-excel-trial-cover

EXCEL Trial

Gregson et al. (2020)

EXCEL Trial

Death vs. cardiovascular death

With respect to all-cause mortality, 18 of the 30 excess deaths at 5 years were deemed noncardiovascular, said Stone, and there was no significant difference in the risk of cardiovascular death, which was 6.8% in the PCI arm and 5.5% in the CABG group (OR 1.26; 95% 0.85-1.85).

https://www.tctmd.com/news/former-excel-investigator-alleges-trial-manipulation-prompting-vehement-denials

Surrogate end-point

Combined surrogate end-point

Negative vaccine efficacy

Barchuk et al. (2022)

We included 1,254 cases and 2,747 controls recruited between the 6th and 14th of October in the final analysis. VE was 56% (95% CI: 48 to 63) for Gam-COVID-Vac (Sputnik V), 49% (95% CI: 29 to 63) for 1-dose Gam-COVID-Vac (Sputnik V) or Sputnik Light, -58% (95% CI: -225 to 23) for EpiVacCorona and 40% (95% CI: 3 to 63) for CoviVac. Without adjustment for the history of confirmed COVID-19 VE for all vaccines was lower, except for one-dose Gam-COVID-Vac (Sputnik Light). The adjusted VE was slightly lower in women — 51% (95% CI: 39 to 60) than men — 65% (95% CI: 5 to 73).

Prevention

  • Control for dependent misclassification

  • Blinded assessment

  • Formal questionnaires

  • Validation samples

  • Different sources of data

  • Valid surrogate end-points

  • etc. :)

Take-home message

‘If you read the inscription buffalo on an elephant’s cage, do not believe your eyes.’

Kozma Prutkov

Further reading

Hernan and Cole (2009)

VanderWeele and Hernan (2012)

References

Barchuk, Anton, Anna Bulina, Mikhail Cherkashin, Natalia Berezina, Tatyana Rakova, Darya Kuplevatskaya, Oksana Stanevich, Dmitriy Skougarevskiy, and Artemiy Okhotin. 2022. “COVID-19 Vaccines Effectiveness Against Symptomatic SARS-CoV-2 During Delta Variant Surge: A Preliminary Assessment from a Case-Control Study in St. Petersburg, Russia.” BMC Public Health 22 (1). https://doi.org/10.1186/s12889-022-14202-9.
Gray, J A M, J Patnick, and R G Blanks. 2008. “Maximising Benefit and Minimising Harm of Screening.” BMJ 336 (7642): 480–83. https://doi.org/10.1136/bmj.39470.643218.94.
Gregson, John, Gregg W. Stone, Ori Ben-Yehuda, Björn Redfors, David E. Kandzari, Marie-Claude Morice, Martin B. Leon, et al. 2020. “Implications of Alternative Definitions of Peri-Procedural Myocardial Infarction After Coronary Revascularization.” Journal of the American College of Cardiology 76 (14): 1609–21. https://doi.org/10.1016/j.jacc.2020.08.016.
Hernan, M. A., and S. R. Cole. 2009. “Invited Commentary: Causal Diagrams and Measurement Bias.” American Journal of Epidemiology 170 (8): 959–62. https://doi.org/10.1093/aje/kwp293.
Park, Sohee, Chang-Mo Oh, Hyunsoon Cho, Joo Young Lee, Kyu-Won Jung, Jae Kwan Jun, Young-Joo Won, et al. 2016. “Association Between Screening and the Thyroid Cancer “Epidemic” in South Korea: Evidence from a Nationwide Study.” BMJ, November, i5745. https://doi.org/10.1136/bmj.i5745.
Stone, Gregg W., A. Pieter Kappetein, Joseph F. Sabik, Stuart J. Pocock, Marie-Claude Morice, John Puskas, David E. Kandzari, et al. 2019. “Five-Year Outcomes After PCI or CABG for Left Main Coronary Disease.” New England Journal of Medicine 381 (19): 1820–30. https://doi.org/10.1056/nejmoa1909406.
VanderWeele, T. J., and M. A. Hernan. 2012. “Results on Differential and Dependent Measurement Error of the Exposure and the Outcome Using Signed Directed Acyclic Graphs.” American Journal of Epidemiology 175 (12): 1303–10. https://doi.org/10.1093/aje/kwr458.