Framework for measuring Excess of Mortality in LMICs

class: center, middle, inverse, title-slide

# Framework for measuring <br> Excess of Mortality in LMICs
## Application to case of Peru
### Lucas Sempe & Peter Lloyd-Sherlock
### University of East Anglia
### 31 August, 2020 <br> (Data: 24 August)

---

# Introduction

Important differences in **standardised mortality metrics** (deaths per pop/million or adjusted mortality rates) across similar regions and countries

+ Reporting deaths attributed to COVID-19 follows **varying protocols and criteria**

+ Estimating the significance of rising mortality rates over a specific period of time/place beyond recent data requires a **counterfactual** based on reliable data

---

# Data Challenges in LMICs (I)

- **Data consistency** issues such as lack of registration over certain periods of time.
   
.center[  
<img src="PAHO_files/figure-html/unnamed-chunk-2-1.png" style="display: block; margin: auto;" />
]

---

# Data Challenges in LMICs (II)

- **Unregistered deaths attributable to COVID-19** may exceed predicted excess mortality.
 
.center[ 
<img src="PAHO_files/figure-html/unnamed-chunk-3-1.png" style="display: block; margin: auto;" />
]

---

# Data Challenges in LMICs (III)

- **Historically unregistered deaths** affects mostly the accountability of older people.

.center[
<img src="PAHO_files/figure-html/unnamed-chunk-4-1.png" style="display: block; margin: auto;" />
]

---

# Data Challenges in LMICs (IV)

- **Poor crude mortality estimations** based on Census and permanent surveys could affect estimation excess mortality.
   
.center[
<img src="PAHO_files/figure-html/unnamed-chunk-5-1.png" style="display: block; margin: auto;" />
]

---

# Data Challenges in LMICs (V)

- **(Real or artificial) seasonality** has to be addressed to understand data behaviour.

.center[  
<img src="PAHO_files/figure-html/unnamed-chunk-6-1.png" style="display: block; margin: auto;" />
]

---

# Conceptual framework (I)

Usually, computing **Excess Mortality** is based on **Registered deaths** analysis.

.center[
<img src="PAHO_files/figure-html/unnamed-chunk-7-1.png" style="display: block; margin: auto;" />
]

---
# Conceptual framework (II)

However, in most of LMICs we need also to address **two additional layers**:

---

# Empirical strategy (I)

Our approach decomposes the estimation of **Total Excess of Deaths** `$\text{Excess.d}_\text{T}$` into three terms: **registered**, **unregistered**, and **unregistered COVID-19** deaths:

`$$\small\widehat{\text{Excess.deaths}}_\text{Total} = \widehat{\text{Excess.deaths}}_\text{Regist}+{\text{Deaths}_\text{Not.regist}} + {\text{Deaths}_\text{COVID.Not.regist}}$$`

+ We perform the analysis by **age groups** and **regions**.

+ We use two different sources of population and mortality data: **National Statistics** across years and **GBD**

+ Additionaly we use data from the SINADEF and MoH

---

# Empirical strategy (II)

To estimate the **first term** `$\widehat{\text{Excess deaths}}_\text{Registered}$`:

+ We fit **Bayesian time series models** using data prior to the pre-intervention and developing a synthetic conterfactual trend that would have occurred in a virtual scenario with no COVID.

Our counterfactual is made from averaging means from 5,000 samples `${\tau}$` of posterior predictive distributions `$\hat y_{t}$`.

The model uses posterior simulations to simulate posterior predictive distributions.

Finally, using posterior predictive samples, we estimate the posterior distribution causal impact as `${\text{y}_{Observerd}} -  \hat{\text{y}}_{Predicted}$` for each time unit `$\text{t}$`.

+ More details about the method and the implementation `R` packages `CausalImpact` and `BSTS`.
 
---

# Empirical strategy (II)

We perform **hypothesis tests** for each region to address if there are significant changes in the number of registered deaths `$y_\text{t}$` in time points `$\text{t}$` in comparison to the simulated conterfactual.

`$$\begin{equation}
  \label{eq:bayes}
   \begin{cases}
  H_0: \phi_{t}^{\tau}= y_{t} -  \hat y_{t}^{\tau} = 0 \\[1ex]
  H_a: \phi_{t}^{\tau}= y_{t} -  \hat y_{t}^{\tau} \neq 0.
   \end{cases}
\end{equation}$$`

---

# Empirical strategy (IV)

To assess the second term **unregistered deaths**:

We rely on the assumption that under-registration in recent years is a reasonable counterfactual to estimate missing registration in 2020.

`$$\small{\text{Deaths}_\text{Not.registered}}=(1-\widehat{\text{Completness.registered}})*\widehat{\text{Excess.deaths}}_\text{Registered}$$`

Where `$\widehat{\text{Completness.register}}$` refers to the completeness registration rate.

---

# Empirical strategy (V)

We use time-series ARIMA and Drift models to estimate the completeness rate `$\widehat{\text{Comp.reg}}$`:

`$$\small\widehat{\text{Completness.registered}}_\text{INEI}= \frac{\text{Deaths}_\text{Registered}}{\widehat{\text{Deaths}}_{{\text{Expected}}_{\text{INEI}}}}\pm  1.96\sqrt{\widehat{\text{MR}}*\widehat{\text{pop}}\sqrt{(\frac{\hat{\sigma}_\text{MR}}{\widehat{\text{MR}}})^2+(\frac{\hat{\sigma}_\text{pop}}{\widehat{\text{pop}}})^2}}$$`

Where `$\widehat{\text{MR}}$` and `$\widehat{\text{pop}}$` refer to the ARIMA estimations of mortality rates and population, respectively.

---
# Empirical strategy (VI)

Finally, the third term **unregistered COVID-19 deaths** is computed to correct for the proportion of cumulative cases of COVID-19 in cases they exceed the number of registered deaths over the period.

`$$\small\text{If }\text{Deaths}_\text{COVID} \ge \text{Excess.d}_\text{Reg}, \text{then} \  {\text{Deaths}_\text{COVID Not.Reg}}=\text{COVID}_\text{Reg}-\widehat{\text{Excess.d}_\text{Reg}},\\[1ex]
\ \text{else} \  {\text{Deaths}_\text{COVID Not.Reg}} = 0$$`

---

# Results (I)

Naive INEI counterfactual:

+ Excess Registered excess: **66,892** (CI uninformative)

+ Total excess: **~ 88,000 / 89,000** (95%CI 74,500-97,000)

Bayesian counterfactual:

+ Excess Registered excess: **64,388** (95%CI 60,682-68,100)

+ Total excess: **~ 82,000 / 83,000** (~ 95%CI 78,000-86,000)

---

# Results (II)

<table class="table table-hover" style="font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;">
<caption style="font-size: initial !important;">Estimated of excess of deaths based on INEI: Bayesian estimation - weeks 12 to 34</caption>
 <thead>
  <tr>
   <th style="text-align:left;"> model.pop </th>
   <th style="text-align:left;"> model.mort </th>
   <th style="text-align:right;"> Total excess Bayes </th>
   <th style="text-align:right;"> Total Bayes Lower CI95% </th>
   <th style="text-align:right;"> Total Bayes Upper CI95% </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> arima </td>
   <td style="text-align:left;"> arima </td>
   <td style="text-align:right;"> 82697.36 </td>
   <td style="text-align:right;"> 78020.87 </td>
   <td style="text-align:right;"> 85967.01 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> arima </td>
   <td style="text-align:left;"> drift </td>
   <td style="text-align:right;"> 82989.58 </td>
   <td style="text-align:right;"> 78439.04 </td>
   <td style="text-align:right;"> 86144.16 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> drift </td>
   <td style="text-align:left;"> arima </td>
   <td style="text-align:right;"> 83208.57 </td>
   <td style="text-align:right;"> 78506.76 </td>
   <td style="text-align:right;"> 86505.19 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> drift </td>
   <td style="text-align:left;"> drift </td>
   <td style="text-align:right;"> 83465.63 </td>
   <td style="text-align:right;"> 78892.63 </td>
   <td style="text-align:right;"> 86643.74 </td>
  </tr>
</tbody>
</table>

<table class="table table-hover" style="font-size: 12px; width: auto !important; margin-left: auto; margin-right: auto;">
<caption style="font-size: initial !important;">Estimated of excess of deaths based on INEI: Naive - weeks 12 to 34</caption>
 <thead>
  <tr>
   <th style="text-align:left;"> model.pop </th>
   <th style="text-align:left;"> model.mort </th>
   <th style="text-align:right;"> Total excess Naive </th>
   <th style="text-align:right;"> Total Naive Lower CI95% </th>
   <th style="text-align:right;"> Total Naive Upper CI95% </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> arima </td>
   <td style="text-align:left;"> arima </td>
   <td style="text-align:right;"> 88433.55 </td>
   <td style="text-align:right;"> 74374.83 </td>
   <td style="text-align:right;"> 96700.57 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> arima </td>
   <td style="text-align:left;"> drift </td>
   <td style="text-align:right;"> 88696.27 </td>
   <td style="text-align:right;"> 74500.83 </td>
   <td style="text-align:right;"> 97104.56 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> drift </td>
   <td style="text-align:left;"> arima </td>
   <td style="text-align:right;"> 88982.70 </td>
   <td style="text-align:right;"> 74931.38 </td>
   <td style="text-align:right;"> 97242.72 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> drift </td>
   <td style="text-align:left;"> drift </td>
   <td style="text-align:right;"> 89207.70 </td>
   <td style="text-align:right;"> 75018.40 </td>
   <td style="text-align:right;"> 97610.07 </td>
  </tr>
</tbody>
</table>

---

# Results (III)

Bayesian GBD counterfactual:

+ Total excess: **~ 79,000 ** (~ 95%CI 75,000-79,000)

Naive GBD counterfactual:

+ Total excess: **~ 84,000** (~95%CI 64,000-97,000)

```
## # A tibble: 2 x 4
##   model.GBD   sin    min     max
##   <chr>     <dbl>  <dbl>   <dbl>
## 1 arima     66892 16875. 116909.
## 2 drift     66892 16875. 116909.
```

---

# Results (IV)

+ COVID deaths in Ayacucho and Amazonas still don't affect lower estimation of total deaths.

---

# Results (V)

Some regions of Peru were not be significantly affected by COVID-19 until recently.

---
# Results (VI)

Some cases (regions or age groups) with no solid evidence suggesting under-registration of deaths:

---

# Results (VII)

Less deaths occurred due to a positive but unintended effect of quarantine or curfews (such as fewer road accidents or violent crimes).

---

# Results (VIII)

Standardised mortality varies significantly across regions shows states less populated and in coastal with higher values.

---

# Discussion (I)

+ Different data challenges in terms of measurement are risks for inferential bias.

+ Importance of understanding and addressing on a case-by-case basis

+ Addressing unregistered historical deaths allows addressing critical measurement bias

+ Significant downward bias of COVID-19 deaths in official sources (less than 50% of the most conservative estimation of registered deaths)

+ Bayes synthetic control accounts for trends and seasonality => more realistic estimations (in this case, -4-5% less than naive)

+ Bayes synthetic control provides reasonable and practical credible intervals for registered and total excess of deaths

---

# Discussion (II)

+ Time-series analysis to deal with missing datapoints (population and mortality rates) works in a logic of training and testing

+ Forecasting TS over longer periods provides uninformative wider CIs

+ GBD underestimates mortality, which leads to underestimation of COVID

---

class: center, middle

**Thank you!**

l.sempe@uea.ac.uk / p.lloyd-sherlock@uea.ac.uk

https://github.com/lsempe77/Excess-Mortality