Comparison results
Optimal shift bewteen the different Rt estimates
First, we will compare the Rt estimations provided by the different methods by estimating the optimal shift between every pair of methods. We assume that the Rt estimates are defined in the time interval [0,T] where the value in T corresponds to the last value of Rt provided by the method. Let R and R’ be two estimations of Rt provided by 2 methods. We define the optimal shift, s(R,R’), by minimizing the average error d(R,R’:s) :
\((1) \ \ \ \ s(R,R^{\prime})=\underset{s\in\lbrack-25,25]}{\arg\min} \ \ d(R,R^{\prime}\!:\!s)\equiv\sum_{t=30}^{T-30}\frac{|R(t)-R^{\prime}(t+s)|+|R(t-s)-R^{\prime}(t)|}{2(T-60)}\)
The value 30 is introduced in the formula to avoid boundary effects in the comparison. The lower is the value of d(R,R’:s) the better is the match between R and R’ after a shift given by s. Notice that the optimal shift is anti-symmetric, that is s(R,R’)=-s(R’,R) and d(R,R’:s) is symmetric, that is d(R,R’:s)=d(R’,R:s). To evaluate R(t-s) or R’(t+s) we use linear interpolation.
In the following 4 figures, we show, for the different countries, the plot of Rt obtained by the methods EE (EpiEstim), EI (EpiInvert), WT (Wallinga-Teunis) and EN (EpiNow2) after applying the optimal shift estimated between EI and the other methods. That is, we compute s(EI,EE), s(EI,WT) and s(EI,EN).
We observe a quite good agreement between the different Rt estimates after applying the optimal shift. The obtained shifted Rt estimates that we show in the above figures can be downloaded from:
FRA_Rt_shift <- read.csv(url("https://www.ctim.es/covid19/FRA_Rt_shift.csv"),sep=";")
DEU_Rt_shift <- read.csv(url("https://www.ctim.es/covid19/DEU_Rt_shift.csv"),sep=";")
USA_Rt_shift <- read.csv(url("https://www.ctim.es/covid19/USA_Rt_shift.csv"),sep=";")
UK_Rt_shift <- read.csv(url("https://www.ctim.es/covid19/UK_Rt_shift.csv"),sep=";")Next, we present 2 tables with the mean and standard deviation of s(R,R’) and d(R,R’,s(R,R’)) for the different countries.
Table 1 : mean and standard deviation of s(R,R’) for the different countries
| EI | EE | WT | EN | |
|---|---|---|---|---|
| EI | 0 \(\pm\) 0 | 7.36 \(\pm\) 0.02 | 3.02 \(\pm\) 0.19 | -8.92 \(\pm\) 0.43 |
| EE | -7.36 \(\pm\) 0.02 | 0 \(\pm\) 0 | -4.32 \(\pm\) 0.03 | -16.71 \(\pm\) 0.41 |
| WT | -3.02 \(\pm\) 0.19 | 4.32 \(\pm\) 0.03 | 0 \(\pm\) 0 | -12.08 \(\pm\) 0.44 |
| EN | 8.92 \(\pm\) 0.43 | 16.71 \(\pm\) 0.41 | -12.08 \(\pm\) 0.44 | 0 \(\pm\) 0 |
Table 2 : mean and standard deviation of d(R,R’,s(R,R’)) for the different countries
| EI | EE | WT | EN | |
|---|---|---|---|---|
| EI | 0 \(\pm\) 0 | 0.034 \(\pm\) 0.011 | 0.026 \(\pm\) 0.009 | 0.057 \(\pm\) 0.021 |
| EE | 0.034 \(\pm\) 0.011 | 0 \(\pm\) 0 | 0.025 \(\pm\) 0.007 | 0.058 \(\pm\) 0.016 |
| WT | 0.026 \(\pm\) 0.009 | 0.025 \(\pm\) 0.007 | 0 \(\pm\) 0 | 0.046 \(\pm\) 0.015 |
| EN | 0.057 \(\pm\) 0.021 | 0.058 \(\pm\) 0.016 | 0.046 \(\pm\) 0.015 | 0 \(\pm\) 0 |
In
PNAS
(2021), we show that the expected value of
s(EI,EE) is about 3
+m where m is closed to a central value (median or mean)
of the serial interval distribution. Since the used serial interval has
a median of 4 and a mean of 4.7, the expected value of
s(EI,EE) is between
7 and 7.7. Notice that, as shown in table 1, the
obtained mean value of
s(EI,EE) for the 4 countries
we use is 7.36 which confirms our expectations. On the other
hand, in
PNAS
(2021), we also show that if R and R’ are the Rt
estimations for WT and EE
respectively, then
\(R(t)=\sum_k\Phi(k)R'(t+k)\)
the second term of this equation can be approximated in the following way:
\(\sum_k\Phi(k)R'(t+k) \approx R'(t+m)\)
where m is a central value of the serial interval \(\Phi\). Therefore, the expected value of s(WT,EE) is between 4 and 4.7 which is confirmed by the obtained value, 4.32, showed in table 1. On the other hand, as it is also expected, we have that, using the values of table 1, s(EI,EE)=7.36 is closed to s(EI,WT)+s(WT,EE)=3.02+4.32=7.34.
Concerning the shift of EN with respect to the other methods, we have that the most significant part of this shift is due to the 7-day forecast included in the EN estimation. In spite of the method EN is quite different to the other ones, the stability of the shift value between EN and the other methods for the different countries suggests that it must exist a theoretical reason which justify such shift value.
In the following plot we show the optimal shift s(R,R’) versus the average error d(R,R’,s(R,R’)) for any pair of methods and the different countries. As stated numerically in tables 1 and 2, we can observe that the value of s(R,R’) is very stable for the different countries and that the largest values of d(R,R’,s(R,R’)) corresponds to the comparison of EN with the other methods and the lowest values to the comparison of WT with EE and EI.
