Predicting Dengue Disease Spread

Introduction

This study was conducted as part of a competition lead by drivendata.org and attempts to develop a model that can accurately forecast outbreaks of dengue fever. Given that dengue is a mosquito-borne disease, its spread can be linked to climate related factors favorable to mosquitos. This suggests that climate change could pose as a health risk as areas previously unaffected by dengue shift towards a more favorable climate for the carrier. That said, the relationship between climate and dengue is fairly complex and may be influenced by other factors. The goal is to see how accurately climate alone can predict dengue fever using data provided by the DengAI competition.

Literature Review

Previous research suggests that there is indeed a connection between the spread of mosquito borne illness are connected. DiSiera et al. conducted a study examining the dengue outbreak of 2018 in Reúnion [1]. They observed that unseasonably warm temperatures and elevated rainfall enabled the mosquito population to avoid a seasonal population decline and hypothesized that contributed to elevated dengue cases. To test their hypothesis they developed several forecasting models to see if it could be predicted in advance. They discovered that when using an integrated subseasonal model with rainfall, they could provide a reliable mosquito population for up to four weeks in advance.

Another study conducted by Liu-Helmersson et al. observed a different dengue outbreak in Madeira, Portugal in 2012 [2]. This was an especially concerning outbreak as it was the first dengue outbreak in Europe since the 1920s. Like the previous study, it examined how climate can influence the mosquito population. This study used an “empirically-informed, process-based mathematical model” that accounted for precipitation and temperature. The study concluded that mosquito infestations could be avoided if temperature rise is kept under 2° Celsius.

Data

Variable Name Variable Definition
ndvi_ne normalized density vegetation index (northeast)
ndvi_nw normalized density vegetation index (northwest)
ndvi_se normalized density vegetation index (southeast)
ndvi_sw normalized density vegetation index (southwest)
precipitation_amt_mm Total precipitation (millimeters)
reanalysis_air_temp_k Air temperature (Kelvin)
reanalysis_avg_temp_k Average air temperature (Kelvin)
reanalysis_dew_point_temp_k Dew point temperature (Kelvin)
reanalysis_max_air_temp_k Maximum air temperature (Kelvin)
reanalysis_min_air_temp_k Minimum air temperature (Kelvin)
reanalysis_precip_amt_kg_per_m2 Total precipitation amount (kilograms per square meter)
reanalysis_relative_humidity_percent Relative humidity (percentage)
reanalysis_sat_precip_amt_mm Total precipitation amount (millimeters)
reanalysis_specific_humidity_g_per_kg Specific humidity (grams per kilogram)
reanalysis_tdtr_k Diurnal temperature range (Kelvin)
station_avg_temp_c Average temperature (Celsius)
station_diur_temp_rng_c Diurnal temperature range (Celsius)
station_max_temp_c Maximum temperature (Celsius)
station_min_temp_c Minimum temperature (Celsius)
station_precip_mm Total precipitation amount (millimeters)
total_cases Total cases of dengue fever

Data Evaluation

The table below provides the summary stats for all variables in the training data. This was done to see how much data cleaning may be needed. All-in-all, there do not seem to be any abnormalities in the data as values fall into an expected range for their region. The one cause for concern was missing observations which is discussed in the next section.

Training Data Descriptive Stats ( 1456 Observations)
Mean Median SD Min Max NAs
year 2001.03 2002.00 5.41 1990.00 2010.00 0
weekofyear 26.50 26.50 15.02 1.00 53.00 0
total_cases 24.68 12.00 43.60 0.00 461.00 0
ndvi_ne 0.14 0.13 0.14 -0.41 0.51 194
ndvi_nw 0.13 0.12 0.12 -0.46 0.45 52
ndvi_se 0.20 0.20 0.07 -0.02 0.54 22
ndvi_sw 0.20 0.19 0.08 -0.06 0.55 22
precipitation_amt_mm 45.76 38.34 43.72 0.00 390.60 13
reanalysis_air_temp_k 298.70 298.65 1.36 294.64 302.20 10
reanalysis_avg_temp_k 299.23 299.29 1.26 294.89 302.93 10
reanalysis_dew_point_temp_k 295.25 295.64 1.53 289.64 298.45 10
reanalysis_max_air_temp_k 303.43 302.40 3.23 297.80 314.00 10
reanalysis_min_air_temp_k 295.72 296.20 2.57 286.90 299.90 10
reanalysis_precip_amt_kg_per_m2 40.15 27.24 43.43 0.00 570.50 10
reanalysis_relative_humidity_percent 82.16 80.30 7.15 57.79 98.61 10
reanalysis_sat_precip_amt_mm 45.76 38.34 43.72 0.00 390.60 13
reanalysis_specific_humidity_g_per_kg 16.75 17.09 1.54 11.72 20.46 10
reanalysis_tdtr_k 4.90 2.86 3.55 1.36 16.03 10
station_avg_temp_c 27.19 27.41 1.29 21.40 30.80 43
station_diur_temp_rng_c 8.06 7.30 2.13 4.53 15.80 43
station_max_temp_c 32.45 32.80 1.96 26.70 42.20 20
station_min_temp_c 22.10 22.20 1.57 14.70 25.60 14
station_precip_mm 39.33 23.85 47.46 0.00 543.30 22

Missing Data

The descriptive stats table showed that null values exist in the data. The table below expands on this by examining null values for both cities in this study. While there are no variables with a notably high frequency of nulls, it is important for time series forecasting that observations are not omitted. Therefore, null values were replaced with the most recent non-NA value using the na.locf operator from the “zoo” package in R.

Missing Values in Training Data
Overall SJ IQ
year 0 0 0
weekofyear 0 0 0
total_cases 0 0 0
ndvi_ne 194 191 3
ndvi_nw 52 49 3
ndvi_se 22 19 3
ndvi_sw 22 19 3
precipitation_amt_mm 13 9 4
reanalysis_air_temp_k 10 6 4
reanalysis_avg_temp_k 10 6 4
reanalysis_dew_point_temp_k 10 6 4
reanalysis_max_air_temp_k 10 6 4
reanalysis_min_air_temp_k 10 6 4
reanalysis_precip_amt_kg_per_m2 10 6 4
reanalysis_relative_humidity_percent 10 6 4
reanalysis_sat_precip_amt_mm 13 9 4
reanalysis_specific_humidity_g_per_kg 10 6 4
reanalysis_tdtr_k 10 6 4
station_avg_temp_c 43 6 37
station_diur_temp_rng_c 43 6 37
station_max_temp_c 20 6 14
station_min_temp_c 14 6 8
station_precip_mm 22 6 16

Methods (Model Building)

Models were carefully created and tested for this section by splitting training data and observing each model’s performance stats before applying them to the competition testing data. There were two neural network models constructed using data from the real analysis variables in the competition data frame. They are are quite similar but use a metric for temperature. The first model uses average air temperature (K) while the second uses minimum air temperature (K). Other than that, both models use kilograms per square meter for precipitation and specific humidity.

Neural Network (Avg Temp)

Sample Test Stats
RMSE MAE MAPE MASE ME MPE
Iquitos 12.93599 7.403846 Inf 1.641768 5.429487 -Inf
San Juan 137.12723 81.986395 Inf 10.725819 -62.517007 -Inf
Overall 95.96264 43.587459 Inf 7.244586 -27.534654 -Inf

When this neural network model was submitted to the DengAI competition, it ranked 443 with an MAE score of 21.5721.

Neural Network (Minimum Temp)

Sample Test Stats
RMSE MAE MAPE MASE ME MPE
Iquitos 11.92337 7.128205 Inf 1.580646 3.833333 -Inf
San Juan 121.06247 58.625850 Inf 7.669690 -38.285714 -Inf
Overall 84.75607 32.112211 Inf 5.337307 -16.600660 -Inf

When the minimum temperature neural network model was submitted to the DengAI competition, it ranked 406 had an MAE score of 21.1538.

Dynamic Regression

Sample Test Stats
RMSE MAE MAPE MASE ME MPE
Iquitos 12.47947 7.544872 Inf 1.673040 3.942308 -Inf
San Juan 33.18696 22.163265 Inf 2.899495 4.299320 -Inf
Overall 24.78934 14.636964 Inf 2.432781 4.115512 -Inf

This model is yet to be submitted to the competition as a bug was noticed in an earlier model which was fixed and resubmitted. This will be determined later.

Discussion

Performance

The models seemed to perform remarkably well. One thing that I did not understand is that mean percent error and mean absolute percent error were “inf” in the performance stats. A likely reason may be that there were observations that had either 0 or close to 0 for error and threw off the metric to get an infinite value. In other regards, the dynamic regression statistically performed much better than the other models but did not seem to actually follow the trends. Instead, it ran along what seemed to be around the average number of cases. The neural networks were able to predict spikes, but their intensity was off. This can be observed by looking at their performance on the split data. At the end, the minimum temperature neural network seemed a better fit and performed better than the average temperature model. This was also backed by performance stats. The min temp scored an MAE of 32.112 compared to the avg temp model’s 43.5875. Both were beat by dynamic reg with a score of 14.6370.

Limitations

The first limitation to note, as pointed out earlier, is that there is a significant difference There could be a few reasons for this that are not present in the data. First is population. If San Juan has a much higher population than Iquitos, the cases for dengue are bound to be higher as well. The second reason may be public policy or pest control. Iquitos could potentially have better pest control that keeps the mosquito population down. Whether it is one or the other, or both cannot be answered with the data at hand. This would require population data and some research on pest control in each region. These models could certainly be enhanced by using population data as a regressor in the modeling process.

The second limitation to note falls in neural network models being reproducible. I ran the code chunk a few times and received different results each time until I finally used the set.seed operator. So while the models I used seemed to fit pretty well, there was a degree of randomness in their selection. Therefore, a better weighted model may have been skipped over.

Future Work

This study, as suggested in limitations could benefit from using more data such as population. While population between cities was accounted for by splitting the data, the model does not account for a rise or fall in the population of each city. It may also be worth looking more into finding ways to improve the neural network. The dynamic regression seemed to outperform the other models simply because it seemed to predict the average cases and run with it as a baseline. The point of the project is to predict when cases will spike over the average number of cases for the coming years. The neural nets were pretty good with predicting when spikes would happen but struggled a bit when it came to their severity. By adding more variables to the model, the neural net has more data points to learn from and can become much more precise.

Conclusion

One key takeaway from this study was that a model that is generally more accurate may not be ideal for use depending on the intended goal. Sure, the dynamic regression was accurate, but it was virtually useless in predicting spikes in cases. Perhaps it could be built up and become better at predicting trends with more variables. A previous study on inflation for this course showed that the dynamic regression can predict sudden changes in the data depending on external regressors. That said, the neural net forecasts are more ideal for use from this study. Even if they may over-exaggerate the severity of spikes. It is better from a policy standpoint to look at predicted spikes and prepare countermeasures.

Citations

  1. DiSera, L., Sjödin, H., Rocklöv, J., Tozan, Y., Súdre, B., Zeller, H., & Muñoz, Á. G. (2020). The mosquito, the virus, the climate: An unforeseen Réunion in 2018. GeoHealth, 4, e2020GH000253. https://doi.org/10.1029/2020GH000253

  2. Jing Liu-Helmersson, Joacim Rocklöv, Macquin Sewe, Åke Brännström, Climate change may enable Aedes aegypti infestation in major European cities by 2100, Environmental Research, Volume 172, 2019, Pages 693-699, ISSN 0013-9351, https://doi.org/105.1016/j.envres.2019.02.026. (https://www.sciencedirect.com/science/article/pii/S0013935119301069)