This report is the same as my report from Week 4. For new analysis part of Week 5, please go down to the “Modelling” section (it is the last section of the report).
According to the CDC, schistosomiasis is a disease caused by parasitic worms in humans, which is second only to malaria in terms of its devastating effect 1 CDC. (2021, January 13). CDC - Schistosomiasis. Www.cdc.gov. https://www.cdc.gov/parasites/schistosomiasis/index.html#:~:text=Schistosomiasis%2C%20also%20known%20as%20bilharzia. The three main species infecting humans are Schistosoma haematobium, Schistosoma japonicum, and Schistosoma mansoni. Risk factors for the disease include regions with poor sanitation and freshwater bodies.
Due to the devastating impacts of this disease, understanding the environments that help it spread can contribute to a more focused and effective prevention and mitigation strategy. Currently, the literature in the space consists of a landscape with several studies that hope to inform disease spread prevention. For instance, there have been geostatistical studies to understand schistosomiasis prevalence 2 Schur, N., Hürlimann, E., Garba, A., Traoré, M. S., Ndir, O., Ratard, R. C., Tchuem Tchuenté, L. A., Kristensen, T. K., Utzinger, J., & Vounatsou, P. (2011). Geostatistical model-based estimates of Schistosomiasis prevalence among individuals aged ≤ 20 years in West Africa. PLoS neglected tropical diseases, 5(6), e1194. https://doi.org/10.1371/journal.pntd.0001194, as well as spatial mapping of Schistosoma infection risk in east Africa 3 Schur, N., Hürlimann, E., Stensgaard, A. S., Chimfwembe, K., Mushinge, G., Simoonga, C., Kabatereine, N. B., Kristensen, T. K., Utzinger, J., & Vounatsou, P. (2013). Spatially explicit Schistosoma infection risk in eastern Africa using Bayesian geostatistical modelling. Acta tropica, 128(2), 365–377. https://doi.org/10.1016/j.actatropica.2011.10.006. However, there is a gap in the existing literature when it comes to the seasonal indices that influence the spread of the disease. My project aims to address this by examining the relationship of biological climate and environmental indicators.
For the SURP-Stats cohort of Summer 2024, I will be researching with the De Leo lab. The De Leo lab is led by Principal Investigator Dr. Giulio De Leo in the Stanford Doerr School of Sustainability. One of the main focuses of the lab is the control and elimination of infectious diseases with an important environmental component in their transmission cycle, with the most notable being schistosomiasis in West Africa. In the lab, I will be working with Project Manager, Andy Chamberlin.
My project for DATASCI 120 is going to be related to the work that I will be doing with the De Leo Lab in the summer so that I can gain some subject matter knowledge and be able to contribute more greatly to the lab with prior experience. The datasets and some analysis techniques were provided by the De Leo lab as these were specific insights they were interested in deriving. Due to these reasons, this project is affiliated with the De Leo Lab.
For the course of this project, I will be receiving feedback from Andy, who can be reached at achamb@stanford.edu.
The dataset that I will be using contains information about the source of schistosomiasis parasite prevalence and location data from West Africa. This dataset is sourced from the Global Neglected Tropical Disease database (GNTD). This dataset was accessed from GTND in 2016, and spans the years 1934-2010, with the majority of the data post-1980. The geophysical and climate variables were sampled at the point locations using Google Earth Engine. The data explores the relationship between two different parasites that cause schistosomiasis in humans: Schistosoma mansoni and Schistosoma haematobium, through spatial data, time, 19 climate predictor variables, and 9 terrain predictor variables.
My dataset is linked here.
The data dictionary that defines this dataset is linked here.
Based on this data, the primary research question that I hope to address is “what environmental and terrain features predict the presence of Schistoma mansoni and Schistosoma haematobium most effectively?”
I have three hypotheses that I hope to prove to accompany my research question investigation:
Originally, I had loaded in my dataset and RStudio read most of the variables as the wrong type. Therefore, I re-imported the dataset by making the necessary type changes to the code.
## percent_pos longitude latitude lat_long
## Min. : 0.00 Min. :-17.440 Min. :-42.383 Length:11330
## 1st Qu.: 0.00 1st Qu.: -3.706 1st Qu.: -1.307 Class :character
## Median : 8.15 Median : 11.450 Median : 7.364 Mode :character
## Mean : 21.87 Mean : 13.390 Mean : 4.574
## 3rd Qu.: 37.50 3rd Qu.: 32.760 3rd Qu.: 13.123
## Max. :100.00 Max. : 47.433 Max. : 20.258
##
## loc_spp parasite_s yearof_pub bio01
## Length:11330 Length:11330 Min. : 0 Min. :12.08
## Class :character Class :character 1st Qu.:1982 1st Qu.:23.00
## Mode :character Mode :character Median :1990 Median :25.91
## Mean :1839 Mean :25.10
## 3rd Qu.:1996 3rd Qu.:27.40
## Max. :2010 Max. :30.43
## NA's :31
## bio02 bio03 bio04 bio05
## Min. : 5.90 Min. :43.00 Min. : 1.469 Min. :20.07
## 1st Qu.:10.70 1st Qu.:57.00 1st Qu.:10.448 1st Qu.:31.30
## Median :13.00 Median :63.00 Median :18.046 Median :34.00
## Mean :12.54 Mean :65.46 Mean :18.371 Mean :34.66
## 3rd Qu.:14.40 3rd Qu.:72.56 3rd Qu.:25.490 3rd Qu.:39.00
## Max. :19.90 Max. :91.87 Max. :53.857 Max. :42.60
## NA's :31 NA's :31 NA's :31 NA's :31
## bio06 bio07 bio08 bio09
## Min. : 1.50 Min. : 7.90 Min. :11.26 Min. :10.30
## 1st Qu.:13.56 1st Qu.:14.50 1st Qu.:23.10 1st Qu.:21.90
## Median :15.30 Median :19.90 Median :25.70 Median :24.60
## Mean :15.11 Mean :19.54 Mean :25.09 Mean :23.72
## 3rd Qu.:17.50 3rd Qu.:24.25 3rd Qu.:27.20 3rd Qu.:26.30
## Max. :22.90 Max. :31.10 Max. :35.60 Max. :31.87
## NA's :31 NA's :31 NA's :31 NA's :31
## bio10 bio11 bio12 bio13
## Min. :13.28 Min. :10.30 Min. : 11 Min. : 3.0
## 1st Qu.:24.60 1st Qu.:21.50 1st Qu.: 615 1st Qu.: 167.4
## Median :27.60 Median :23.43 Median : 977 Median : 216.0
## Mean :27.35 Mean :22.66 Mean :1043 Mean : 230.5
## 3rd Qu.:30.50 3rd Qu.:24.96 3rd Qu.:1314 3rd Qu.: 282.0
## Max. :36.15 Max. :28.70 Max. :4226 Max. :1139.4
## NA's :31 NA's :31 NA's :31 NA's :31
## bio14 bio15 bio16 bio17
## Min. : 0.000 Min. : 25.16 Min. : 7.0 Min. : 0.00
## 1st Qu.: 0.000 1st Qu.: 66.00 1st Qu.: 408.0 1st Qu.: 0.00
## Median : 1.000 Median : 99.00 Median : 536.2 Median : 6.00
## Mean : 8.618 Mean : 98.11 Mean : 573.4 Mean : 39.41
## 3rd Qu.:13.000 3rd Qu.:128.00 3rd Qu.: 694.0 3rd Qu.: 64.00
## Max. :94.000 Max. :183.00 Max. :2798.8 Max. :337.74
## NA's :31 NA's :31 NA's :31 NA's :31
## bio18 bio19 hnd upa
## Min. : 4.00 Min. : 0.0 Min. : 0.000 Min. : 0.01
## 1st Qu.: 92.34 1st Qu.: 2.0 1st Qu.: 2.267 1st Qu.: 0.01
## Median :188.00 Median : 26.0 Median : 6.740 Median : 0.03
## Mean :212.19 Mean : 223.8 Mean : 14.653 Mean : 161.65
## 3rd Qu.:301.00 3rd Qu.: 380.6 3rd Qu.: 16.476 3rd Qu.: 0.10
## Max. :996.00 Max. :2798.8 Max. :408.948 Max. :276274.60
## NA's :31 NA's :31 NA's :25 NA's :25
## slope gHM elevation pH
## Min. : 0.000 Min. :0.0000 Min. : -2.748 Min. :4.024
## 1st Qu.: 1.256 1st Qu.:0.3145 1st Qu.: 195.749 1st Qu.:5.601
## Median : 1.891 Median :0.5270 Median : 365.933 Median :6.023
## Mean : 2.793 Mean :0.5062 Mean : 565.760 Mean :6.116
## 3rd Qu.: 3.002 3rd Qu.:0.6850 3rd Qu.: 950.891 3rd Qu.:6.501
## Max. :47.420 Max. :0.9630 Max. :3181.426 Max. :9.039
## NA's :24 NA's :79 NA's :24 NA's :169
## aspect clay sand
## Min. : 0.0 Min. : 0.00 Min. :16.13
## 1st Qu.:116.4 1st Qu.:17.99 1st Qu.:47.65
## Median :166.0 Median :23.90 Median :56.85
## Mean :167.1 Mean :25.09 Mean :56.45
## 3rd Qu.:217.4 3rd Qu.:31.88 3rd Qu.:65.47
## Max. :350.1 Max. :58.72 Max. :99.00
## NA's :24 NA's :169 NA's :169It is interesting to see that the number of NA’s have a pattern that corresponds to the different Google Earth Engine libraries used. For example, the “clay” and “sand” variables are derived from the same Google Earth Engine library and both have 169 NA’s. Another example of this can be seen through the DEM-derived variables (slope, elevation, and aspect), which all have 24 NA’s. All missing values are summarized and dealt with as “NA”s in the summary table.
It is defined as aggregated schistosomiasis parasitology study percentage positivity of parasite presence at each “loc_spp.” This data is collected from the GNTD database mentioned earlier. It is a quantitative variable whose values range from 0 to 100 with units in percentage. The distribution of the values of the variable can be plotted for greater insight:
This contains the longitude coordinates of each observation. This data is collected from the GNTD database mentioned earlier. It is a quantitative variable whose values range from -180 to 180 with units in degrees. The distribution of the values of the variable can be plotted for greater insight:
Since the dataset primarily represents data from Africa, the longitudes seem to be concentrated there.
This contains the latitude coordinates of each observation. This data is collected from the GNTD database mentioned earlier. It is a quantitative variable whose values range from -90 to 90 with units in degrees. The distribution of the values of the variable can be plotted for greater insight:
Since the dataset primarily represents data from Africa, the latitudes seem the be concentrated there.
The latitude-longitude coordinates are merged into one column. I don’t think I will be using this data at this point since I already have the latitude and longitude columns separately. Hence, I will not be plotting it or describing it again in this report after already mentioning it in the data dictionary.
This column contains the latitude-longitude coordinates for each location and also contains the species of parasite: Schistosoma haematobium, Schistosoma mansoni or Schistosoma intercalatum. I don’t think I will be using this data at this point since I already have the latitude, longitude, and specie columns separately. Hence, I will not be plotting it or describing it again in this report after already mentioning it in the data dictionary.
This contains the parasite species for each observation. This data is collected from the GNTD database mentioned earlier. It is a qualitative variable whose categories are Schistosoma haematobium, Schistosoma mansoni or Schistosoma intercalatum. The distribution of the values of the variable can be plotted for greater insight:
This contains the year each study was published. This data is collected from the GNTD database mentioned earlier. It is a quantitative variable whose values can range from 1900 to 2010. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 856 rows containing non-finite outside the scale range
## (`stat_bin()`).## Warning: Removed 1 row containing missing values or values outside the scale range
## (`geom_bar()`).
This contains the annual mean temperature. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from -29 to 32 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the mean diurnal range. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0.9 to 21.4 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the isothermality of each observation. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 7 to 96 with units in percentage. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the Temperature seasonality of each observation, which is just the standard deviation * 100. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0.6 to 227 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the Max temperature of warmest month. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from -9.6 to 49 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the Min temperature of coldest month. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from -57.3 to 25.8 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the Temperature annual range. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 5.3 to 72.5 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the mean temperature of wettest quarter. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from -28.5 to 37.8 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the Mean temperature of driest quarter. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from -52.1 to 36.6 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the Mean temperature of warmest quarter. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from -14.3 to 38.3 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
# Variable 18: bio11
This contains the Mean temperature of coldest quarter. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from -52.1 to 28.9 with units in degree celsius. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the annual precipitation. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0 to 11401 with units in millimeters. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the Precipitation of wettest month. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0 to 2949 with units in millimeters. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the Precipitation of driest month. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0 to 752 with units in millimeters. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the precipitation seasonality. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0 to 265 with units as coefficient of variation. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the precipitation of wettest quarter. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0 to 8019 with units as millimeter. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the precipitation of driest quarter. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0 to 2495 with units as millimeter. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the precipitation of warmest quarter. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0 to 6090 with units as millimeter. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the precipitation of coldest quarter. This data is collected from the Google Earth Engine “WorldClim BIO Variables V1” dataset provided by UC Berkeley. It is a quantitative variable whose values range from 0 to 5162 with units as millimeter. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 31 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the hydrologically adjusted elevation, also know as “hand” (height above the nearest drainage). This data is collected from the Google Earth Engine “MERIT Hydro: Global Hydrography Datasets” dataset provided by University of Tokyo. It is a quantitative variable whose values range from 0 to 90 with units as meter. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 25 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the upstream drainage area (flow accumulation area). This data is collected from the Google Earth Engine “MERIT Hydro: Global Hydrography Datasets” dataset provided by University of Tokyo. It is a quantitative variable whose values range from 0 to 200 with units as kilometer^2. The distribution of the values is described in the summary table at the start of this report - there was an error I was getting when trying to generate this histogram. I hope to address the issue in Office Hours can provide an update in my report for Week 5.
This contains the slope of terrain (digital elevation model [DEM]-derived). This data is collected from the Google Earth Engine “SRTM Digital Elevation Data Version 4” dataset provided by NASA. It is a quantitative variable whose values range from 0 to 90 with units as degrees. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 24 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the global human modification: a cumulative measure of human modification of terrestrial lands globally at 1 square-kilometer resolution. This data is collected from the Google Earth Engine “CSP gHM: Global Human Modification” dataset provided by Conservation Science Partners. It is a quantitative variable whose values range from 0 to 1 with units as kilometer^2. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 79 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the elevation of terrain (digital elevation model [DEM]-derived). This data is collected from the Google Earth Engine “SRTM Digital Elevation Data Version 4” dataset provided by NASA. It is a quantitative variable whose values range from -444 to 8806 with units as meter. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 24 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains the metric to measure soil acidity - this data had to be transformed by scaling the values by 0.1 to represent a regular pH scale which I did through Excel. This data is collected from the Google Earth Engine “OpenLandMap Soil pH in H2O” dataset provided by EnvirometriX Ltd. It is a qualitative variable whose values range from 0 to 14 with 1 unit increments (eg. 1, 2, …, 13, 14). The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 169 rows containing non-finite outside the scale range
## (`stat_bin()`).
Note: Even though this is a qualitative variable, I plotted it as a histogram because the pH could be any value from 0 - 14. I hope to clarify my approach with regards to this variable in Office Hours and include it in my Week 5 report.
This contains the aspect of terrain (digital elevation model [DEM]-derived). This data is collected from the Google Earth Engine “SRTM Digital Elevation Data Version 4” dataset provided by NASA. It is a quantitative variable whose values range from 0 to 360 with units as degrees. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 24 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains how much of the soil content is clay. This data is collected from the Google Earth Engine “OpenLandMap Clay Content” dataset provided by EnvirometriX Ltd. It is a quantitative variable whose values range from 0 to 100 as a percentage. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 169 rows containing non-finite outside the scale range
## (`stat_bin()`).
This contains how much of the soil content is sand. This data is collected from the Google Earth Engine “OpenLandMap Sand Content” dataset provided by EnvirometriX Ltd. It is a quantitative variable whose values range from 0 to 100 as a percentage. The distribution of the values of the variable can be plotted for greater insight:
## Warning: Removed 169 rows containing non-finite outside the scale range
## (`stat_bin()`).
The first step of quality control is imputing the NaN values as per the summary table provided at the start of the exploratory data analysis.
pH, clay, and sand have 169 missing values. Since these factors are so heavily dependant on location, I am going to fill in thos missing values with the average of the 10 nearest neighbors of those observations in terms of longitude and latitude.
## pH clay sand
## 0 0 0The biological climate variables, “bio01” to “bio19”, have 31 missing values. Upon investigation, I saw that these missing values were all in the same observation. Because of this, I decided to remove those 31 observations entirely as none of them would have recorded data for those 19 variables. My dataset size was reduced from 11,330 observations to 11,299 observations.
## [1] 11299 35“hnd” and “upa” both had 25 missing values. Since the hydrological metrics of a particular region would be similar to the geography around it, I handled these values through the 10 nearest neighbors average method based on longitude and latitude.
## hnd upa
## 0 0“Slope,” “elevation,” and “aspect” all had 24 missing values. Since the landscape metrics of a particular region would be similar to the geography around it, I handled these values through the 10 nearest neighbors average method based on longitude and latitude.
## slope elevation aspect
## 0 0 0Lastly, “gHM” had 55 missing values. Since the human modification and terrain metrics of a particular region would be similar to the locations close to it, I handled these values through the 10 nearest neighbors average method based on longitude and latitude.
## gHM
## 0## percent_pos longitude latitude lat_long
## Min. : 0.00 Min. :-17.440 Min. :-25.967 Length:11299
## 1st Qu.: 0.00 1st Qu.: -3.708 1st Qu.: -1.296 Class :character
## Median : 8.10 Median : 11.450 Median : 7.370 Mode :character
## Mean : 21.87 Mean : 13.378 Mean : 4.594
## 3rd Qu.: 37.50 3rd Qu.: 32.750 3rd Qu.: 13.128
## Max. :100.00 Max. : 47.433 Max. : 20.258
## loc_spp parasite_s yearof_pub bio01
## Length:11299 Length:11299 Min. : 0 Min. :12.08
## Class :character Class :character 1st Qu.:1982 1st Qu.:23.00
## Mode :character Mode :character Median :1990 Median :25.91
## Mean :1839 Mean :25.10
## 3rd Qu.:1996 3rd Qu.:27.40
## Max. :2010 Max. :30.43
## bio02 bio03 bio04 bio05
## Min. : 5.90 Min. :43.00 Min. : 1.469 Min. :20.07
## 1st Qu.:10.70 1st Qu.:57.00 1st Qu.:10.448 1st Qu.:31.30
## Median :13.00 Median :63.00 Median :18.046 Median :34.00
## Mean :12.54 Mean :65.46 Mean :18.371 Mean :34.66
## 3rd Qu.:14.40 3rd Qu.:72.56 3rd Qu.:25.490 3rd Qu.:39.00
## Max. :19.90 Max. :91.87 Max. :53.857 Max. :42.60
## bio06 bio07 bio08 bio09
## Min. : 1.50 Min. : 7.90 Min. :11.26 Min. :10.30
## 1st Qu.:13.56 1st Qu.:14.50 1st Qu.:23.10 1st Qu.:21.90
## Median :15.30 Median :19.90 Median :25.70 Median :24.60
## Mean :15.11 Mean :19.54 Mean :25.09 Mean :23.72
## 3rd Qu.:17.50 3rd Qu.:24.25 3rd Qu.:27.20 3rd Qu.:26.30
## Max. :22.90 Max. :31.10 Max. :35.60 Max. :31.87
## bio10 bio11 bio12 bio13
## Min. :13.28 Min. :10.30 Min. : 11 Min. : 3.0
## 1st Qu.:24.60 1st Qu.:21.50 1st Qu.: 615 1st Qu.: 167.4
## Median :27.60 Median :23.43 Median : 977 Median : 216.0
## Mean :27.35 Mean :22.66 Mean :1043 Mean : 230.5
## 3rd Qu.:30.50 3rd Qu.:24.96 3rd Qu.:1314 3rd Qu.: 282.0
## Max. :36.15 Max. :28.70 Max. :4226 Max. :1139.4
## bio14 bio15 bio16 bio17
## Min. : 0.000 Min. : 25.16 Min. : 7.0 Min. : 0.00
## 1st Qu.: 0.000 1st Qu.: 66.00 1st Qu.: 408.0 1st Qu.: 0.00
## Median : 1.000 Median : 99.00 Median : 536.2 Median : 6.00
## Mean : 8.618 Mean : 98.11 Mean : 573.4 Mean : 39.41
## 3rd Qu.:13.000 3rd Qu.:128.00 3rd Qu.: 694.0 3rd Qu.: 64.00
## Max. :94.000 Max. :183.00 Max. :2798.8 Max. :337.74
## bio18 bio19 hnd upa
## Min. : 4.00 Min. : 0.0 Min. : 0.000 Min. : 0.01
## 1st Qu.: 92.34 1st Qu.: 2.0 1st Qu.: 2.267 1st Qu.: 0.01
## Median :188.00 Median : 26.0 Median : 6.740 Median : 0.03
## Mean :212.19 Mean : 223.8 Mean : 14.656 Mean : 161.74
## 3rd Qu.:301.00 3rd Qu.: 380.6 3rd Qu.: 16.477 3rd Qu.: 0.10
## Max. :996.00 Max. :2798.8 Max. :408.948 Max. :276274.60
## slope gHM elevation pH
## Min. : 0.000 Min. :0.0000 Min. : -2.748 Min. :4.024
## 1st Qu.: 1.256 1st Qu.:0.3147 1st Qu.: 196.086 1st Qu.:5.602
## Median : 1.891 Median :0.5264 Median : 366.192 Median :6.018
## Mean : 2.794 Mean :0.5060 Mean : 566.103 Mean :6.116
## 3rd Qu.: 3.002 3rd Qu.:0.6845 3rd Qu.: 952.402 3rd Qu.:6.501
## Max. :47.420 Max. :0.9630 Max. :3181.426 Max. :9.039
## aspect clay sand
## Min. : 0.0 Min. : 0.00 Min. :16.13
## 1st Qu.:116.4 1st Qu.:18.00 1st Qu.:47.47
## Median :166.0 Median :23.92 Median :56.79
## Mean :167.1 Mean :25.12 Mean :56.40
## 3rd Qu.:217.4 3rd Qu.:31.99 3rd Qu.:65.33
## Max. :350.1 Max. :58.72 Max. :99.00This concludes the section of quality control dealing with missing values as there are no more missing values left in my dataset.
There were no obvious outliers in the dataset. I would also include them in my dataset because they could be good examples to uncover some unique characteristic behind the spread of schistosomiasis and parasite prevalence.
To test Hypothesis 2, I thought that it would be a good idea to plot the correlation of pH and percent_pos.
The correlation plot shows how there is a higher concentration of points in the pH 5 - 7 range. This is further confirmed by the graph of the points representing average percent positivity per pH bucket. This disproved Hypothesis 2, which expected that there would be a higher percentage positivity closer to pH = 7. Through these visualizations, the highest average percentage positivity found in the pH 9 to 9.5 range.
To test Hypothesis 3, I will be plotting a correlation plot between “bio12,” the mean annual precipitation, and percent_pos.
##
## Pearson's product-moment correlation
##
## data: Datasci120_Dataset$pH and Datasci120_Dataset$percent_pos
## t = 8.6014, df = 11297, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## 0.06231608 0.09895414
## sample estimates:
## cor
## 0.08066236The correlation test signifies a very weak positively linear correlation between mean annual precipitation and percent_pos. The Pearson’s Correlation Co-efficient is 0.081 with a p-value of 2.2e-16 < 0.05, exhibiting statistical significance. I feel like I still want to investigate this hypothesis more to display a clearer image. I believe some other sort of causal method is necessary which takes into account other precipitation variables.
With this result in mind, I need to consider new ways to account for all the variables linked to an environmental condition (eg. precipitation). I will be discussing this further with Andy. Based on our discussion, I will revise my hypotheses for Week 5.
Since the nature of my question revolves around modelling different combinations of features in order to develop seasonal indices for the spread of schistosomiasis, I am going to be running different machine learning models in an attempt to get high accuracy at predicting percent_pos.
The first model that I will be building will be trying out different regression models in order to predict percent_pos based on precipitation variables. The variables being used to predict include bio12 through bio19 and pH (close to 7 for water). This stems from my hypothesis regarding how precipitation levels and soil pH inform us about water presence and schistosomiasis is best spread through water bodies. While the correlation plots with annual precipitation, pH, and percent_pos did not reveal much information, I still wanted to test for a linear relationship using linear regression.
Before modelling, I also scaled the features because of how bio12 to bio19 were majorly precipitation variables (measured in mm), while pH was on a completely different scale.
## Loading required package: lattice## RMSE: 27.49734The RMSE of this model is ~27.5. This means that this linear regression model is ~27.5% off the actual percent_pos, which is a poor performance.
Next, I will try and capture more complex relationships by introducing polynomial terms in my regression. The maximum polynomial degree will be set to 3.
## RMSE: 27.99734The RMSE went from ~27.5 to ~28, which shows that the model is not getting better.
To inform which direction I should head in, my next model is going to be a random forest to understand the importance of the different precipitation related features that may contribute to a lower RMSE and higher accuracy.
## randomForest 4.7-1.1## Type rfNews() to see new features/changes/bug fixes.##
## Attaching package: 'randomForest'## The following object is masked from 'package:gridExtra':
##
## combine## The following object is masked from 'package:dplyr':
##
## combine## The following object is masked from 'package:ggplot2':
##
## margin## RMSE: 26.19179## %IncMSE IncNodePurity
## bio12 18.71160 760106.0
## bio13 20.29406 711763.4
## bio14 19.35597 264452.9
## bio15 22.36266 663444.1
## bio16 17.24521 690808.2
## bio17 17.18870 395089.5
## bio18 27.52506 752815.3
## bio19 28.88613 525231.0
## pH 27.47590 849176.5
At this point, I will try and optimize the random forest model by trying to coarsely increase the ntrees parameter. This is just to investigate how I can potentially tune this model in future reports.
## RMSE: 26.17219## %IncMSE IncNodePurity
## bio12 31.17049 765143.9
## bio13 34.25310 708296.9
## bio14 30.64120 261526.5
## bio15 34.58636 664299.2
## bio16 31.34496 704933.4
## bio17 28.07282 393726.0
## bio18 43.96247 753503.3
## bio19 42.26161 526055.6
## pH 42.11598 850896.2
## RMSE: 26.13861## %IncMSE IncNodePurity
## bio12 41.66118 769789.6
## bio13 50.36810 712689.8
## bio14 42.18767 259946.3
## bio15 50.12475 656717.1
## bio16 46.64173 709819.0
## bio17 40.97751 395392.5
## bio18 63.79080 755937.4
## bio19 54.48978 519261.0
## pH 61.03038 847040.5
After reaching an RMSE of ~26.1%, it seems like the numtrees parameter does not have much impact on the performance of the model.
Next, I want to try and evaluate the temperature related variables. This includes “bio01” through “bio11.” I am not going to need to perform any scaling as they are all on a similar scale of degree Celsius, with the data distributed similarly.
I will be using the same models to evaluate the temperature variables as I did for the precipitation variables.
## RMSE: 27.31273## RMSE: 124.2487## RMSE: 26.03471## %IncMSE IncNodePurity
## bio01 16.27019 444485.4
## bio02 16.97824 494071.0
## bio03 19.36415 378267.2
## bio04 18.24491 705245.8
## bio05 15.96811 520077.3
## bio06 21.14883 498983.6
## bio07 21.27133 522903.5
## bio08 18.02647 538884.4
## bio09 20.79928 526717.0
## bio10 16.65569 472562.5
## bio11 14.91093 480532.2
## RMSE: 25.98278## %IncMSE IncNodePurity
## bio01 26.32341 443236.7
## bio02 29.19178 492033.0
## bio03 28.38614 377715.2
## bio04 29.00288 710593.8
## bio05 26.05010 518198.1
## bio06 33.20102 506426.7
## bio07 34.48934 522136.7
## bio08 30.26219 542975.2
## bio09 30.91606 523327.5
## bio10 22.81584 478060.2
## bio11 24.60950 473626.1
## RMSE: 26.01661## %IncMSE IncNodePurity
## bio01 36.19831 443094.2
## bio02 43.56605 497173.4
## bio03 39.01341 377290.2
## bio04 42.86117 707575.8
## bio05 36.97380 516220.5
## bio06 48.00249 506324.9
## bio07 48.22672 521843.0
## bio08 41.56086 543248.6
## bio09 45.95790 525776.5
## bio10 33.30884 480590.7
## bio11 37.14408 471689.8
The final RMSE from the random forest models came to be ~26%.
From this point, I plan to meet my supervisor early next week and include more relevant and targeted models in my Week 6 report. I have informed Ran about why I cannot meet him this week and will add to my analysis after my meeting with him.
Dr. Sabatti’s feedback proposed several changes to my existing report, which I will make after I rewrite my report with the added analysis. However, Dr. Sabatti said one thing that was missing was where my data was centered since it had latitude and longitude coordinates. To incorporate this in my report, the data is centered around Africa with all of the studies (or observations) in the dataset are in different locations in Africa.
A plot of the data in geographical form is below.
The plot shows that the data is all from Africa.
I plan on providing more geographical insight into my analyses and modelling for future reports.