CPLN 675: Homework 3

1. Introduction

Climate change impacts such as sea level rise and increasingly severe and frequent weather events like heavy rains exacerbate present flood threats. Identifying flood vulnerable areas and predicting where flooding may occur is integral to planning for a climate-changed future. The following report is a part of an effort to develop and test a predictive flood model:

We have been tasked with creating a model that predicts areas vulnerable to flooding. In order to do so, we chose an area where we already had recorded flood data: Calgary, Alberta (CAN). In addition, we needed to choose a comparable city to Calgary, one that could test our model. We chose Washington DC. Both Calgary and DC have large rivers running through the cities and are large population centers.

After choosing our cities, we came up with potential factors that might lead to flooding. Those factors are:

• Elevation

• Impervious Surfaces

• Parks

• Low Permeable Soils (e.g. clay)

• Waterbodies

For each factor, we came up with a hypothesis:

• Elevation: If an area is at a low elevation, it will most likely flood.

• Impervious Surfaces: If an area has or is made up of many impervious surfaces, it will most likely flood.

• Parks: If an area has or is made up of many green spaces, it will most likely not flood.

• Low Permeable Soils: If an area has or is made up of many low permeable soils, it will most likely flood.

• Waterbodies: If an area has or is made up of existing waterbodies, it will most likely flood.

To test these hypotheses and build a predictive model, we conducted feature engineering, spatial lags, and a generalized linear model (GLM). The following sections highlight our key features, our regression model summary, and our final flood predictions for Calgary and DC.

Ultimately, the DC predictive model did not perform how we would have hoped; however, we found promising results in our Calgary model. This draws us to conclude that Washington DC may not have been a good fit as a comparison city. For example, while Calgary is far more inland and has a drier climate, DC is relatively close to the Chesapeake Bay and tends to be more humid. Other US cities that might have been more comparable to Calgary could have been Denver, CO, or Salt Lake City, UT.

2. Maps of Variables

To begin our modeling process in R, we uploaded our Calgary fishnet and our predictive factors or “variables”: elevation, impervious surfaces, low permeable soils, parks, and waterbodies. A fishnet is a grid made up of squares–like a net–called “cells” overlaid on some geographic area. Fishnets have many uses, but our main purpose was to use the fishnet for spatial analysis and modeling.

The fishnet allows us to measure the presence or “how much” of a variable is found within a certain area or cell. By running zonal statistics for each variable in ArcGIS, we determined what was the best measure for each variable:

• Elevation: We decided to measure the lowest point of each cell, or the minimum value of elevation. We operated under the assumption that the lowest point of every cell is where most water within the cell will gather.

• Impervious Surfaces: If the mean count of “impervious surface” pixels within a cell was equal to or exceeded 25% of a cell, then the cell possessed impervious surfaces.

• Low Permeable Soils: If the mean count of “low permeable soil” pixels within a cell was equal to or exceeded 25% of a cell, then the cell possessed low permeable soils.

• Parks: If the mean count of “park” pixels within a cell was equal to or exceeded 25%, then the cell possessed a park.

• Waterbodies: If the mean count of “waterbodies” pixels within a cell was equal to or exceeded 25%, then the cell possessed a waterbody.

Four of these maps are visualized in the next section.

2.1. Impervious Surfaces

2.2. Parks

2.3. Low Permeable Soils

2.4. Existing Waterbodies

3. Model Validation

3.1. Exploratory Analysis: Summary Table

The summary table shows us some interesting aspects of our model. For example, our hypothesis concerning low elevation was that cells with low elevation are more likely to flood. However, the summary table shows that out of the total cells with low elevation, many did not flood. The same is said for low permeable soils and impervious surfaces. We predicted that areas with low permeability or impermeability were more likely to flood, but the summary table shows the opposite was true.

Furthermore, our prediction concerning parks followed that cells with parks were less likely to flood, but the summary table shows that cells with parks were actually more likely to flood.

Our summary table, however, may be telling us a positive story: what do we know about Calgary’s stormwater infrastructure? If Calgary has effective stormwater infrastructure, then this may explain why areas of low elevation and parks are flooded while areas with impervious surfaces and soils are not. For example, it is standard green stormwater infrastructure practice to use parks as multi-use spaces: during heavy rains, parks can serve as stormwater holding basins and/or help absorb excess water. In this sense, parks are engineered or designed to be flooded.

Additionally, if Calgary’s stormwater infrastructure is effective, then water should be redirected away from urban areas. Stormwater infrastructure is often paired with city’s water systems, and many cities–including Calgary–utilize gravity systems, which take advantage of gravity as a force to direct water flow “downwards” or towards areas of low elevation (often waterbodies or water treatment plants).

Although our summary table shows that our variables are not necessarily following our assumptions, they still may be indicative of a relationship, and even a positive story!

3.2. Logistic Regression Model Summary

Next, we run a regression summary to determine our variables statistical significance. Based on our p-values and coefficient estimates, our regression summary shows that our variables are statistically significant, meaning there is some relationship between flooding and the presence of our variables.

floodModel <- glm(Flooded ~ ., 
                     family="binomial"(link="logit"), data = floodTrain %>%
                       as.data.frame() %>%
                       dplyr::select(-geometry, -cell))
summary(floodModel)

## 
## Call:
## glm(formula = Flooded ~ ., family = binomial(link = "logit"), 
##     data = floodTrain %>% as.data.frame() %>% dplyr::select(-geometry, 
##         -cell))
## 
## Coefficients:
##                      Estimate Std. Error z value             Pr(>|z|)    
## (Intercept)          1.873002   0.214924   8.715 < 0.0000000000000002 ***
## Low_permeability    -1.348975   0.142291  -9.480 < 0.0000000000000002 ***
## Parks_Majority       1.977298   0.165694  11.933 < 0.0000000000000002 ***
## Impervious_majority -0.520810   0.139380  -3.737             0.000187 ***
## elevation_min       -0.028850   0.002025 -14.246 < 0.0000000000000002 ***
## Water_majority       2.037572   0.312215   6.526      0.0000000000675 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 2253.5  on 2498  degrees of freedom
## Residual deviance: 1559.3  on 2493  degrees of freedom
## AIC: 1571.3
## 
## Number of Fisher Scoring iterations: 6

3.3. Confusion Metrics & ROC Curve

Confusion Matrix and Statistics

      Reference

Prediction 0 1 0 766 56 1 137 111

           Accuracy : 0.8196          
             95% CI : (0.7953, 0.8422)
No Information Rate : 0.8439          
P-Value [Acc > NIR] : 0.986           
                                      
              Kappa : 0.4283          
                                      

Mcnemar’s Test P-Value : 0.000000008485

        Sensitivity : 0.6647          
        Specificity : 0.8483          
     Pos Pred Value : 0.4476          
     Neg Pred Value : 0.9319          
         Prevalence : 0.1561          
     Detection Rate : 0.1037          

Detection Prevalence : 0.2318
Balanced Accuracy : 0.7565

   'Positive' Class : 1               
                                      

According to the Confusion Matrix, our model runs at about an 82% accuracy rate and is best at predicting which cells are not flooded, but is less accurate at predicting which cells will flood.

For example, the Confusion Matrix shows that our model predicted 903 cells would not flood. Out of those 903 cells, 766 cells were predicted to not flood and in reality, did not flood based off of our recorded Calgary flood data; however, 137 of the cells predicted to not flood, actually did flood. On the other hand, out of 167 cells, our model correctly predicted 111 cells would flood, but incorrectly predicted 56 cells would flood. In actuality, those 56 cells did not flood.

This information is summed up in the sensitivity and specificity measure of our model. The Confusion Matrix shows that our model’s sensitivity, that is, its accuracy in predicting cells that will flood is about 67%; our model’s specificity, that is, its accuracy in predicting cells that will not flood is about 85%.

Initially, we ran our confusion matrix with a 50% probability level. However, after analyzing our ROC curve and confusion metrics, we decided to lower our probability threshold.

Our ROC curve visualizes a general diagnostic of our model’s performance. Based on our probability curve and confusion metrics, we decided that a threshold of 25% is the best probability level for our model to predict cells as flooded or not flooded. In other words, a 25% probability threshold is where our model most accurately predicts flooded and not flooded cells.

3.4. Cross Validation

Our cross validation plot shows similar accuracies across 100 tests, making it a promising model to introduce ‘out-of-sample’ data like DC.

4. Prediction Maps

4.1. Confusion Metrics for Training Set

4.2. Calgary Predictions

4.3. DC Predictions