reproducible research assignment 2 by Kris Vercauteren

Synopsis

Storms and other severe weather events can cause both public health and economic problems for communities and municipalities. Many severe events can result in fatalities, injuries, and property damage, and preventing such outcomes to the extent possible is a key concern.

This project involves exploring the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.

Data processing

The data for this assignment comes in the form of a comma-separated-value file compressed via the bzip2 algorithm to reduce its size. You can download the file from the course web site:

There is also some documentation of the database available. Here you will find how some of the variables are constructed/defined.

Read the data in

file.url <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"

if( !file.exists( "StormData.csv.bz2" )) {
  download.file( file.url, "StormData.csv.bz2" )
}

noaaDF <- read.csv( bzfile( 'StormData.csv.bz2' ), header = TRUE, stringsAsFactors = FALSE )

Libraries to load

require( dplyr )
## Loading required package: dplyr
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
require( tidyr )
## Loading required package: tidyr
require( lubridate )
## Loading required package: lubridate
## 
## Attaching package: 'lubridate'
## The following objects are masked from 'package:base':
## 
##     date, intersect, setdiff, union
require( ggplot2 )
## Loading required package: ggplot2

Preliminary analysis

First a summary of the NU.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database

summary( noaaDF )
##     STATE__       BGN_DATE           BGN_TIME          TIME_ZONE        
##  Min.   : 1.0   Length:902297      Length:902297      Length:902297     
##  1st Qu.:19.0   Class :character   Class :character   Class :character  
##  Median :30.0   Mode  :character   Mode  :character   Mode  :character  
##  Mean   :31.2                                                           
##  3rd Qu.:45.0                                                           
##  Max.   :95.0                                                           
##                                                                         
##      COUNTY       COUNTYNAME           STATE              EVTYPE         
##  Min.   :  0.0   Length:902297      Length:902297      Length:902297     
##  1st Qu.: 31.0   Class :character   Class :character   Class :character  
##  Median : 75.0   Mode  :character   Mode  :character   Mode  :character  
##  Mean   :100.6                                                           
##  3rd Qu.:131.0                                                           
##  Max.   :873.0                                                           
##                                                                          
##    BGN_RANGE          BGN_AZI           BGN_LOCATI          END_DATE        
##  Min.   :   0.000   Length:902297      Length:902297      Length:902297     
##  1st Qu.:   0.000   Class :character   Class :character   Class :character  
##  Median :   0.000   Mode  :character   Mode  :character   Mode  :character  
##  Mean   :   1.484                                                           
##  3rd Qu.:   1.000                                                           
##  Max.   :3749.000                                                           
##                                                                             
##    END_TIME           COUNTY_END COUNTYENDN       END_RANGE       
##  Length:902297      Min.   :0    Mode:logical   Min.   :  0.0000  
##  Class :character   1st Qu.:0    NA's:902297    1st Qu.:  0.0000  
##  Mode  :character   Median :0                   Median :  0.0000  
##                     Mean   :0                   Mean   :  0.9862  
##                     3rd Qu.:0                   3rd Qu.:  0.0000  
##                     Max.   :0                   Max.   :925.0000  
##                                                                   
##    END_AZI           END_LOCATI            LENGTH              WIDTH         
##  Length:902297      Length:902297      Min.   :   0.0000   Min.   :   0.000  
##  Class :character   Class :character   1st Qu.:   0.0000   1st Qu.:   0.000  
##  Mode  :character   Mode  :character   Median :   0.0000   Median :   0.000  
##                                        Mean   :   0.2301   Mean   :   7.503  
##                                        3rd Qu.:   0.0000   3rd Qu.:   0.000  
##                                        Max.   :2315.0000   Max.   :4400.000  
##                                                                              
##        F               MAG            FATALITIES          INJURIES        
##  Min.   :0.0      Min.   :    0.0   Min.   :  0.0000   Min.   :   0.0000  
##  1st Qu.:0.0      1st Qu.:    0.0   1st Qu.:  0.0000   1st Qu.:   0.0000  
##  Median :1.0      Median :   50.0   Median :  0.0000   Median :   0.0000  
##  Mean   :0.9      Mean   :   46.9   Mean   :  0.0168   Mean   :   0.1557  
##  3rd Qu.:1.0      3rd Qu.:   75.0   3rd Qu.:  0.0000   3rd Qu.:   0.0000  
##  Max.   :5.0      Max.   :22000.0   Max.   :583.0000   Max.   :1700.0000  
##  NA's   :843563                                                           
##     PROPDMG         PROPDMGEXP           CROPDMG         CROPDMGEXP       
##  Min.   :   0.00   Length:902297      Min.   :  0.000   Length:902297     
##  1st Qu.:   0.00   Class :character   1st Qu.:  0.000   Class :character  
##  Median :   0.00   Mode  :character   Median :  0.000   Mode  :character  
##  Mean   :  12.06                      Mean   :  1.527                     
##  3rd Qu.:   0.50                      3rd Qu.:  0.000                     
##  Max.   :5000.00                      Max.   :990.000                     
##                                                                           
##      WFO             STATEOFFIC         ZONENAMES            LATITUDE   
##  Length:902297      Length:902297      Length:902297      Min.   :   0  
##  Class :character   Class :character   Class :character   1st Qu.:2802  
##  Mode  :character   Mode  :character   Mode  :character   Median :3540  
##                                                           Mean   :2875  
##                                                           3rd Qu.:4019  
##                                                           Max.   :9706  
##                                                           NA's   :47    
##    LONGITUDE        LATITUDE_E     LONGITUDE_       REMARKS         
##  Min.   :-14451   Min.   :   0   Min.   :-14455   Length:902297     
##  1st Qu.:  7247   1st Qu.:   0   1st Qu.:     0   Class :character  
##  Median :  8707   Median :   0   Median :     0   Mode  :character  
##  Mean   :  6940   Mean   :1452   Mean   :  3509                     
##  3rd Qu.:  9605   3rd Qu.:3549   3rd Qu.:  8735                     
##  Max.   : 17124   Max.   :9706   Max.   :106220                     
##                   NA's   :40                                        
##      REFNUM      
##  Min.   :     1  
##  1st Qu.:225575  
##  Median :451149  
##  Mean   :451149  
##  3rd Qu.:676723  
##  Max.   :902297  
## 

As you can see, a lot of columns are present. Lets filter out only the ones required for our analysis.

stormDF <- select( noaaDF, c( "EVTYPE", "FATALITIES", "INJURIES", "PROPDMG", 
                              "PROPDMGEXP", "CROPDMG", "CROPDMGEXP" ))
summary( stormDF )
##     EVTYPE            FATALITIES          INJURIES            PROPDMG       
##  Length:902297      Min.   :  0.0000   Min.   :   0.0000   Min.   :   0.00  
##  Class :character   1st Qu.:  0.0000   1st Qu.:   0.0000   1st Qu.:   0.00  
##  Mode  :character   Median :  0.0000   Median :   0.0000   Median :   0.00  
##                     Mean   :  0.0168   Mean   :   0.1557   Mean   :  12.06  
##                     3rd Qu.:  0.0000   3rd Qu.:   0.0000   3rd Qu.:   0.50  
##                     Max.   :583.0000   Max.   :1700.0000   Max.   :5000.00  
##   PROPDMGEXP           CROPDMG         CROPDMGEXP       
##  Length:902297      Min.   :  0.000   Length:902297     
##  Class :character   1st Qu.:  0.000   Class :character  
##  Mode  :character   Median :  0.000   Mode  :character  
##                     Mean   :  1.527                     
##                     3rd Qu.:  0.000                     
##                     Max.   :990.000

Based on the information above, the data table now has 902.297 rows and 7 columns. Below is a brief description of each variable :

  1. evtype : storm event type
  2. fatalities: amount of fatalities per event
  3. injuries : amount of injuries per event
  4. propdmg : property damage amount
  5. propdmgexp: property damage in exponents
  6. cropdmg : crop damage amount
  7. cropdmgexp: crop damage in exponents

The variables PROPDMGEXP and CROPDMGEXP are regarding property damage expenses, so it can be utilized to denote the events with greatest economic consequences. The values are messy, so I created a function to deal with that, and to calculate the cost with their respective exponent values (but in millions).

cost <- function( x ) {
  if( x == "H" )
    1E-4
  else if( x == "K" )
    1E-3
  else if( x == "M" )
    1
  else if( x == "B" )
    1E3
  else
    1-6
}

Results

1. Address the question of which types of events are most harmful to population health

First calculate the fatalities and sort them descending ( only retain top 20 of the list )

totFatalities <- aggregate( stormDF$FATALITIES, by = list( stormDF$EVTYPE ), "sum" )
names( totFatalities ) <- c( "Event", "Fatalities" )
totFatalitiesSorted <- totFatalities[ order( -totFatalities$Fatalities ), ][ 1:20, ]
totFatalitiesSorted
##                       Event Fatalities
## 834                 TORNADO       5633
## 130          EXCESSIVE HEAT       1903
## 153             FLASH FLOOD        978
## 275                    HEAT        937
## 464               LIGHTNING        816
## 856               TSTM WIND        504
## 170                   FLOOD        470
## 585             RIP CURRENT        368
## 359               HIGH WIND        248
## 19                AVALANCHE        224
## 972            WINTER STORM        206
## 586            RIP CURRENTS        204
## 278               HEAT WAVE        172
## 140            EXTREME COLD        160
## 760       THUNDERSTORM WIND        133
## 310              HEAVY SNOW        127
## 141 EXTREME COLD/WIND CHILL        125
## 676             STRONG WIND        103
## 30                 BLIZZARD        101
## 350               HIGH SURF        101

and then then injuries seperately.

totInjuries <- aggregate( stormDF$INJURIES, by = list( stormDF$EVTYPE ), "sum" )
names( totInjuries ) <- c( "Event", "Injuries" )
totInjuriesSorted <- totInjuries[ order( -totInjuries$Injuries ), ][ 1:20, ]
totInjuriesSorted
##                  Event Injuries
## 834            TORNADO    91346
## 856          TSTM WIND     6957
## 170              FLOOD     6789
## 130     EXCESSIVE HEAT     6525
## 464          LIGHTNING     5230
## 275               HEAT     2100
## 427          ICE STORM     1975
## 153        FLASH FLOOD     1777
## 760  THUNDERSTORM WIND     1488
## 244               HAIL     1361
## 972       WINTER STORM     1321
## 411  HURRICANE/TYPHOON     1275
## 359          HIGH WIND     1137
## 310         HEAVY SNOW     1021
## 957           WILDFIRE      911
## 786 THUNDERSTORM WINDS      908
## 30            BLIZZARD      805
## 188                FOG      734
## 955   WILD/FOREST FIRE      545
## 117         DUST STORM      440

Finally plot both the fatalities and injuries in a single plot

par( mfrow = c( 1, 2 ), mar = c( 10, 4, 2, 2 ), las = 3, cex = 0.7, cex.main = 1.4, cex.lab = 1.2 )
barplot( totFatalitiesSorted$Fatalities, names.arg = totFatalitiesSorted$Event, col = 'red',
         main = 'Top 20 Weather Events for Fatalities', ylab = 'Number of Fatalities' )
barplot( totInjuriesSorted$Injuries, names.arg = totInjuriesSorted$Event, col = 'orange',
         main = 'Top 20 Weather Events for Injuries', ylab = 'Number of Injuries' )

Conclusion : we see that tornados cause most deaths and injuries in the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. But excessive heat causes second most deaths, whereas as far as injuries are concerned second to fourth causes have very similar values.

2. Address the question of which types of events have the greatest economic consequences

Let align on what we have done for question 1 and first calculate the cost of property

totProperty <- aggregate( stormDF$PROPDMG * sapply( stormDF$PROPDMGEXP, FUN = "cost" ), 
                          by = list( stormDF$EVTYPE ), "sum" )
names( totProperty ) <- c( "Event", "Property" )
totPropertySorted <- totProperty[ order( -totProperty$Property ), ][ 1:20, ]
totPropertySorted
##                          Event   Property
## 170                      FLOOD 144622.710
## 411          HURRICANE/TYPHOON  69305.840
## 834                    TORNADO  55374.660
## 670                STORM SURGE  43323.536
## 244                       HAIL  14089.367
## 153                FLASH FLOOD  13355.312
## 402                  HURRICANE  11868.319
## 848             TROPICAL STORM   7703.891
## 972               WINTER STORM   6683.497
## 590                RIVER FLOOD   5118.945
## 359                  HIGH WIND   5095.046
## 957                   WILDFIRE   4765.114
## 671           STORM SURGE/TIDE   4641.188
## 856                  TSTM WIND   4209.928
## 427                  ICE STORM   3694.928
## 409             HURRICANE OPAL   3052.846
## 955           WILD/FOREST FIRE   3001.829
## 760          THUNDERSTORM WIND   2763.121
## 298  HEAVY RAIN/SEVERE WEATHER   2500.000
## 842 TORNADOES, TSTM WIND, HAIL   1600.000

And seperately calculate crop damages

totCrop <- aggregate( stormDF$CROPDMG * sapply( stormDF$CROPDMGEXP, FUN = "cost" ), 
                      by = list( stormDF$EVTYPE), "sum" )
names( totCrop ) <- c( "Event", "Crop" )
totCropSorted <- totCrop[ order( -totCrop$Crop), ][ 1:20, ]
totCropSorted
##                 Event       Crop
## 95            DROUGHT 13972.5660
## 170             FLOOD  5661.9685
## 590       RIVER FLOOD  5029.4590
## 427         ICE STORM  5022.1135
## 402         HURRICANE  2741.9100
## 411 HURRICANE/TYPHOON  2607.8728
## 153       FLASH FLOOD  1421.3171
## 140      EXTREME COLD  1292.9730
## 212      FROST/FREEZE  1094.0860
## 244              HAIL   825.5375
## 290        HEAVY RAIN   733.3998
## 848    TROPICAL STORM   678.3460
## 359         HIGH WIND   638.5713
## 856         TSTM WIND   554.0073
## 130    EXCESSIVE HEAT   492.4020
## 192            FREEZE   446.2250
## 760 THUNDERSTORM WIND   414.8431
## 275              HEAT   401.4615
## 957          WILDFIRE   295.4728
## 87    DAMAGING FREEZE   262.1000

Next plot both the cost of property and crop damages in a single plot:

par( mfrow = c( 1, 2 ), mar = c( 10, 4, 2, 2 ), las = 3, cex = 0.7, cex.main = 1.4, cex.lab = 1.2 )
barplot( totPropertySorted$Property, names.arg = totPropertySorted$Event, col = 'Brown',
         main = 'Top 20 Weather Events for Property Damage ', ylab = 'Amount of Property Damage' )
barplot( totCropSorted$Crop, names.arg = totCropSorted$Event, col = 'Green',
         main = 'Top 20 Weather Events for Crop Damage', ylab = 'Amount of  Crop Damage' )

Finally the total damage by adding both costs (property and crop damage)

totTotalCost <- aggregate(( stormDF$CROPDMG * sapply( stormDF$CROPDMGEXP, FUN = "cost" )) + 
                          ( stormDF$PROPDMG * sapply( stormDF$PROPDMGEXP, FUN = "cost" )), 
                          by = list( stormDF$EVTYPE), "sum" )
names( totTotalCost ) <- c( "Event", "TotalCost" )
totTotalCostSorted <- totTotalCost[ order( -totTotalCost$TotalCost ), ][ 1:20, ]
totTotalCostSorted
##                         Event  TotalCost
## 170                     FLOOD 150284.678
## 411         HURRICANE/TYPHOON  71913.713
## 834                   TORNADO  54989.614
## 670               STORM SURGE  43323.541
## 95                    DROUGHT  15018.672
## 244                      HAIL  14914.905
## 153               FLASH FLOOD  14776.629
## 402                 HURRICANE  14610.229
## 590               RIVER FLOOD  10148.405
## 427                 ICE STORM   8717.041
## 848            TROPICAL STORM   8382.237
## 972              WINTER STORM   6710.441
## 359                 HIGH WIND   5733.618
## 957                  WILDFIRE   5060.587
## 856                 TSTM WIND   4763.936
## 671          STORM SURGE/TIDE   4642.038
## 760         THUNDERSTORM WIND   3177.964
## 955          WILD/FOREST FIRE   3108.626
## 409            HURRICANE OPAL   3011.846
## 298 HEAVY RAIN/SEVERE WEATHER   2500.000

And a single plot for visualization

par( mfrow = c( 1, 1 ), mar = c( 10, 4, 2, 2 ), las = 3, cex = 0.7, cex.main = 1.4, cex.lab = 1.2 )
barplot( totTotalCostSorted$TotalCost, names.arg = totTotalCostSorted$Event, col = 'Black',
         main = 'Top 20 Weather Events for total Damage ', ylab = 'Amount of total Damage' )

Conclusion : we see that flood causes the most economic damage overall, as well as for property damage. But drought tops the ranking for crop damage.