reproducible research assignment 2 by Kris Vercauteren
Storms and other severe weather events can cause both public health and economic problems for communities and municipalities. Many severe events can result in fatalities, injuries, and property damage, and preventing such outcomes to the extent possible is a key concern.
This project involves exploring the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. This database tracks characteristics of major storms and weather events in the United States, including when and where they occur, as well as estimates of any fatalities, injuries, and property damage.
The data for this assignment comes in the form of a comma-separated-value file compressed via the bzip2 algorithm to reduce its size. You can download the file from the course web site:
There is also some documentation of the database available. Here you will find how some of the variables are constructed/defined.
file.url <- "https://d396qusza40orc.cloudfront.net/repdata%2Fdata%2FStormData.csv.bz2"
if( !file.exists( "StormData.csv.bz2" )) {
download.file( file.url, "StormData.csv.bz2" )
}
noaaDF <- read.csv( bzfile( 'StormData.csv.bz2' ), header = TRUE, stringsAsFactors = FALSE )
require( dplyr )
## Loading required package: dplyr
##
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
##
## filter, lag
## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union
require( tidyr )
## Loading required package: tidyr
require( lubridate )
## Loading required package: lubridate
##
## Attaching package: 'lubridate'
## The following objects are masked from 'package:base':
##
## date, intersect, setdiff, union
require( ggplot2 )
## Loading required package: ggplot2
First a summary of the NU.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database
summary( noaaDF )
## STATE__ BGN_DATE BGN_TIME TIME_ZONE
## Min. : 1.0 Length:902297 Length:902297 Length:902297
## 1st Qu.:19.0 Class :character Class :character Class :character
## Median :30.0 Mode :character Mode :character Mode :character
## Mean :31.2
## 3rd Qu.:45.0
## Max. :95.0
##
## COUNTY COUNTYNAME STATE EVTYPE
## Min. : 0.0 Length:902297 Length:902297 Length:902297
## 1st Qu.: 31.0 Class :character Class :character Class :character
## Median : 75.0 Mode :character Mode :character Mode :character
## Mean :100.6
## 3rd Qu.:131.0
## Max. :873.0
##
## BGN_RANGE BGN_AZI BGN_LOCATI END_DATE
## Min. : 0.000 Length:902297 Length:902297 Length:902297
## 1st Qu.: 0.000 Class :character Class :character Class :character
## Median : 0.000 Mode :character Mode :character Mode :character
## Mean : 1.484
## 3rd Qu.: 1.000
## Max. :3749.000
##
## END_TIME COUNTY_END COUNTYENDN END_RANGE
## Length:902297 Min. :0 Mode:logical Min. : 0.0000
## Class :character 1st Qu.:0 NA's:902297 1st Qu.: 0.0000
## Mode :character Median :0 Median : 0.0000
## Mean :0 Mean : 0.9862
## 3rd Qu.:0 3rd Qu.: 0.0000
## Max. :0 Max. :925.0000
##
## END_AZI END_LOCATI LENGTH WIDTH
## Length:902297 Length:902297 Min. : 0.0000 Min. : 0.000
## Class :character Class :character 1st Qu.: 0.0000 1st Qu.: 0.000
## Mode :character Mode :character Median : 0.0000 Median : 0.000
## Mean : 0.2301 Mean : 7.503
## 3rd Qu.: 0.0000 3rd Qu.: 0.000
## Max. :2315.0000 Max. :4400.000
##
## F MAG FATALITIES INJURIES
## Min. :0.0 Min. : 0.0 Min. : 0.0000 Min. : 0.0000
## 1st Qu.:0.0 1st Qu.: 0.0 1st Qu.: 0.0000 1st Qu.: 0.0000
## Median :1.0 Median : 50.0 Median : 0.0000 Median : 0.0000
## Mean :0.9 Mean : 46.9 Mean : 0.0168 Mean : 0.1557
## 3rd Qu.:1.0 3rd Qu.: 75.0 3rd Qu.: 0.0000 3rd Qu.: 0.0000
## Max. :5.0 Max. :22000.0 Max. :583.0000 Max. :1700.0000
## NA's :843563
## PROPDMG PROPDMGEXP CROPDMG CROPDMGEXP
## Min. : 0.00 Length:902297 Min. : 0.000 Length:902297
## 1st Qu.: 0.00 Class :character 1st Qu.: 0.000 Class :character
## Median : 0.00 Mode :character Median : 0.000 Mode :character
## Mean : 12.06 Mean : 1.527
## 3rd Qu.: 0.50 3rd Qu.: 0.000
## Max. :5000.00 Max. :990.000
##
## WFO STATEOFFIC ZONENAMES LATITUDE
## Length:902297 Length:902297 Length:902297 Min. : 0
## Class :character Class :character Class :character 1st Qu.:2802
## Mode :character Mode :character Mode :character Median :3540
## Mean :2875
## 3rd Qu.:4019
## Max. :9706
## NA's :47
## LONGITUDE LATITUDE_E LONGITUDE_ REMARKS
## Min. :-14451 Min. : 0 Min. :-14455 Length:902297
## 1st Qu.: 7247 1st Qu.: 0 1st Qu.: 0 Class :character
## Median : 8707 Median : 0 Median : 0 Mode :character
## Mean : 6940 Mean :1452 Mean : 3509
## 3rd Qu.: 9605 3rd Qu.:3549 3rd Qu.: 8735
## Max. : 17124 Max. :9706 Max. :106220
## NA's :40
## REFNUM
## Min. : 1
## 1st Qu.:225575
## Median :451149
## Mean :451149
## 3rd Qu.:676723
## Max. :902297
##
As you can see, a lot of columns are present. Lets filter out only the ones required for our analysis.
stormDF <- select( noaaDF, c( "EVTYPE", "FATALITIES", "INJURIES", "PROPDMG",
"PROPDMGEXP", "CROPDMG", "CROPDMGEXP" ))
summary( stormDF )
## EVTYPE FATALITIES INJURIES PROPDMG
## Length:902297 Min. : 0.0000 Min. : 0.0000 Min. : 0.00
## Class :character 1st Qu.: 0.0000 1st Qu.: 0.0000 1st Qu.: 0.00
## Mode :character Median : 0.0000 Median : 0.0000 Median : 0.00
## Mean : 0.0168 Mean : 0.1557 Mean : 12.06
## 3rd Qu.: 0.0000 3rd Qu.: 0.0000 3rd Qu.: 0.50
## Max. :583.0000 Max. :1700.0000 Max. :5000.00
## PROPDMGEXP CROPDMG CROPDMGEXP
## Length:902297 Min. : 0.000 Length:902297
## Class :character 1st Qu.: 0.000 Class :character
## Mode :character Median : 0.000 Mode :character
## Mean : 1.527
## 3rd Qu.: 0.000
## Max. :990.000
Based on the information above, the data table now has 902.297 rows and 7 columns. Below is a brief description of each variable :
The variables PROPDMGEXP and CROPDMGEXP are regarding property damage expenses, so it can be utilized to denote the events with greatest economic consequences. The values are messy, so I created a function to deal with that, and to calculate the cost with their respective exponent values (but in millions).
cost <- function( x ) {
if( x == "H" )
1E-4
else if( x == "K" )
1E-3
else if( x == "M" )
1
else if( x == "B" )
1E3
else
1-6
}
First calculate the fatalities and sort them descending ( only retain top 20 of the list )
totFatalities <- aggregate( stormDF$FATALITIES, by = list( stormDF$EVTYPE ), "sum" )
names( totFatalities ) <- c( "Event", "Fatalities" )
totFatalitiesSorted <- totFatalities[ order( -totFatalities$Fatalities ), ][ 1:20, ]
totFatalitiesSorted
## Event Fatalities
## 834 TORNADO 5633
## 130 EXCESSIVE HEAT 1903
## 153 FLASH FLOOD 978
## 275 HEAT 937
## 464 LIGHTNING 816
## 856 TSTM WIND 504
## 170 FLOOD 470
## 585 RIP CURRENT 368
## 359 HIGH WIND 248
## 19 AVALANCHE 224
## 972 WINTER STORM 206
## 586 RIP CURRENTS 204
## 278 HEAT WAVE 172
## 140 EXTREME COLD 160
## 760 THUNDERSTORM WIND 133
## 310 HEAVY SNOW 127
## 141 EXTREME COLD/WIND CHILL 125
## 676 STRONG WIND 103
## 30 BLIZZARD 101
## 350 HIGH SURF 101
and then then injuries seperately.
totInjuries <- aggregate( stormDF$INJURIES, by = list( stormDF$EVTYPE ), "sum" )
names( totInjuries ) <- c( "Event", "Injuries" )
totInjuriesSorted <- totInjuries[ order( -totInjuries$Injuries ), ][ 1:20, ]
totInjuriesSorted
## Event Injuries
## 834 TORNADO 91346
## 856 TSTM WIND 6957
## 170 FLOOD 6789
## 130 EXCESSIVE HEAT 6525
## 464 LIGHTNING 5230
## 275 HEAT 2100
## 427 ICE STORM 1975
## 153 FLASH FLOOD 1777
## 760 THUNDERSTORM WIND 1488
## 244 HAIL 1361
## 972 WINTER STORM 1321
## 411 HURRICANE/TYPHOON 1275
## 359 HIGH WIND 1137
## 310 HEAVY SNOW 1021
## 957 WILDFIRE 911
## 786 THUNDERSTORM WINDS 908
## 30 BLIZZARD 805
## 188 FOG 734
## 955 WILD/FOREST FIRE 545
## 117 DUST STORM 440
Finally plot both the fatalities and injuries in a single plot
par( mfrow = c( 1, 2 ), mar = c( 10, 4, 2, 2 ), las = 3, cex = 0.7, cex.main = 1.4, cex.lab = 1.2 )
barplot( totFatalitiesSorted$Fatalities, names.arg = totFatalitiesSorted$Event, col = 'red',
main = 'Top 20 Weather Events for Fatalities', ylab = 'Number of Fatalities' )
barplot( totInjuriesSorted$Injuries, names.arg = totInjuriesSorted$Event, col = 'orange',
main = 'Top 20 Weather Events for Injuries', ylab = 'Number of Injuries' )
Conclusion : we see that tornados cause most deaths and injuries in the U.S. National Oceanic and Atmospheric Administration’s (NOAA) storm database. But excessive heat causes second most deaths, whereas as far as injuries are concerned second to fourth causes have very similar values.
Let align on what we have done for question 1 and first calculate the cost of property
totProperty <- aggregate( stormDF$PROPDMG * sapply( stormDF$PROPDMGEXP, FUN = "cost" ),
by = list( stormDF$EVTYPE ), "sum" )
names( totProperty ) <- c( "Event", "Property" )
totPropertySorted <- totProperty[ order( -totProperty$Property ), ][ 1:20, ]
totPropertySorted
## Event Property
## 170 FLOOD 144622.710
## 411 HURRICANE/TYPHOON 69305.840
## 834 TORNADO 55374.660
## 670 STORM SURGE 43323.536
## 244 HAIL 14089.367
## 153 FLASH FLOOD 13355.312
## 402 HURRICANE 11868.319
## 848 TROPICAL STORM 7703.891
## 972 WINTER STORM 6683.497
## 590 RIVER FLOOD 5118.945
## 359 HIGH WIND 5095.046
## 957 WILDFIRE 4765.114
## 671 STORM SURGE/TIDE 4641.188
## 856 TSTM WIND 4209.928
## 427 ICE STORM 3694.928
## 409 HURRICANE OPAL 3052.846
## 955 WILD/FOREST FIRE 3001.829
## 760 THUNDERSTORM WIND 2763.121
## 298 HEAVY RAIN/SEVERE WEATHER 2500.000
## 842 TORNADOES, TSTM WIND, HAIL 1600.000
And seperately calculate crop damages
totCrop <- aggregate( stormDF$CROPDMG * sapply( stormDF$CROPDMGEXP, FUN = "cost" ),
by = list( stormDF$EVTYPE), "sum" )
names( totCrop ) <- c( "Event", "Crop" )
totCropSorted <- totCrop[ order( -totCrop$Crop), ][ 1:20, ]
totCropSorted
## Event Crop
## 95 DROUGHT 13972.5660
## 170 FLOOD 5661.9685
## 590 RIVER FLOOD 5029.4590
## 427 ICE STORM 5022.1135
## 402 HURRICANE 2741.9100
## 411 HURRICANE/TYPHOON 2607.8728
## 153 FLASH FLOOD 1421.3171
## 140 EXTREME COLD 1292.9730
## 212 FROST/FREEZE 1094.0860
## 244 HAIL 825.5375
## 290 HEAVY RAIN 733.3998
## 848 TROPICAL STORM 678.3460
## 359 HIGH WIND 638.5713
## 856 TSTM WIND 554.0073
## 130 EXCESSIVE HEAT 492.4020
## 192 FREEZE 446.2250
## 760 THUNDERSTORM WIND 414.8431
## 275 HEAT 401.4615
## 957 WILDFIRE 295.4728
## 87 DAMAGING FREEZE 262.1000
Next plot both the cost of property and crop damages in a single plot:
par( mfrow = c( 1, 2 ), mar = c( 10, 4, 2, 2 ), las = 3, cex = 0.7, cex.main = 1.4, cex.lab = 1.2 )
barplot( totPropertySorted$Property, names.arg = totPropertySorted$Event, col = 'Brown',
main = 'Top 20 Weather Events for Property Damage ', ylab = 'Amount of Property Damage' )
barplot( totCropSorted$Crop, names.arg = totCropSorted$Event, col = 'Green',
main = 'Top 20 Weather Events for Crop Damage', ylab = 'Amount of Crop Damage' )
Finally the total damage by adding both costs (property and crop damage)
totTotalCost <- aggregate(( stormDF$CROPDMG * sapply( stormDF$CROPDMGEXP, FUN = "cost" )) +
( stormDF$PROPDMG * sapply( stormDF$PROPDMGEXP, FUN = "cost" )),
by = list( stormDF$EVTYPE), "sum" )
names( totTotalCost ) <- c( "Event", "TotalCost" )
totTotalCostSorted <- totTotalCost[ order( -totTotalCost$TotalCost ), ][ 1:20, ]
totTotalCostSorted
## Event TotalCost
## 170 FLOOD 150284.678
## 411 HURRICANE/TYPHOON 71913.713
## 834 TORNADO 54989.614
## 670 STORM SURGE 43323.541
## 95 DROUGHT 15018.672
## 244 HAIL 14914.905
## 153 FLASH FLOOD 14776.629
## 402 HURRICANE 14610.229
## 590 RIVER FLOOD 10148.405
## 427 ICE STORM 8717.041
## 848 TROPICAL STORM 8382.237
## 972 WINTER STORM 6710.441
## 359 HIGH WIND 5733.618
## 957 WILDFIRE 5060.587
## 856 TSTM WIND 4763.936
## 671 STORM SURGE/TIDE 4642.038
## 760 THUNDERSTORM WIND 3177.964
## 955 WILD/FOREST FIRE 3108.626
## 409 HURRICANE OPAL 3011.846
## 298 HEAVY RAIN/SEVERE WEATHER 2500.000
And a single plot for visualization
par( mfrow = c( 1, 1 ), mar = c( 10, 4, 2, 2 ), las = 3, cex = 0.7, cex.main = 1.4, cex.lab = 1.2 )
barplot( totTotalCostSorted$TotalCost, names.arg = totTotalCostSorted$Event, col = 'Black',
main = 'Top 20 Weather Events for total Damage ', ylab = 'Amount of total Damage' )
Conclusion : we see that flood causes the most economic damage overall, as well as for property damage. But drought tops the ranking for crop damage.