M. Fawcett - 08/23/2016
It is possible to easily download weather data for anywhere in the United States, and also pretty much anywhere in the world using R.
A demonstration follows using the weatherData package written by Ram Narasimhan. Documentation for the package can be found here: https://cran.r-project.org/web/packages/weatherData/weatherData.pdf
knitr::opts_chunk$set(echo = TRUE)
## Clean up work space
rm(list = ls())
## Set working directory for R code
workingdir = "\\\\ccor_ds\\common\\dbase\\MitchFawcett\\CapacityManagement\\R\\Code"
setwd(workingdir)
## Load libraries
require(weatherData) ## for weather data retrieval## Warning: package 'weatherData' was built under R version 3.3.1require("RODBC") ## for ODBC data commandsThe weatherData package takes 4 character weather station codes (see Weather Underground for complete list). The package can also take 3 character airport codes. Using 4 character weather station codes provides greater granularity and locality than 3 character airport codes.
Googling New Castle Airport finds its airport code to be “ILG”.
## This gets the weather station code corresponding to the airport code for New Castle Airport.
getStationCode("ILG")## [1] "USA DE WILMINGTON KILG ILG 39 40N 075 36W 28 X T A 4 US"
## [2] " GILGIT OPGT 41517 35 55N 074 19E 1454 X 7 PK"## The Weather station code for airport "ILG" is "KILG".
## To get a list of all weather stations in the world, run
data(IntlWxStations)
## To get a list of USA weather stations
data(USAirportWeatherStations)This is the weather station listing for Delaware (from USAirportWeatherStations data frame). There are 3 weather stations: KDOV, KGED, KILG
351 Dover AFB DE KDOV 39.13 -75.47 9 99999
352 Georgetown DE KGED 38.69 -75.36 15 99999
353 Wilmington DE KILG 39.68 -75.60 24 99999
This gets the current temperature at New Castle Airport.
## Can check to see what data is available for a particular date at a particular location.
checkDataAvailability("ILG", "2016-08-23", station_type = "airportCode")## [1] "TimeEDT" "TemperatureF" "Dew_PointF"
## [4] "Humidity" "Sea_Level_PressureIn" "VisibilityMPH"
## [7] "Wind_Direction" "Wind_SpeedMPH" "Gust_SpeedMPH"
## [10] "PrecipitationIn" "Events" "Conditions"
## [13] "WindDirDegrees" "DateUTC"## [1] 1## Get current temperature at New Castle Airport.
getCurrentTemperature("ILG")## Time TemperatureF
## 9 2016-10-09 08:51:00 54## I hard-coded the current date here for convenience, but in practice a variable would be used.There are options to control which columns of the weather data to return.
## This gets all the columns
getDetailedWeather("ILG", "2016-08-23", opt_all_columns=T)## Time TimeEDT TemperatureF Dew_PointF Humidity
## 1 2016-08-23 00:51:00 12:51 AM 64.0 55.9 75
## 2 2016-08-23 01:51:00 1:51 AM 62.1 55.0 78
## 3 2016-08-23 02:51:00 2:51 AM 60.1 55.9 86
## 4 2016-08-23 03:51:00 3:51 AM 57.9 54.0 87
## 5 2016-08-23 04:51:00 4:51 AM 57.9 54.0 87
## 6 2016-08-23 05:51:00 5:51 AM 57.0 53.1 87
## 7 2016-08-23 06:51:00 6:51 AM 57.0 54.0 89
## 8 2016-08-23 07:51:00 7:51 AM 63.0 55.9 78
## 9 2016-08-23 08:51:00 8:51 AM 70.0 55.9 61
## 10 2016-08-23 09:51:00 9:51 AM 73.9 52.0 46
## 11 2016-08-23 10:51:00 10:51 AM 75.0 52.0 44
## 12 2016-08-23 11:51:00 11:51 AM 79.0 50.0 36
## 13 2016-08-23 12:51:00 12:51 PM 79.0 54.0 42
## 14 2016-08-23 13:51:00 1:51 PM 79.0 54.0 42
## 15 2016-08-23 14:51:00 2:51 PM 80.1 55.0 42
## 16 2016-08-23 15:51:00 3:51 PM 82.9 54.0 37
## 17 2016-08-23 16:51:00 4:51 PM 82.0 53.1 37
## 18 2016-08-23 17:51:00 5:51 PM 82.0 55.0 39
## 19 2016-08-23 18:51:00 6:51 PM 78.1 55.9 46
## 20 2016-08-23 19:51:00 7:51 PM 75.0 57.0 53
## 21 2016-08-23 20:51:00 8:51 PM 73.9 57.0 55
## 22 2016-08-23 21:51:00 9:51 PM 73.0 57.9 59
## 23 2016-08-23 22:51:00 10:51 PM 68.0 59.0 73
## 24 2016-08-23 23:51:00 11:51 PM 69.1 57.9 68
## Sea_Level_PressureIn VisibilityMPH Wind_Direction Wind_SpeedMPH
## 1 30.25 10 Calm Calm
## 2 30.25 10 NNW 4.6
## 3 30.26 10 NW 3.5
## 4 30.27 10 NW 3.5
## 5 30.29 10 NNW 3.5
## 6 30.30 10 NNW 4.6
## 7 30.32 10 NW 5.8
## 8 30.34 10 NNW 3.5
## 9 30.35 10 Variable 3.5
## 10 30.35 10 Calm Calm
## 11 30.35 10 South 4.6
## 12 30.35 10 Calm Calm
## 13 30.34 10 West 5.8
## 14 30.32 10 South 9.2
## 15 30.31 10 South 9.2
## 16 30.29 10 Variable 4.6
## 17 30.28 10 WNW 8.1
## 18 30.27 10 Variable 6.9
## 19 30.27 10 SW 5.8
## 20 30.27 10 SW 4.6
## 21 30.27 10 WSW 4.6
## 22 30.28 10 WSW 3.5
## 23 30.30 10 West 4.6
## 24 30.30 10 WSW 3.5
## Gust_SpeedMPH PrecipitationIn Events Conditions WindDirDegrees
## 1 - N/A NA Clear 0
## 2 - N/A NA Clear 340
## 3 - N/A NA Clear 310
## 4 - N/A NA Clear 320
## 5 - N/A NA Clear 330
## 6 - N/A NA Clear 330
## 7 - N/A NA Clear 310
## 8 - N/A NA Clear 330
## 9 - N/A NA Clear 0
## 10 - N/A NA Clear 0
## 11 - N/A NA Clear 180
## 12 - N/A NA Clear 0
## 13 - N/A NA Clear 260
## 14 - N/A NA Clear 170
## 15 - N/A NA Clear 190
## 16 - N/A NA Clear 0
## 17 - N/A NA Clear 300
## 18 - N/A NA Clear 0
## 19 - N/A NA Clear 230
## 20 - N/A NA Clear 230
## 21 - N/A NA Clear 240
## 22 - N/A NA Clear 240
## 23 - N/A NA Clear 260
## 24 - N/A NA Clear 250
## DateUTC
## 1 2016-08-23 04:51:00
## 2 2016-08-23 05:51:00
## 3 2016-08-23 06:51:00
## 4 2016-08-23 07:51:00
## 5 2016-08-23 08:51:00
## 6 2016-08-23 09:51:00
## 7 2016-08-23 10:51:00
## 8 2016-08-23 11:51:00
## 9 2016-08-23 12:51:00
## 10 2016-08-23 13:51:00
## 11 2016-08-23 14:51:00
## 12 2016-08-23 15:51:00
## 13 2016-08-23 16:51:00
## 14 2016-08-23 17:51:00
## 15 2016-08-23 18:51:00
## 16 2016-08-23 19:51:00
## 17 2016-08-23 20:51:00
## 18 2016-08-23 21:51:00
## 19 2016-08-23 22:51:00
## 20 2016-08-23 23:51:00
## 21 2016-08-24 00:51:00
## 22 2016-08-24 01:51:00
## 23 2016-08-24 02:51:00
## 24 2016-08-24 03:51:00Detailed weather can be obtained for a range of dates. Each day’s data is retrieved as a separate fetch but all of the returned data will be stored as a single data frame. Retrieving data takes about two seconds per day.
d3<- getWeatherForDate("KILG", start_date="2016-08-01",
end_date = "2016-08-23",
opt_detailed = TRUE,
opt_all_columns = TRUE)## [1] "TimeEDT" "TemperatureF" "Dew_PointF"
## [4] "Humidity" "Sea_Level_PressureIn" "VisibilityMPH"
## [7] "Wind_Direction" "Wind_SpeedMPH" "Gust_SpeedMPH"
## [10] "PrecipitationIn" "Events" "Conditions"
## [13] "WindDirDegrees" "DateUTC"
## [1] "TimeEDT" "TemperatureF" "Dew_PointF"
## [4] "Humidity" "Sea_Level_PressureIn" "VisibilityMPH"
## [7] "Wind_Direction" "Wind_SpeedMPH" "Gust_SpeedMPH"
## [10] "PrecipitationIn" "Events" "Conditions"
## [13] "WindDirDegrees" "DateUTC"
## [1] "Time" "TimeEDT" "TemperatureF"
## [4] "Dew_PointF" "Humidity" "Sea_Level_PressureIn"
## [7] "VisibilityMPH" "Wind_Direction" "Wind_SpeedMPH"
## [10] "Gust_SpeedMPH" "PrecipitationIn" "Events"
## [13] "Conditions" "WindDirDegrees" "DateUTC"## (not sure why the columns names are repeated three times)
## Here is the structure of the data returned by the above command.
str(d3)## 'data.frame': 651 obs. of 15 variables:
## $ Time : POSIXct, format: "2016-08-01 00:51:00" "2016-08-01 01:51:00" ...
## $ TimeEDT : chr "12:51 AM" "1:51 AM" "2:51 AM" "3:51 AM" ...
## $ TemperatureF : num 79 78.1 77 75.9 75.9 75.9 75.9 75.9 77 77 ...
## $ Dew_PointF : num 75 73.9 73 73 73 73 73 73 73.9 73.9 ...
## $ Humidity : int 88 87 88 91 91 91 91 91 90 90 ...
## $ Sea_Level_PressureIn: num 30 30 30 30 30 ...
## $ VisibilityMPH : num 10 10 10 10 10 10 10 10 10 10 ...
## $ Wind_Direction : chr "ENE" "East" "ENE" "ENE" ...
## $ Wind_SpeedMPH : chr "4.6" "4.6" "4.6" "6.9" ...
## $ Gust_SpeedMPH : chr "-" "-" "-" "-" ...
## $ PrecipitationIn : chr "N/A" "N/A" "N/A" "N/A" ...
## $ Events : chr "" "" "" "" ...
## $ Conditions : chr "Scattered Clouds" "Clear" "Clear" "Scattered Clouds" ...
## $ WindDirDegrees : int 60 80 60 60 60 70 80 70 60 80 ...
## $ DateUTC : chr "2016-08-01 04:51:00" "2016-08-01 05:51:00" "2016-08-01 06:51:00" "2016-08-01 07:51:00" ...This assumes that a User ODBC object has been created on your workstation.
The following will save the weather data to a table called dbo.tblWeather.
## Create a connection channel to the SQL Server database. Uses integrated security.
ch <- odbcConnect("Mitch")
## Create & execute a SQL query to drop the results table if it exists
s1 <- "IF OBJECT_ID('dbo.tblWeather','U') IS NOT NULL DROP TABLE dbo.tblWeather"
sqlQuery(ch, s1)## character(0)## Need to override some data types that would otherwise be used and cause an "Invalid character
## value for cast specification" error.
varTypes = c(TimeEDT="varchar(50)", Time="varchar(50)")
## Save the results for the weather data frame to a table in the SQL Server database
sqlSave(ch, d3, tablename = "dbo.tblWeather", append = TRUE, varTypes=varTypes)
close(ch)