Week 3B Assignment: Code Base

Window Functions

Author

Jocelyn Slater

Introduction

After initially intending to use currency exchange rate data for my data set of interest for this assignment, I switched focus to a weather data set from NOAA. The weather data tracks the daily maximum and minimum temperature taken from the weather station in Central Park, NYC. To prepare the data for analysis, I reformated the date strings and also cleaned up extraneous columns. Then, using window functions, I calculated the year to date average as well as the six day moving average for the daily maximum and minimum temperatures and plotted the results.

Data Import

Data is imported from the NOAA website Climate Data Online (CDO). I filtered the data to just include a 12 month record of average daily maximum and minimum temperatures in Central Park in 2010. I did this through the website before exporting. There is minimal tidying to do. I ran the janitor package, cleaned up the location columns, and formatted the date strings in the data frame.

# Import and tidy
url <- "https://raw.githubusercontent.com/jocslater-code/DATA607/refs/heads/main/Week3B/NOAA_temp_data.csv"
raw_noaa_df <- read_csv(url, show_col_types = FALSE)
raw_noaa_df<- clean_names(raw_noaa_df)
glimpse(raw_noaa_df)

Rows: 365
Columns: 5
$ station         <chr> "USW00094728", "USW00094728", "USW00094728", "USW00094…
$ name            <chr> "NY CITY CENTRAL PARK, NY US", "NY CITY CENTRAL PARK, …
$ date            <chr> "1-Jan", "2-Jan", "3-Jan", "4-Jan", "5-Jan", "6-Jan", …
$ dly_tmax_normal <dbl> 38.9, 38.8, 38.7, 38.5, 38.4, 38.3, 38.2, 38.2, 38.1, …
$ dly_tmin_normal <dbl> 27.9, 27.7, 27.6, 27.4, 27.3, 27.2, 27.1, 27.0, 26.9, …

noaa_df<- select(raw_noaa_df, date, dly_tmax_normal, dly_tmin_normal)
noaa_df <- noaa_df %>% rename(temp_max = dly_tmax_normal)
noaa_df <- noaa_df %>% rename(temp_min = dly_tmin_normal)

# Reformat date column for easy handling and make sure things are in proper order
noaa_df <- noaa_df %>%
  mutate(date = dmy(paste0(date, "-2010"))) %>%
  arrange(date)
glimpse(noaa_df)

Rows: 365
Columns: 3
$ date     <date> 2010-01-01, 2010-01-02, 2010-01-03, 2010-01-04, 2010-01-05, …
$ temp_max <dbl> 38.9, 38.8, 38.7, 38.5, 38.4, 38.3, 38.2, 38.2, 38.1, 38.0, 3…
$ temp_min <dbl> 27.9, 27.7, 27.6, 27.4, 27.3, 27.2, 27.1, 27.0, 26.9, 26.8, 2…

Window Functions

Using the dplyr library, I calculated the cumulative mean (year to date average) as well as a rolling mean (moving average) and added them as extra columns on my data frame.

noaa_df <- noaa_df %>%
  # Calculate cumulative averages
  mutate(
    ytd_avg_min = cummean(temp_min),
    ytd_avg_max = cummean(temp_max)
  ) %>%
  # Calculate 6-day moving averages
  mutate(
    mavg_6day_min = rollmean(temp_min, k = 6, fill = NA, align = "right"),
    mavg_6day_max = rollmean(temp_max, k = 6, fill = NA, align = "right")
  )

Results

To make the plot of the rolling average of the min and max temperatures, I also plotted the original raw data for comparison. Important to note is the short gap in data at the beginning of the year for the rolling average. This is because my window is six days, so the average can only begin to be calculated after six data points.

ggplot(noaa_df, aes(x = date)) +
  # Unprocessed Data
  geom_line(aes(y = temp_max, color = "Max Temp (Unprocessed)"), linetype = "dotted", linewidth = 0.8) +
  geom_line(aes(y = temp_min, color = "Min Temp (Unprocessed)"), linetype = "dotted", linewidth = 0.8) +
  
  # 6 Day Moving Average
  geom_line(aes(y = mavg_6day_max, color = "Max Temp (6-day MAvg)"), linetype = "solid", linewidth = 1.1) +
  geom_line(aes(y = mavg_6day_min, color = "Min Temp (6-day MAvg)"), linetype = "solid", linewidth = 1.1) +
  
  # Colors
  scale_color_manual(
    values = c(
      "Max Temp (Unprocessed)"      = "firebrick1",
      "Max Temp (6-day MAvg)" = "firebrick4",
      "Min Temp (Unprocessed)"      = "lightblue3",
      "Min Temp (6-day MAvg)" = "midnightblue"
    ),
    breaks = c("Max Temp (Unproce.)", "Max Temp (6-day MAvg)", "Min Temp (Unproc.)", "Min Temp (6-day MAvg)")
  ) +
  
  # Labeling
  labs(
    title = "Daily Temperatures, NYC 2010",
    subtitle = "Dotted = Unprocessed | Solid = 6-Day Moving Average",
    x = "Date",
    y = "Temperature (F)",
    color = "Series"
  ) +
  scale_x_date(date_labels = "%b", date_breaks = "1 month") +
  theme_minimal() +
  theme(
    legend.position = "bottom"
  )

Warning: Removed 5 rows containing missing values or values outside the scale range
(`geom_line()`).
Removed 5 rows containing missing values or values outside the scale range
(`geom_line()`).

I have also used ggplot to visualize the year to date average temperatures.

ggplot(noaa_df, aes(x = date)) +
  geom_line(aes(y = ytd_avg_max, color = "Max Temp (YTD Avg)"), linetype = "solid", linewidth = 0.9) +
  geom_line(aes(y = ytd_avg_min, color = "Min Temp (YTD Avg)"), linetype = "solid", linewidth = 0.9) +
  
  # Colors
  scale_color_manual(
    values = c(
      "Max Temp (YTD Avg)"  = "coral2",
      "Min Temp (YTD Avg)"  = "steelblue2"
    ),
    breaks = c(
      "Max Temp (YTD Avg)", "Min Temp (YTD Avg)")
  ) +
  
  # Labels
  labs(
    title = "YTD Average Temperatures, NYC 2010",
    x = "Date",
    y = "Temperature (F)",
    color = "Series"
  ) +
  scale_x_date(date_labels = "%b", date_breaks = "1 month") +
  theme_minimal() +
  theme(
    legend.position = "bottom",
    legend.box = "horizontal",
    panel.grid.minor = element_blank()
  ) +
  guides(color = guide_legend(ncol = 2))

Summary and Next Steps

Although my approach outlined the use of IMF currency exchange data, I quickly found that the source I wanted to use was not regular enough for my application. Months were skipped for many of the countries and the time period available was very short. I briefly tried to find a better source, but when that proved to be time consuming, I pivoted to more easily accessible weather data from NOAA. The NOAA online data portal required me to place a free “order” through their online portal which was very easy. I also did much of my data filtering through the portal, so the data set I was delivered in my email inbox had very limited extraneous data.

I opted to use the functions built into the dplyr library to calculate the year to date averages and six day moving averages. My previous experience with a moving average function is to use it as a smoothing function for very noisy data. Because the temperature data was quite smooth and well behaved already, there is very difference between the unprocessed and the smoothed data sets, as seen in the plot. Both the daily high and low temperatures share behavior, with yearly lows in the winter and highs in the summer, just as expected. We also see expected behavior for the year to date averages of min and max temps. Yearly average starts low in the winter, rises through the spring and summer, and then starts to drop again into the fall and winter.

Next steps could be to include more aspects of the weather such as precipitation or wind speed, and see how these values relate to the daily minimum and maximum temperatures.