Class 607, Assignment 1

Author

Troy Tournat

Published

September 4, 2026

Introduction

This dataset is from WastewaterSCAN, an organization part of the WastewaterSCAN / SCAN project, a partnership between Stanford University, Emory University, and Verily funded philanthropically through a gift to Stanford University.

They publicly monitor infectious diseases across the USA through local wastewater systems. As an epidemiologist, this information is important to our work monitoring the spread of diseases. Since the new respiratory season has officially started August 1st and there has been an increase in COVID in California according to the CDC.

Tackling the problem: Plan is to load data into Rstudio, creating a subset, data cleaning

Anticipated data challenges: Matching the correct data file format to read in correctly into R as well as cleaning date variables

Citations:

A. B. Boehm, M. K. Wolfe, A. L. Bidwell, A. Zulli, B. J. White, B. Shelden, D. Duong. Pathogen nucleic acids data in wastewater solids from 147 treatment plants in the United States: 2024-2025. Data in Brief, 2026. Link to paper.
About CDC’s wastewater monitoring program (2026) Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/wastewater/about/index.html#cdc_survey_profile_how_surveys_are_conducted-data-sources (Accessed: 03 September 2026).
Anthropic. (2025). Claude Opus 4.5 [Large language model]. (https://claude.ai/) Accessed January 8, 2026. Link to chat.

Analysis

#Not shown: Location on computer where I am grabbing the data and creating a saved location

#Install packages needed 
pacman::p_load(
  # Package Install and Management
  pacman,           # package install/load
  janitor,          #clean up data names 
  
  # Project and File Management
  readr,            # import data
  httr,             #github passkey 
  # General Data Management
  dplyr,            # data management
  tidyr,            # data management
  lubridate,        # work with dates
  zoo,              # work with dates
  tidyverse        # work with dates
)


#Load data
#github 
csv_location <- GET(
   "https://raw.githubusercontent.com/troy-tournat/607-assignments/refs/heads/main/WWSCAN_categories_20260902.csv?token=GHSAT0AAAAAAEHYW24NIASCUDUT2WJ6YTYK2U3LXCQ",
  add_headers(Authorization = paste("token", passkey_classic))) 

ww_data <- read_csv(
  content(csv_location, "text"),
  show_col_types = FALSE,
  progress = FALSE )

#desktop
#create date that changes when you download it 
#current_date <- format(Sys.Date()-1, "%Y%m%d")

#ww_data <- read.csv(paste0(csv_location, "WWSCAN_categories_", 
#  current_date, ".csv"), header=TRUE, stringsAsFactors = FALSE)

##Review and clean data 

#looking at dataframe 
ls(ww_data)

 [1] "BA.2 LPPA24S category"           "BA.2 LPPA24S method"            
 [3] "BA.4 ORF1a Del 141-143 category" "BA.4 ORF1a Del 141-143 method"  
 [5] "C_auris category"                "C_auris method"                 
 [7] "Delta 156-157 category"          "Delta 156-157 method"           
 [9] "EVD68 category"                  "EVD68 level"                    
[11] "EVD68 method"                    "EVD68 trend"                    
[13] "HAdV_F category"                 "HAdV_F method"                  
[15] "HAV category"                    "HAV method"                     
[17] "HMPV_4 category"                 "HMPV_4 level"                   
[19] "HMPV_4 method"                   "HMPV_4 trend"                   
[21] "HPIV category"                   "HPIV method"                    
[23] "HV 69-70 Del category"           "HV 69-70 Del method"            
[25] "InfA_H1 category"                "InfA_H1 method"                 
[27] "InfA_H3_V2 category"             "InfA_H3_V2 method"              
[29] "InfA_H5 category"                "InfA_H5 method"                 
[31] "Influenza A category"            "Influenza A level"              
[33] "Influenza A method"              "Influenza A trend"              
[35] "Influenza B category"            "Influenza B method"             
[37] "last sample date"                "MeV_Roy category"               
[39] "MeV_Roy method"                  "MPXV_dD14-16 category"          
[41] "MPXV_dD14-16 method"             "MPXV_G2R_G category"            
[43] "MPXV_G2R_G method"               "MPXV_G2R_WA category"           
[45] "MPXV_G2R_WA method"              "N Gene category"                
[47] "N Gene level"                    "N Gene method"                  
[49] "N Gene trend"                    "NDM category"                   
[51] "NDM method"                      "Noro_G2 category"               
[53] "Noro_G2 level"                   "Noro_G2 method"                 
[55] "Noro_G2 trend"                   "Omicron Del 143-145 category"   
[57] "Omicron Del 143-145 method"      "Parvo_B19 category"             
[59] "Parvo_B19 method"                "plant"                          
[61] "Rota category"                   "Rota level"                     
[63] "Rota method"                     "Rota trend"                     
[65] "RSV category"                    "RSV level"                      
[67] "RSV method"                      "RSV trend"                      
[69] "S Gene category"                 "S Gene method"                  
[71] "site name"                       "state"                          
[73] "TB_RD9 category"                 "TB_RD9 method"                  
[75] "us census region"                "WNV category"                   
[77] "WNV method"                      "XBB_bkpt category"              
[79] "XBB_bkpt method"

#cleaning names slightly 
ww_data_clean <- ww_data %>% clean_names
ls(ww_data_clean)

 [1] "ba_2_lppa24s_category"           "ba_2_lppa24s_method"            
 [3] "ba_4_orf1a_del_141_143_category" "ba_4_orf1a_del_141_143_method"  
 [5] "c_auris_category"                "c_auris_method"                 
 [7] "delta_156_157_category"          "delta_156_157_method"           
 [9] "evd68_category"                  "evd68_level"                    
[11] "evd68_method"                    "evd68_trend"                    
[13] "h_ad_v_f_category"               "h_ad_v_f_method"                
[15] "hav_category"                    "hav_method"                     
[17] "hmpv_4_category"                 "hmpv_4_level"                   
[19] "hmpv_4_method"                   "hmpv_4_trend"                   
[21] "hpiv_category"                   "hpiv_method"                    
[23] "hv_69_70_del_category"           "hv_69_70_del_method"            
[25] "inf_a_h1_category"               "inf_a_h1_method"                
[27] "inf_a_h3_v2_category"            "inf_a_h3_v2_method"             
[29] "inf_a_h5_category"               "inf_a_h5_method"                
[31] "influenza_a_category"            "influenza_a_level"              
[33] "influenza_a_method"              "influenza_a_trend"              
[35] "influenza_b_category"            "influenza_b_method"             
[37] "last_sample_date"                "me_v_roy_category"              
[39] "me_v_roy_method"                 "mpxv_d_d14_16_category"         
[41] "mpxv_d_d14_16_method"            "mpxv_g2r_g_category"            
[43] "mpxv_g2r_g_method"               "mpxv_g2r_wa_category"           
[45] "mpxv_g2r_wa_method"              "n_gene_category"                
[47] "n_gene_level"                    "n_gene_method"                  
[49] "n_gene_trend"                    "ndm_category"                   
[51] "ndm_method"                      "noro_g2_category"               
[53] "noro_g2_level"                   "noro_g2_method"                 
[55] "noro_g2_trend"                   "omicron_del_143_145_category"   
[57] "omicron_del_143_145_method"      "parvo_b19_category"             
[59] "parvo_b19_method"                "plant"                          
[61] "rota_category"                   "rota_level"                     
[63] "rota_method"                     "rota_trend"                     
[65] "rsv_category"                    "rsv_level"                      
[67] "rsv_method"                      "rsv_trend"                      
[69] "s_gene_category"                 "s_gene_method"                  
[71] "site_name"                       "state"                          
[73] "tb_rd9_category"                 "tb_rd9_method"                  
[75] "us_census_region"                "wnv_category"                   
[77] "wnv_method"                      "xbb_bkpt_category"              
[79] "xbb_bkpt_method"

#subset to West in California for respiratory disease 
#(Influenza, RSV, SARS-CoV-2, Human Metapneumovirus,
#EVD68, Parainfluenza,Parvovirus) in the last year. 
#Also renaming variables for dif diseases 
ww_data_subset <- ww_data_clean %>%
  select(state, plant, site_name, last_sample_date, 
         inf_a_h1_category, inf_a_h3_v2_category, 
         inf_a_h5_category, influenza_a_category, 
         influenza_b_category, rsv_category, 
         n_gene_category, hmpv_4_category, 
         evd68_category, hpiv_category, 
         parvo_b19_category)%>% 
  filter(state == "California", 
         last_sample_date > 2025)%>% 
  rename(
      flu_a_h1 = inf_a_h1_category, 
      flu_a_h3 = inf_a_h3_v2_category, 
      flu_a_h5 = inf_a_h5_category, 
      flu_a = influenza_a_category, 
      flu_b = influenza_b_category, 
      rsv = rsv_category, 
      sars_cov_2 = n_gene_category, 
      human_metapneumovirus = hmpv_4_category, 
      evd68 = evd68_category, 
      parainfluenz = hpiv_category, 
      parovirus = parvo_b19_category
  )

head(ww_data_subset)

# A tibble: 6 × 15
  state  plant site_name last_sample_date flu_a_h1 flu_a_h3 flu_a_h5 flu_a flu_b
  <chr>  <chr> <chr>     <date>           <chr>    <chr>    <chr>    <chr> <chr>
1 Calif… CODI… CODIGA    2024-05-31       not cal… not cal… not cal… not … not …
2 Calif… Coas… Coastal … 2024-06-28       not cal… not cal… not cal… not … not …
3 Calif… Cont… Central … 2024-06-25       not cal… not cal… not cal… not … not …
4 Calif… Davi… City of … 2026-08-31       low      low      not det… low   low  
5 Calif… Espa… Esparto … 2024-06-28       not cal… not cal… not cal… not … not …
6 Calif… Fair… Fairfiel… 2024-06-28       not cal… not cal… not cal… not … not …
# ℹ 6 more variables: rsv <chr>, sars_cov_2 <chr>, human_metapneumovirus <chr>,
#   evd68 <chr>, parainfluenz <chr>, parovirus <chr>

Conclusion

The challenges I faced were getting used to using .qmd as well as github. My biggest challenge was ingesting the data from github using a private repo.

Now that the data is ingested and cleaned, it would be interesting to compare this data source with the CDC’s wastewater data. Also it would be interesting to visualize the information through data table summaries and graphs over time.