S and diff indentifiability

Purpose

The objective of this document is to look at the interactions between S and diff when optim solves for S and diff is set to specific values.

I used optim to solve for S, alpha, and volscl for diff values from 0.001 to 15 that will best fit the CESM1-CAM5 results.

Results

# Import the results and format into a data frame
lapply(list.files(OUTPUT_DIR, '.rds', full.names = TRUE), 
       function(input){
           
           data <- readRDS(input)
           
           output        <- data$hector_output 
           output$alpha  <- data$optim_rslts$par[['alpha']]
           output$volscl <- data$optim_rslts$par[['volscl']]
           output$MSE    <- data$optim_rslts$value
           
           output
       }) %>% 
    bind_rows -> 
    fit_rslts 

What is the relationship between S and diff?

fit_rslts %>% 
    select(diff, S) %>% 
    distinct %>% 
    ggplot(aes(diff, S)) + 
    geom_point() + 
    labs(title = 'Fixed diff and optim S')

---

What is the relationship between diff and the MSE?

fit_rslts %>% 
    select(diff, MSE) %>% 
    distinct %>% 
    ggplot(aes(diff, MSE)) + 
    geom_point() + 
    labs(title = 'Sampled diff and MSE') + 
    coord_cartesian(ylim = c(0.3, 0.45))

There is a bit of a pattern in the variability but honestly it does not look that different from one another. And if we check out some summary stats of the MSE we see that over the range of diff we sampled (0.001 to 15) the MSE only changes by 0.03 which seems pretty darn small may be even negligible.

summary(fit_rslts$MSE)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.3928  0.3995  0.4092  0.4091  0.4188  0.4265

sd(fit_rslts$MSE)

## [1] 0.01078392

diff(range(fit_rslts$MSE))

## [1] 0.0336306

---

What does the temp look like for runs with a high, medium, and low diff?

cmip_comp_data <- hectorcal::cmip_individual %>% filter(model == 'CESM1-CAM5' & variable == 'tas' & !grepl('esm', experiment) & year <= 2100) 
cmip_comp_data$cmip_experiment <- cmip_comp_data$experiment
cmip_comp_data$experiment <- paste0(cmip_comp_data$cmip_experiment, '_', cmip_comp_data$ensemble)


cmip_comp_data %>% 
    filter(cmip_experiment == 'historical') %>% 
    separate(experiment, c('experiment', 'ensemble')) %>%
    select(-experiment) %>% 
    mutate(join = 1) %>% 
    left_join(tibble(experiment = c('rcp26', 'rcp45', 'rcp60', 'rcp85'), 
                     join = 1)) %>% 
    select(-join) -> 
    hist_data

cmip_comp_data %>% 
    filter(cmip_experiment != 'historical') %>% 
    separate(experiment, c('experiment', 'ensemble')) %>% 
    bind_rows(hist_data) %>%
    select(year, model, variable, value, experiment, ensemble) %>% 
    na.omit -> 
    plot_comp_data 


fit_rslts %>% 
    separate(scenario, c("experiment", "ensemble")) %>% 
    filter(variable == 'Tgav' & experiment != 'historical') %>% 
    mutate(name = if_else(diff == min(fit_rslts$diff), 'hector min diff', NA_character_)) %>% 
    mutate(name = if_else(diff == max(fit_rslts$diff), 'hector max diff', name)) %>% 
    na.omit() %>% 
    ggplot() + 
    geom_point(data = plot_comp_data, aes(year, value, color = 'CESM1-CAN5')) + 
    geom_line(aes(year, value, color = name)) + 
    facet_wrap('experiment')

Two things

1. I would have thought that rcp85 would have had the largest difference between the the 2100 temps, but it looks like rcp26 does!
2. Increasing the diff means warms that period around 1960 where we consistently see Hector underestimate the comparison temp, may be that is an artifact of the way that optim is solving not inconsistencies in the ESM responding to volcanic forcing.

---

What happens with ocean heat flux?

fit_rslts %>% 
    separate(scenario, c("experiment", "ensemble")) %>% 
    filter(variable != 'Tgav' & experiment != 'historical') %>% 
    mutate(name = if_else(diff == min(fit_rslts$diff), 'hector min diff', NA_character_)) %>% 
    mutate(name = if_else(diff == max(fit_rslts$diff), 'hector max diff', name)) %>% 
    na.omit() %>% 
    ggplot() + 
    geom_line(aes(year, value, color = name)) + 
    facet_wrap('experiment')

We are really seeing differences in the ocean heat flux, when the diff is small it is also nothing.