Purpose

So after a saga of resolving issues that I will not bore you with we finally have some results for the concentration runs! (The emission driven runs are having some issues with convergence). Anyways before thinking about what the calibration results tell us about the models I wanted to check to check the following…

  1. Did the optim calibration work?
  2. Did adding the volsc do what we thought it would do?

Convergence

Well the first thing I wanted to do was see if the optim runs even worked by checking the convergence.

How many models did we calibrate to?

## [1] 35

How many were actually successful?

## [1] 21

Which models were unsuccessful? What did their final characterization look like before optim gave up?

model S alpha diff volscl
ACCESS1-3 38.81 1.99 16.20 1.81
CESM1-BGC 2.21 1.47 0.00 0.51
CESM1-FASTCHEM 3.13 0.68 539.59 5.00
CESM1-WACCM 1.91 1.06 0.00 0.27
CMCC-CESM 1.90 2.48 0.00 0.04
EC-EARTH 2.11 1.81 0.00 0.54
FGOALS-g2 1.68 2.12 0.00 0.04
GISS-E2-H-CC 23.02 -0.81 190.59 4.66
GISS-E2-R-CC 32.05 0.40 218.55 3.59
IPSL-CM5B-LR 4234.96 -0.44 252.13 4.50
MPI-ESM-MR 2.18 1.84 0.02 0.46
MPI-ESM-P 3.02 0.29 394.35 4.43
MRI-CGCM3 755.60 1.64 49.26 1.99
MRI-ESM1 0.24 3.23 0.19 1.32

Some of these values are what I call ridiculous, like sure Hector could have a Hector climate sensitivity of 4,234 but that seems unlikely to me. Some of these models that failed have what I presume to be a related model converge. For example ACESS1-3 failed to converge but ACCESS1-0 converged. I think that I could try using the parameterization of ACESS1-0 to see if that would be a better initial parameter guess for optim.

For the models that do not have parameter values from a related converged optim run, like EC-Earth, FGoals-g2, and the MRI-models I wonder if the calibration before we added the volscl parameter would be a useful starting point.


Did the parameterization before the volscl work?

How many times did the calibration fail to converge?

## [1] 0

:rock_on:

So to me this means that we can use the before volscl calibration results as starting initial param values.


What difference does volscl make?

So the reason why we added the the volcanic scalar was because we were saw Hector was underestimating global mean temperature because it had no mechanism to emulate the ESM response to volcanoes. Did adding the volscl to the calibration help with the under estimation of CMIP Tgav during the historical period?

Results

These plots compare the hector (with the volsc calibrated), the CMIP5 comparison data being calibrated to and hector novolsc (an earlier version of hector where volsc was not a hector parameter yet.) I only plotted the comparison data for the historic period where we are seeing all the problems with the volcanoes.

Panneled Comparison Plot

It looks like the two hector runs (the less with the less squiggles) are different from one another and typically there is one of these lines closer to the CMIP models than the others. It will probably be more useful to look a the individual comparison plots on the next tab.

Individual Comparison Plots

For the most part the hector calibrated with the volscl looks better than the hector calibrated without. However that does not seem to be the case for the CMCC model family or NorESM1-ME. Which is interesting.

Parameter Comparisons

How does adding the volsc change the values for the other parameters? (Only for the calibration that converged).

param calibration min mean max sd
alpha novol -0.15 1.57 2.45 0.58
vol -0.21 1.51 2.29 0.58
diff novol 0.00 2.95 20.41 4.50
vol 0.00 16.99 265.98 57.31
S novol 0.67 3.46 18.93 3.18
vol 0.22 5.50 44.19 9.04
volscl vol -0.36 1.09 3.60 0.84

Hmmmm the there a really large diff value for the experiments that use the volscl I wonder if that is an outlier or is expected?

Looks that that diff value is an outlier, what model has that large diff value and what does the rest of the distribution.

##    model calibration param    value
## 1 inmcm4         vol  diff 265.9785

When we subset the diff parameter space so that is excludes the outlier, it looks like the hector calibration with the volscl also has larger values of diff where as hector calibrated without the volsc is skewed closer to 0.


What happens to S when we remove the outliers?

The IQR is warmer for the S values from the vol calibration experiment.

Conclusions

  1. Not all of the models converged :( which is not too surprising but means there is some more work to be done for the concentration driven calibration runs.
  2. I think that volscl did what it was intended to do which is great.

Next Steps

Questions