(1) Loading data and filtering for oak species and forest types

Even though this FVS run was specified to FIA stands that occurred in oak woodlands, we filter for those that contain black oaks. We also filter for post-treatment years to identify where prescriptions were carried out in FVS (Step 2)

We might also want to compare changes in mortality later that are due to treatment. The value is calculated as the mortality between time steps. In general, we use a 5 year timestep in our simulation, but there are a couple individual years that are recorded for specifying the timing of disturbance/response. We select those and multiply the ‘MrtTPA’ values by 5 to be consistent with estimates at other years.

library(tidyverse)
db <- '/Users/eyackulic/Downloads/FVSOut_disturbance.db'
dbTables(db)
##  [1] "CmpCompute"        "CmpMetaData"       "CmpStdStk"        
##  [4] "CmpSummary2"       "FVS_ATRTList"      "FVS_BurnReport"   
##  [7] "FVS_CanProfile"    "FVS_Carbon"        "FVS_Cases"        
## [10] "FVS_Compute"       "FVS_Consumption"   "FVS_CutList"      
## [13] "FVS_Down_Wood_Cov" "FVS_Down_Wood_Vol" "FVS_Fuels"        
## [16] "FVS_Hrv_Carbon"    "FVS_Mortality"     "FVS_PotFire"      
## [19] "FVS_PotFire_Cond"  "FVS_Regen_Sprouts" "FVS_SnagSum"      
## [22] "FVS_StrClass"      "FVS_Summary2"      "FVS_TreeList"     
## [25] "StdStk"
cases <- dbGetData(db, 'FVS_Cases') |>
  dplyr::filter(substr(MgmtID,0,3) %in% 'FIC' |
                  substr(MgmtID,0,3) %in% 'BAS' )#(1) 

summary <- dbGetData(db, 'FVS_Summary2') |>
  dplyr::filter(CaseID %in% cases$CaseID) |>
  dplyr::left_join(cases, by = c('CaseID','StandID'))

stdstk <- dbGetData(db, 'StdStk') |>
  dplyr::filter(CaseID %in% cases$CaseID) |>
  dplyr::left_join(cases, by = c('CaseID')) |>
  dplyr::mutate(
    group = NA,
    group = dplyr::if_else(
      substr(Species,0,4) %in% 'QUKE','Oaks', group),
    group = dplyr::if_else(substr(Species,0,2) == 'PI','Pines', group),
    MrtTPA = dplyr::if_else(Year %in% c(2030,2031,2036),MrtTPA * 5, MrtTPA)
  )

(2) identify where actions occurred

FVS delivers stand reports for every stand in a simulation, regardless of whether or not it qualified for treatment based on FVS’ internal parameters. When trees are removed for commercial uses, FVS clearly reports this in the ‘RmvCode’ field in the summary document. However, it is more difficult disentangling which stands received treatment when the prescription is burning (i.e., prescribed burn / grapple pile) or designed to have a minimal, non-harvest effect (i.e., mowing, herbivory, herbicide, revegetation). In the case of herbivory, trees are neither being added or removed, but simply pruned. While the record of treatment is not intuitive for these treatments, there is an artifact left behind by FVS: by comparing each record with its counterfactual, BAU expectation, we can see fractional differences in TPA in the years following treatment, even in the case of minimally invasive treatments. The following function iteratively compares management and BAU scenarios to determine which stands received treatment for each different prescription.

treatedStands <- function(summary){
  for(i in 1:length(unique(summary$MgmtID))){
    mgmt_id <-unique(summary$MgmtID)[i]
    if(i == 1){
      out <-  
        summary |>
        dplyr::filter(MgmtID %in% c('BASE',mgmt_id) & Year %in% 2036) |>
        dplyr::group_by(StandID) |>
        dplyr::select(StandID, MgmtID, CaseID) |>
        dplyr::distinct()  
      tot_out <- out
    }  
    out <-  
      summary |>
      dplyr::filter(MgmtID %in% c('BASE',mgmt_id) & Year %in% 2036) |>
      dplyr::group_by(StandID) |>
      dplyr::arrange(StandID, MgmtID) |>
      dplyr::mutate(r = Tpa - dplyr::first(Tpa)) |>
      dplyr::filter(r > 0.005 | r < -0.005) |>
      dplyr::select(StandID, MgmtID, CaseID) |>
      dplyr::distinct()
    
    tot_out <- dplyr::bind_rows(out, tot_out)
  }
  tot_out
}

treated_stands <- treatedStands(summary)
table(treated_stands$MgmtID)
## 
## BASE FIC1 FIC2 FIC3 FIC4 FIC5 FIC6 
##  550  538  538  538  538  538  538

The following plots display stdstk changes across pines, oaks, and other trees. In particular, note the lack of harvested oaks in the fourth plot and the general decreases in non-oaks associated with treatment.

#(3) compare (sdi/tpa/ba/mort) responses across functional groups (oaks/pines/etc) and treatments

stdstk |>
  ggplot(aes(x = Year, y = LiveBA, color = group)) + facet_wrap(~MgmtID, scales = 'free_y') +
  geom_smooth() + xlim(2025,2060) + tidyquant::theme_tq()

stdstk |>
  ggplot(aes(x = Year, y = LiveTpa, color = group)) + facet_wrap(~MgmtID, scales = 'free_y') +
  geom_smooth() + xlim(2025,2060) + tidyquant::theme_tq()

stdstk |>
  ggplot(aes(x = Year, y = MrtTPA, color = group)) + facet_wrap(~MgmtID, scales = 'free_y') +
  geom_smooth() + xlim(2025,2060)+ tidyquant::theme_tq()

#no oaks being harvested or cut
stdstk |>
  ggplot(aes(x = MgmtID, y = HrvTPA, color = group)) + facet_wrap(~MgmtID, scales = 'free') +
  geom_boxplot()+ tidyquant::theme_tq()

RF Generation

In the next step, RFs are calculated for oak trees for all years in comparison to the pre-treatment observation (2029). SDI is not inherently part of the std-stk output, so it is calculated here along with qmd (SDI is correct - I have compared with stand level estimates in the Summary table). Values are summed to the stand level and RFs are calculated as (post-pre)/pre, with all values constrained to a range of -1:1. RFs are calculated for SDI, BA, Tpa, and Mortality for comparison.

#(4) present rfs and n

oak_responses <-
  stdstk |>
  dplyr::filter(Year %in% c(2034:2060) & #years for RFs
                  !Species %in% 'All' &
                  DBHClass %in% 'All' &
                  group %in% 'Oaks' & #oak specific
                  CaseID %in% treated_stands$CaseID) |> #treatments that happened
  dplyr::group_by(MgmtID, StandID, Species, Year) |>
  dplyr::arrange(StandID, MgmtID,Year) |>
  dplyr::mutate(
    ba_per_tree = LiveBA/LiveTpa,
    qmd = sqrt((ba_per_tree/0.005454154)), #calculate qmd
    sdi = LiveTpa * (qmd/10)^1.605 #and sdi
  ) |>
  dplyr::ungroup() |>#drop species grouping
  dplyr::group_by(MgmtID, StandID, Year) |>
  dplyr::reframe(
    year = Year,
    stand_sdi = sum(sdi),#sum to stand level
    stand_tpa = sum(LiveTpa),
    runTitle = RunTitle,
    stand_ba = sum(LiveBA),
    stand_mort = sum(MrtTPA)
  ) |>
  dplyr::distinct() |>
  dplyr::group_by(MgmtID, StandID) |>
  dplyr::mutate(
    sdi_response = ( stand_sdi - dplyr::first(stand_sdi)) / (dplyr::first(stand_sdi) + stand_sdi),
    tpa_response = (stand_tpa - dplyr::first(stand_tpa)) / (dplyr::first(stand_tpa) + stand_tpa),
    ba_response = (stand_ba - dplyr::first(stand_ba)) / (dplyr::first(stand_ba) + stand_ba),
    mort_response = (stand_mort - dplyr::first(stand_mort)) / (dplyr::first(stand_mort) + stand_mort)
  ) |>
  dplyr::as_tibble() |>
  dplyr::select(c(
    year, MgmtID, StandID, sdi_response,tpa_response,ba_response,mort_response))

RF Reporting

The RFs recorded in the last step are what we are currently reporting for FVS-derived RFs. But there is a systematic flaw with these values: because FVS does not record treatment at every stand, each RF is based on a unique set of stands. Those stands each have a distinct character that are inherent to the selected treatment; a clearcut is not going to be prescribed in a bareground or seedling dominated stand, and mowing isnt going to be selected in a large diameter, old growth stand. This is important, because those stand conditions are affecting the background rates of growth and mortality. By not accounting for stand conditions, we are predominately reporting those differences in our RFs, rather than what the treatment is directly influencing.

One way to circumvent this problem is to compare each RF to what the RF would have been for the same stands in a BAU scenario. The following function does just that : for each mgmtID, it filters and compares the records that received treatment with the BAU scenario for those same records. The result is fully normalized RFs that reflect treatment benefits.

#(5) contrast with base scenarios
normalize_rfs <- function(oak_responses, response_var){

  oak_responses$variable <- 
    oak_responses |> 
  dplyr::select(
      dplyr::all_of(response_var)
    ) |> 
    unlist() |> 
    as.numeric()

  for(i in 1:length(unique(oak_responses$MgmtID))){
  mgmt_id <- unique(oak_responses$MgmtID)[i]
  ids <-
    oak_responses |> 
    dplyr::filter(MgmtID %in% mgmt_id) |>
    dplyr::select(StandID) |> 
    dplyr::distinct()
  
  out <-
  oak_responses |>
    dplyr::filter(
      MgmtID %in% c('BASE',mgmt_id) &
      StandID %in% ids$StandID) |>
    dplyr::group_by(StandID, year) |>
    dplyr::arrange(StandID, MgmtID, year) |>
    dplyr::mutate(
      base_rf = variable - dplyr::first(variable),
      base_rf = dplyr::if_else(base_rf >= 1, 1, base_rf),
      base_rf = dplyr::if_else(base_rf <= -1, -1, base_rf)
                  ) |>
    dplyr::filter(MgmtID %in% mgmt_id) 
if( i == 1){
  tot_out <- out
}else{
  tot_out <- dplyr::bind_rows(out,tot_out)
}
  }
  tot_out
}

The following function prints results, so that we can compare the effects of normalization for every response function, in plot and tabular forms.

#(6) refactored rfs

printResults <- function(oak_responses, years, response_var, normalized = TRUE, plot = TRUE){

  if(normalized == TRUE){
  responses <- 
    oak_responses |>
    normalize_rfs(response_var = response_var) |> 
    dplyr::filter(year %in% years) |>
    dplyr::group_by(MgmtID, year) |>
    dplyr::select(base_rf)|> 
    rstatix::get_summary_stats()
  }else{
  responses <-
    oak_responses  |>
    dplyr::filter(year %in% years) |>
    dplyr::group_by(MgmtID, year) |>
    dplyr::select(response_var)|> 
    rstatix::get_summary_stats()
  }
  if(plot %in% FALSE){
    responses
  }else{
    
plot <- 
ggplot(
  data = responses |> 
    dplyr::filter(MgmtID != 'BASE'),
  aes(x= year, 
      y = median, 
      ymin = median -ci, 
      ymax = median + ci, 
      color = MgmtID, 
      fill = MgmtID)) + 
  geom_path() +
  geom_ribbon(
    alpha = .05, 
    color = NA) +
  ylab(glue::glue(' Normalized = ', normalized)) +
  tidyquant::theme_tq() + 
  scale_fill_viridis_d() +
  scale_colour_viridis_d()
  
 plot |> plotly::ggplotly()
  }
}
sdi_plot <- printResults(oak_responses, years = c(2029:2060), response_var = 'sdi_response', normalized = T,plot = T)
sdi_plot_non <- printResults(oak_responses, years = c(2029:2060), response_var = 'sdi_response', normalized = F,plot = T)
sdi_table <- printResults(oak_responses, years = c(2029:2060), response_var = 'sdi_response', normalized = T,plot = F) |> dplyr::mutate(variable = 'sdi')
sdi_table_non <- printResults(oak_responses, years = c(2029:2060), response_var = 'sdi_response', normalized = F,plot = F) |> dplyr::mutate(variable = 'sdi')


tpa_plot <- printResults(oak_responses, years = c(2029:2060), response_var = 'tpa_response', normalized = T,plot = T)
tpa_plot_non <- printResults(oak_responses, years = c(2029:2060), response_var = 'tpa_response', normalized = F,plot = T)
tpa_table <- printResults(oak_responses, years = c(2029:2060), response_var = 'tpa_response', normalized = T,plot = F) |> dplyr::mutate(variable = 'tpa')
tpa_table_non <- printResults(oak_responses, years = c(2029:2060), response_var = 'tpa_response', normalized = F,plot = F) |> dplyr::mutate(variable = 'tpa')


ba_plot <- printResults(oak_responses, years = c(2029:2060), response_var = 'ba_response', normalized = T,plot = T)
ba_plot_non <- printResults(oak_responses, years = c(2029:2060), response_var = 'ba_response', normalized = F,plot = T)
ba_table <- printResults(oak_responses, years = c(2029:2060), response_var = 'ba_response', normalized = T,plot = F) |> dplyr::mutate(variable = 'ba')
ba_table_non <- printResults(oak_responses, years = c(2029:2060), response_var = 'ba_response', normalized = F,plot = F) |> dplyr::mutate(variable = 'ba')


mort_plot <- printResults(oak_responses, years = c(2029:2060), response_var = 'mort_response', normalized = T,plot = T)
mort_plot_non <- printResults(oak_responses, years = c(2029:2060), response_var = 'mort_response', normalized = F,plot = T)
mort_table <- printResults(oak_responses, years = c(2029:2060), response_var = 'mort_response', normalized = T,plot = F)  |> dplyr::mutate(variable = 'mort')
mort_table_non <- printResults(oak_responses, years = c(2029:2060), response_var = 'mort_response', normalized = F,plot = F)  |> dplyr::mutate(variable = 'mort')

Normalization effect - SDI RFs

annotations = list( 
  list( 
    x = 0.2,  
    y = 1.0,  
    text = "Normalized",  
    xref = "paper",  
    yref = "paper",  
    xanchor = "center",  
    yanchor = "bottom",  
    showarrow = FALSE 
  ),  
  list( 
    x = 0.8,  
    y = 1,  
    text = "Regular RF",  
    xref = "paper",  
    yref = "paper",  
    xanchor = "center",  
    yanchor = "bottom",  
    showarrow = FALSE 
  )
)
plotly::subplot(sdi_plot, sdi_plot_non, nrows = 1) |> plotly::layout(title = list(text = 'SDI'), showlegend = F, annotations = annotations)

Normalization effect - TPA RFs

This plot exemplifies the problem with the former RF approach – everything is overstocked in the West, so FVS wants to drive down the TPA of oaks through increased mortality. By comparing with the base line scenario, we see that there is actually an increase in TPA following most treatments.

On the other hand, look at the 30 year projections for herbivory and targeted herbicide – these treatments are not that extreme! But the sample size for these treatments is low and outlier site conditions are accentuated

plotly::subplot(tpa_plot,tpa_plot_non, nrows = 1) |>  plotly::layout(title = list(text = 'TPA'), showlegend = F, annotations = annotations)

Normalization effect - BA RFs

plotly::subplot(ba_plot, ba_plot_non, nrows = 1) |>  plotly::layout(title = list(text = 'BA'), showlegend = F, annotations = annotations)

Normalization effect - Mort RFs

plotly::subplot(mort_plot, mort_plot_non, nrows = 1) |> plotly::layout(title = list(text = 'Mortality'), showlegend = F, annotations = annotations)

Tabular RFs - Normalized

sdi_table |>
  dplyr::bind_rows(tpa_table, ba_table, mort_table) |>
  dplyr::filter(year %in% 2040) |>
  data.frame()
##    MgmtID year variable   n    min    max median     q1     q3   iqr   mad
## 1    BASE 2040      sdi 170  0.000  0.000  0.000  0.000  0.000 0.000 0.000
## 2    FIC1 2040      sdi 170 -0.658 -0.022 -0.217 -0.343 -0.130 0.213 0.152
## 3    FIC2 2040      sdi 170 -0.658 -0.022 -0.218 -0.343 -0.130 0.213 0.153
## 4    FIC3 2040      sdi 170 -1.000 -0.039 -0.239 -0.368 -0.149 0.219 0.154
## 5    FIC4 2040      sdi 170 -1.000 -0.058 -0.258 -0.384 -0.169 0.215 0.153
## 6    FIC5 2040      sdi 170 -0.750 -0.266 -0.453 -0.552 -0.375 0.178 0.128
## 7    FIC6 2040      sdi 170 -0.958  0.355 -0.787 -0.842 -0.746 0.096 0.072
## 8    BASE 2040      tpa 170  0.000  0.000  0.000  0.000  0.000 0.000 0.000
## 9    FIC1 2040      tpa 170 -0.044  0.264  0.110  0.022  0.164 0.142 0.089
## 10   FIC2 2040      tpa 170 -0.044  0.264  0.110  0.022  0.164 0.142 0.089
## 11   FIC3 2040      tpa 170 -0.052  0.286  0.115  0.019  0.171 0.152 0.092
## 12   FIC4 2040      tpa 170 -0.051  0.297  0.119  0.020  0.177 0.157 0.095
## 13   FIC5 2040      tpa 170 -0.049  0.324  0.192  0.030  0.232 0.201 0.079
## 14   FIC6 2040      tpa 170 -0.058  0.366  0.264  0.040  0.282 0.242 0.066
## 15   BASE 2040       ba 170  0.000  0.000  0.000  0.000  0.000 0.000 0.000
## 16   FIC1 2040       ba 170 -0.795 -0.036 -0.305 -0.442 -0.191 0.250 0.181
## 17   FIC2 2040       ba 170 -0.795 -0.036 -0.305 -0.442 -0.191 0.251 0.181
## 18   FIC3 2040       ba 170 -1.000 -0.061 -0.329 -0.479 -0.215 0.264 0.184
## 19   FIC4 2040       ba 170 -1.000 -0.088 -0.352 -0.491 -0.240 0.251 0.177
## 20   FIC5 2040       ba 170 -0.879 -0.365 -0.569 -0.675 -0.492 0.183 0.132
## 21   FIC6 2040       ba 170 -0.993  0.388 -0.884 -0.923 -0.848 0.076 0.056
## 22   BASE 2040     mort 170  0.000  0.000  0.000  0.000  0.000 0.000 0.000
## 23   FIC1 2040     mort 170 -1.000  0.946 -0.190 -0.596  0.171 0.766 0.558
## 24   FIC2 2040     mort 170 -1.000  0.946 -0.205 -0.616  0.171 0.786 0.571
## 25   FIC3 2040     mort 170 -1.000  0.947 -0.173 -0.584  0.183 0.766 0.584
## 26   FIC4 2040     mort 170 -1.000  0.949 -0.162 -0.565  0.191 0.756 0.567
## 27   FIC5 2040     mort 170 -1.000  0.971 -0.097 -0.531  0.245 0.775 0.593
## 28   FIC6 2040     mort 170 -1.000  0.980 -0.068 -0.479  0.283 0.762 0.584
##      mean    sd    se    ci
## 1   0.000 0.000 0.000 0.000
## 2  -0.250 0.153 0.012 0.023
## 3  -0.250 0.153 0.012 0.023
## 4  -0.277 0.177 0.014 0.027
## 5  -0.291 0.164 0.013 0.025
## 6  -0.463 0.113 0.009 0.017
## 7  -0.772 0.169 0.013 0.026
## 8   0.000 0.000 0.000 0.000
## 9   0.096 0.083 0.006 0.013
## 10  0.096 0.083 0.006 0.013
## 11  0.100 0.086 0.007 0.013
## 12  0.105 0.088 0.007 0.013
## 13  0.143 0.110 0.008 0.017
## 14  0.178 0.134 0.010 0.020
## 15  0.000 0.000 0.000 0.000
## 16 -0.327 0.173 0.013 0.026
## 17 -0.327 0.173 0.013 0.026
## 18 -0.357 0.187 0.014 0.028
## 19 -0.376 0.177 0.014 0.027
## 20 -0.579 0.112 0.009 0.017
## 21 -0.859 0.174 0.013 0.026
## 22  0.000 0.000 0.000 0.000
## 23 -0.177 0.469 0.036 0.071
## 24 -0.181 0.473 0.036 0.072
## 25 -0.165 0.477 0.037 0.072
## 26 -0.150 0.476 0.036 0.072
## 27 -0.097 0.489 0.038 0.074
## 28 -0.060 0.500 0.038 0.076

Tabular RFs - Raw

sdi_table_non |>
  dplyr::bind_rows(tpa_table_non, ba_table_non, mort_table_non) |>
  dplyr::filter(year %in% 2040) |>
  data.frame()
##    MgmtID year variable   n    min    max median     q1     q3   iqr   mad
## 1    BASE 2040      sdi 170 -0.145  0.515  0.005 -0.010  0.013 0.023 0.017
## 2    FIC1 2040      sdi 170 -0.551  0.180 -0.224 -0.336 -0.136 0.200 0.149
## 3    FIC2 2040      sdi 170 -0.551  0.180 -0.223 -0.336 -0.136 0.200 0.149
## 4    FIC3 2040      sdi 170 -0.863 -0.038 -0.242 -0.354 -0.156 0.198 0.147
## 5    FIC4 2040      sdi 170 -0.807 -0.057 -0.262 -0.371 -0.175 0.195 0.145
## 6    FIC5 2040      sdi 170 -0.704  0.158 -0.454 -0.538 -0.378 0.160 0.117
## 7    FIC6 2040      sdi 170 -0.962  0.815 -0.791 -0.852 -0.756 0.096 0.067
## 8    BASE 2040      tpa 170 -0.256  0.000 -0.012 -0.039 -0.006 0.033 0.013
## 9    FIC1 2040      tpa 170 -0.076  0.198  0.068 -0.015  0.141 0.156 0.111
## 10   FIC2 2040      tpa 170 -0.076  0.198  0.068 -0.015  0.141 0.156 0.111
## 11   FIC3 2040      tpa 170 -0.078  0.202  0.079 -0.019  0.149 0.168 0.117
## 12   FIC4 2040      tpa 170 -0.082  0.207  0.092 -0.015  0.155 0.170 0.110
## 13   FIC5 2040      tpa 170 -0.059  0.241  0.180 -0.004  0.212 0.216 0.075
## 14   FIC6 2040      tpa 170 -0.063  0.280  0.232  0.003  0.269 0.266 0.069
## 15   BASE 2040       ba 170 -0.140  0.649  0.008 -0.006  0.018 0.025 0.017
## 16   FIC1 2040       ba 170 -0.641  0.241 -0.296 -0.439 -0.190 0.248 0.170
## 17   FIC2 2040       ba 170 -0.641  0.241 -0.297 -0.436 -0.190 0.246 0.170
## 18   FIC3 2040       ba 170 -0.924 -0.059 -0.319 -0.458 -0.219 0.239 0.170
## 19   FIC4 2040       ba 170 -0.881 -0.086 -0.343 -0.477 -0.243 0.233 0.166
## 20   FIC5 2040       ba 170 -0.792  0.210 -0.567 -0.648 -0.489 0.160 0.118
## 21   FIC6 2040       ba 170 -0.985  0.894 -0.881 -0.922 -0.856 0.066 0.050
## 22   BASE 2040     mort 170 -0.850  0.838 -0.007 -0.039  0.015 0.054 0.039
## 23   FIC1 2040     mort 170 -0.993  0.915 -0.146 -0.615  0.177 0.792 0.565
## 24   FIC2 2040     mort 170 -0.993  0.915 -0.146 -0.627  0.177 0.804 0.565
## 25   FIC3 2040     mort 170 -0.998  0.932 -0.153 -0.613  0.186 0.799 0.580
## 26   FIC4 2040     mort 170 -0.939  0.943 -0.148 -0.566  0.192 0.758 0.570
## 27   FIC5 2040     mort 170 -0.931  0.975 -0.096 -0.527  0.251 0.779 0.579
## 28   FIC6 2040     mort 170 -0.930  0.983 -0.074 -0.487  0.296 0.783 0.599
##      mean    sd    se    ci
## 1   0.007 0.059 0.005 0.009
## 2  -0.243 0.140 0.011 0.021
## 3  -0.243 0.140 0.011 0.021
## 4  -0.272 0.155 0.012 0.023
## 5  -0.285 0.141 0.011 0.021
## 6  -0.456 0.114 0.009 0.017
## 7  -0.765 0.218 0.017 0.033
## 8  -0.027 0.036 0.003 0.005
## 9   0.069 0.081 0.006 0.012
## 10  0.069 0.081 0.006 0.012
## 11  0.073 0.085 0.006 0.013
## 12  0.078 0.087 0.007 0.013
## 13  0.116 0.111 0.008 0.017
## 14  0.150 0.136 0.010 0.021
## 15  0.015 0.071 0.005 0.011
## 16 -0.313 0.160 0.012 0.024
## 17 -0.313 0.160 0.012 0.024
## 18 -0.346 0.168 0.013 0.025
## 19 -0.364 0.154 0.012 0.023
## 20 -0.564 0.117 0.009 0.018
## 21 -0.844 0.236 0.018 0.036
## 22  0.003 0.152 0.012 0.023
## 23 -0.175 0.475 0.036 0.072
## 24 -0.180 0.478 0.037 0.072
## 25 -0.163 0.483 0.037 0.073
## 26 -0.149 0.480 0.037 0.073
## 27 -0.095 0.498 0.038 0.075
## 28 -0.059 0.508 0.039 0.077