This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.

When you click the Knit button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:

Recipes for the Design of Experiments: Recipe Outline

as of August 28, 2014, superceding the version of August 24. Always use the most recent version.

Design of Experiments: Recipe 3 - Two or More Factor Analysis Part 2

Kevin Toth

RPI

10/08/2014 V2.0

1. Setting

Traffic Fatality and Drunk Driving Laws

This analysis uses drinking laws and demographic statistics by state to test the effect different factors may have on traffic fatality rate in deaths per 10,000.

Below is the installation and initial examination of the dataset:

#Obtaining data package
library("Ecdat", lib.loc="~/R/win-library/3.1")
## Loading required package: Ecfun
## 
## Attaching package: 'Ecdat'
## 
## The following object is masked from 'package:datasets':
## 
##     Orange
data<-Fatality
head(data)
##   state year mrall beertax  mlda jaild comserd vmiles unrate perinc
## 1     1 1982 2.128   1.539 19.00    no      no  7.234   14.4  10544
## 2     1 1983 2.348   1.789 19.00    no      no  7.836   13.7  10733
## 3     1 1984 2.336   1.714 19.00    no      no  8.263   11.1  11109
## 4     1 1985 2.193   1.653 19.67    no      no  8.727    8.9  11333
## 5     1 1986 2.669   1.610 21.00    no      no  8.953    9.8  11662
## 6     1 1987 2.719   1.560 21.00    no      no  9.166    7.8  11944
attach(data)

Factors and Levels

For this multi-factor analysis we will be examining the factors “mlda” which represents the minimum legal drinking age in that state, “jaild” which represents which represents whether or not a mandatory jail sentence exists for that state and “perinc” which represents the per capita personal income for that state.

Minimum legal drinking age is broken down by 4 levels which are 18, 19, 20, 21. Mandatory jail sentence is a yes or no variable representing whether or not it does or does not have a mandatory jail sentence. Per capita personal income will have 3 levels, 9,000-14,000, 14,000-19,000 and 19,000-24,000.

#Subset data for specific values of age and unemployment rate and provide summary statistics
data$mlda[data$mlda>=18 & data$mlda < 19] ="18"
data$mlda[data$mlda>=19 & data$mlda < 20] ="19"
data$mlda[data$mlda>=20 & data$mlda < 21] ="20"
data$mlda[data$mlda>=21] ="21"

data$perinc[data$perinc>= 9000 & data$perinc <14000] = "9,000-14,000"
data$perinc[data$perinc>= 14000 & data$perinc <19000] = "14,000-19,000"
data$perinc[data$perinc>= 19000 & data$perinc <24000] = "19,000-24,000"

head(data)
##   state year mrall beertax mlda jaild comserd vmiles unrate       perinc
## 1     1 1982 2.128   1.539   19    no      no  7.234   14.4 9,000-14,000
## 2     1 1983 2.348   1.789   19    no      no  7.836   13.7 9,000-14,000
## 3     1 1984 2.336   1.714   19    no      no  8.263   11.1 9,000-14,000
## 4     1 1985 2.193   1.653   19    no      no  8.727    8.9 9,000-14,000
## 5     1 1986 2.669   1.610   21    no      no  8.953    9.8 9,000-14,000
## 6     1 1987 2.719   1.560   21    no      no  9.166    7.8 9,000-14,000
tail(data)
##     state year mrall beertax mlda jaild comserd vmiles unrate       perinc
## 331    56 1983 3.353 0.05161   19   yes      no  9.804    8.4 9,000-14,000
## 332    56 1984 3.060 0.04945   19   yes      no  9.994    6.3 9,000-14,000
## 333    56 1985 2.986 0.04767   19   yes      no 10.611    7.1 9,000-14,000
## 334    56 1986 3.314 0.04644   19   yes      no 10.619    9.0 9,000-14,000
## 335    56 1987 2.633 0.04500   19   yes      no 10.953    8.6 9,000-14,000
## 336    56 1988 3.236 0.04331   19   yes      no 11.812    6.3 9,000-14,000
summary(data)
##      state           year          mrall          beertax      
##  Min.   : 1.0   Min.   :1982   Min.   :0.821   Min.   :0.0433  
##  1st Qu.:18.8   1st Qu.:1983   1st Qu.:1.624   1st Qu.:0.2088  
##  Median :30.5   Median :1985   Median :1.956   Median :0.3526  
##  Mean   :30.2   Mean   :1985   Mean   :2.040   Mean   :0.5133  
##  3rd Qu.:42.5   3rd Qu.:1987   3rd Qu.:2.418   3rd Qu.:0.6516  
##  Max.   :56.0   Max.   :1988   Max.   :4.218   Max.   :2.7208  
##      mlda           jaild     comserd       vmiles          unrate     
##  Length:336         no :242   no :274   Min.   : 4.58   Min.   : 2.40  
##  Class :character   yes: 94   yes: 62   1st Qu.: 7.18   1st Qu.: 5.47  
##  Mode  :character                       Median : 7.80   Median : 7.00  
##                                         Mean   : 7.89   Mean   : 7.35  
##                                         3rd Qu.: 8.50   3rd Qu.: 8.90  
##                                         Max.   :26.15   Max.   :18.00  
##     perinc         
##  Length:336        
##  Class :character  
##  Mode  :character  
##                    
##                    
## 
#Levels of Minimum Legal Drinking Age
unique(data$mlda)
## [1] "19" "21" "18" "20"
#Levels of Mandatory Jail Sentence
unique(data$jaild)
## [1] no  yes
## Levels: no yes
#Levels of Per Capita Personal Income
unique(data$perinc)
## [1] "9,000-14,000"  "14,000-19,000" "19,000-24,000"

Continuous variables (if any)

The continuous variables in this data are the death rate (“mrall”), the beer tax (“beertax”), the minimum legal drinking age (“mlda”), average miles per drive (“vmiles”), unemployment rate (“unrate”) and per capita personal income (“perinc”).

Response variables

For this experiment we will be focusing on the traffic fatality rate (deaths per 1 0,000) as the response variable being effected by the minimum legal drinking age, whether or not there is a mandatory jail sentence and per capita personal income.

The Data: How is it organized and what does it look like?

The data set is organized by the following variables: state, year, mrall, beertax, mlda, jaild, comserd, vmiles, unrate, and perinc.

State and year indicate the state ID number in which this data is observed and what year the data was recorded. Mrall represents the traffic fatality rate in deaths per 10,000 people.

beertax, mlda, jaild, and comserd all have to do with alcohol related laws in that state. Beer tax is the tax rate on a case of beer, mlda is the minimum legal drinking age, and jaild and comserd represent whether or not there is a mandatory jail sentence or community service time for that state in the event of alcohol related driving offenses.

vmiles is the average miles per drive.

Lastly unrate and perinc represent demographic information for that state. unrate is the unemployment rate and perinc is the per capita personal income for that observation.

The data itself is in order by state ID and then by year.

Randomization

It is unknown whether or not the data collected for this study was collected by a randomly designed experiment. Not much information surrounding the study is available.

2. (Experimental) Design

How will the experiment be organized and conducted to test the hypothesis?

To perform the experiment the data will first be subset for the continuous variables of age and per capita personal income.This is done to create the 3 levels for both factors. Then an analysis of variance will be performed on each factor individually, then on each combination of two factors and finally on a combination of all three factors. From these analysis we will be able to test the null hypothesis, that the traffic fatality rate is independant of the factors.

What is the rationale for this design?

The rationale for using an analysis of variance test is used when multiple factors are considered. It checks whether the means of several groups are equal. The alternative would be to use multiple two-sample t-tests however there is more likely chance of the test resulting in a false hypothesis.

Randomize: What is the Randomization Scheme?

The data was collected in an unknown way so we do not know if there was any randomization to the collection.

Replicate: Are there replicates and/or repeated measures?

There are no replicated or repeated measures in the data. Each state and year had its statistics measured once.

Block: Did you use blocking in the design?

There was no blocking performed in the design of this experiment. All data observed was analyzed together.

3. Statistical Analysis

Exploratory Data Analysis: Graphics and Descriptive summary

To start our statistical analysis we will make our variables factors for the analysis of variance and look at some boxplots of those factors.

#Defining Minimum Legal Drinking Age as a factor
data$mlda = as.factor(data$mlda)

#Defining Mandatory Jail Sentence as a factor
data$jaild = as.factor(data$jaild)

#Defining Per Capital Personal Income as a factor
data$perinc = as.factor(data$perinc)

#Boxplots of of the means of each variable against response variable
boxplot(mrall~mlda, data=data, xlab="Minimum Drinking Age", ylab="Fatality Rate")

plot of chunk unnamed-chunk-3

boxplot(mrall~jaild, data=data, xlab="Mandatory Jail Time", ylab="Fatality Rate")

plot of chunk unnamed-chunk-3

boxplot(mrall~perinc, data=data, xlab="Per Capita Personal Income", ylab="Fatality Rate")

plot of chunk unnamed-chunk-3

The initial boxplots don’t appear to imply that there is any relationship between traffic fatality rates and the chosen factors. We see similar means and ranges for each level of each factor.

Testing

To test the hypotheses we perform an ANOVA test on the factors individually and then in combination.

The null hypothesis of the first three tests is that the single factor does not have an effect on the response variable of traffic fatality rates.

#Analysis of Variance for Minimum Legal Drinking Age
model1=aov(mrall~mlda, data=data)
anova(model1)
## Analysis of Variance Table
## 
## Response: mrall
##            Df Sum Sq Mean Sq F value  Pr(>F)    
## mlda        3    5.7   1.908    6.14 0.00045 ***
## Residuals 332  103.2   0.311                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#Analysis of Variance for Mandatory Jail Time
model2=aov(mrall~jaild, data=data)
anova(model2)
## Analysis of Variance Table
## 
## Response: mrall
##            Df Sum Sq Mean Sq F value  Pr(>F)    
## jaild       1    8.4    8.43      28 2.2e-07 ***
## Residuals 334  100.5    0.30                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#Analysis of Variance for Per Capita Personal Income
model3=aov(mrall~perinc, data=data)
anova(model3)
## Analysis of Variance Table
## 
## Response: mrall
##            Df Sum Sq Mean Sq F value  Pr(>F)    
## perinc      2   17.9    8.93    32.7 1.1e-13 ***
## Residuals 333   91.0    0.27                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#Analysis of Variance for combination of Minimum Legal Drinking Age and Mandatory Jail Time
model12=aov(mrall~mlda*jaild, data=data)
anova(model12)
## Analysis of Variance Table
## 
## Response: mrall
##             Df Sum Sq Mean Sq F value  Pr(>F)    
## mlda         3    5.7    1.91    6.85 0.00017 ***
## jaild        1    7.7    7.71   27.69 2.6e-07 ***
## mlda:jaild   3    4.2    1.38    4.97 0.00219 ** 
## Residuals  328   91.3    0.28                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#Analysis of Variance for combination of Minimum Legal Drinking Age and Per Capita Personal Income
model13=aov(mrall~mlda*perinc, data=data)
anova(model13)
## Analysis of Variance Table
## 
## Response: mrall
##              Df Sum Sq Mean Sq F value  Pr(>F)    
## mlda          3    5.7    1.91    7.24   1e-04 ***
## perinc        2   16.3    8.13   30.85 5.4e-13 ***
## mlda:perinc   4    1.1    0.27    1.02   4e-01    
## Residuals   326   85.9    0.26                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#Analysis of Variance for combination of Mandatory Jail Time and Per Capita Personal Income
model23=aov(mrall~jaild*perinc, data=data)
anova(model23)
## Analysis of Variance Table
## 
## Response: mrall
##               Df Sum Sq Mean Sq F value  Pr(>F)    
## jaild          1    8.4    8.43   33.44 1.7e-08 ***
## perinc         2   15.7    7.85   31.14 4.1e-13 ***
## jaild:perinc   2    1.6    0.80    3.17   0.043 *  
## Residuals    330   83.2    0.25                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#Analysis of Variance for all three factors
model123=aov(mrall~mlda*jaild*perinc, data=data)
anova(model123)
## Analysis of Variance Table
## 
## Response: mrall
##                    Df Sum Sq Mean Sq F value  Pr(>F)    
## mlda                3    5.7    1.91    8.37 2.3e-05 ***
## jaild               1    7.7    7.71   33.83 1.5e-08 ***
## perinc              2   14.8    7.39   32.44 1.5e-13 ***
## mlda:jaild          3    3.0    0.99    4.35  0.0051 ** 
## mlda:perinc         4    0.4    0.10    0.45  0.7759    
## jaild:perinc        2    3.1    1.57    6.89  0.0012 ** 
## mlda:jaild:perinc   1    1.5    1.48    6.49  0.0113 *  
## Residuals         319   72.7    0.23                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

All individual factor tests had resulting p-values of less than .001.The p-value represents the probability that we can get the F value using the null hypothesis. Since our probability is extremely close to 0 we can assume that each factordemonstrates an effect on the response variable. We are lead to reject the null hypothesis on an individual factor basis.

Our two factor combination models still show relatively low p-values for the indivdual factors however we also see p-values for the interaction effect of two factors. Minimum Age and Mandatory Jail Sentence has a significant p-value. The interaction effect of Age and Income has a high p-value, leading us to accept the null hypothesis that they do not have an effect on the response variable. Lastly the interaction effect of Jail Sentence and Income still has a significant p-value, however it is not as strong as the Age and Jail Sentence analysis of variance.

Lastly we looked at the analysis of all three factors. We saw simliar results as the individual and combination analysis. All individual factors still appear to have an effect, age and jail sentence and jail sentence and income still have somewhat of an effect, and lastly we see that a combination of the three factors, while it is large for a p-value, we still consider it acceptable.

Diagnostics/Model Adequacy Checking

Next we graph Q-Q plots to check our data in our model for normality. If the data is not normal the results of the analysis may not be valid. We also do a Shapiro-Wilke normality test on our data as a secondary normality test for our continuous variables.

#QQ plots for residuals of Minimum Drinking Age and Mandatory Jail Sentence model
qqnorm(residuals(model12))
qqline(residuals(model12))

plot of chunk unnamed-chunk-5

#QQ plots for residuals of Minimum Drinking Age and Per Capita Personal Income model
qqnorm(residuals(model13))
qqline(residuals(model13))

plot of chunk unnamed-chunk-5

#QQ plots for residuals of Mandatory Jail Sentence and Per Capita Personal Income model
qqnorm(residuals(model23))
qqline(residuals(model23))

plot of chunk unnamed-chunk-5

#QQ plots for residuals of combination of all factors model
qqnorm(residuals(model123))
qqline(residuals(model123))

plot of chunk unnamed-chunk-5

#Shapiro-Wilke Test of Normality for our numeric factors
shapiro.test(Fatality[,"mlda"])
## 
##  Shapiro-Wilk normality test
## 
## data:  Fatality[, "mlda"]
## W = 0.639, p-value < 2.2e-16
shapiro.test(Fatality[,"perinc"])
## 
##  Shapiro-Wilk normality test
## 
## data:  Fatality[, "perinc"]
## W = 0.968, p-value = 9.169e-07

Our qq-plots show data that appears normal.

We use the original data set in our Shapiro-Wilke test to include the original values for each factor. We feel the shapiro-wilke test shows adequate normality if our p is < 0.1. In both cases our p-value is adequate.

We also perform the Tukey test which is used to determine which groups in the sample differ as opposed to the anova which tells whether or not there are differences in groups.

#Tukey HSD Test for combination Minimum Drinking Age and Mandatory Jail Time model
TukeyHSD(model12, ordered = FALSE, conf.level = 0.95)
##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov(formula = mrall ~ mlda * jaild, data = data)
## 
## $mlda
##          diff      lwr      upr  p adj
## 19-18  0.2512 -0.14217  0.64453 0.3527
## 20-18 -0.2806 -0.71399  0.15273 0.3401
## 21-18  0.0263 -0.33672  0.38932 0.9977
## 20-19 -0.5318 -0.83998 -0.22364 0.0001
## 21-19 -0.2249 -0.42223 -0.02753 0.0182
## 21-20  0.3069  0.03855  0.57531 0.0177
## 
## $jaild
##          diff    lwr    upr p adj
## yes-no 0.3349 0.2088 0.4611     0
## 
## $`mlda:jaild`
##                   diff      lwr     upr  p adj
## 19:no-18:no    0.12974 -0.49505  0.7545 0.9984
## 20:no-18:no   -0.17223 -0.83683  0.4924 0.9935
## 21:no-18:no    0.14344 -0.43871  0.7256 0.9953
## 18:yes-18:no   0.35940 -0.47373  1.1925 0.8924
## 19:yes-18:no   0.95592  0.28711  1.6247 0.0005
## 20:yes-18:no   0.07350 -0.75962  0.9066 1.0000
## 21:yes-18:no   0.34234 -0.26416  0.9488 0.6729
## 20:no-19:no   -0.30197 -0.73119  0.1273 0.3876
## 21:no-19:no    0.01370 -0.27165  0.2990 1.0000
## 18:yes-19:no   0.22966 -0.43112  0.8904 0.9644
## 19:yes-19:no   0.82618  0.39047  1.2619 0.0000
## 20:yes-19:no  -0.05624 -0.71701  0.6045 1.0000
## 21:yes-19:no   0.21260 -0.11961  0.5448 0.5160
## 21:no-20:no    0.31566 -0.04871  0.6800 0.1446
## 18:yes-20:no   0.53162 -0.16693  1.2302 0.2850
## 19:yes-20:no   1.12814  0.63704  1.6192 0.0000
## 20:yes-20:no   0.24573 -0.45282  0.9443 0.9620
## 21:yes-20:no   0.51456  0.11243  0.9167 0.0029
## 18:yes-21:no   0.21596 -0.40466  0.8366 0.9641
## 19:yes-21:no   0.81248  0.44049  1.1845 0.0000
## 20:yes-21:no  -0.06993 -0.69055  0.5507 1.0000
## 21:yes-21:no   0.19890 -0.04379  0.4416 0.1990
## 19:yes-18:yes  0.59652 -0.10603  1.2991 0.1632
## 20:yes-18:yes -0.28589 -1.14634  0.5746 0.9722
## 21:yes-18:yes -0.01706 -0.66057  0.6265 1.0000
## 20:yes-19:yes -0.88241 -1.58497 -0.1799 0.0038
## 21:yes-19:yes -0.61358 -1.02262 -0.2045 0.0002
## 21:yes-20:yes  0.26883 -0.37468  0.9123 0.9077
#Tukey HSD Test for combination Minimum Drinking Age and Per Capita Personal Income model
TukeyHSD(model13, ordered = FALSE, conf.level = 0.95)
##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov(formula = mrall ~ mlda * perinc, data = data)
## 
## $mlda
##          diff     lwr      upr  p adj
## 19-18  0.2512 -0.1314  0.63377 0.3277
## 20-18 -0.2806 -0.7021  0.14087 0.3153
## 21-18  0.0263 -0.3268  0.37939 0.9975
## 20-19 -0.5318 -0.8316 -0.23208 0.0000
## 21-19 -0.2249 -0.4168 -0.03293 0.0142
## 21-20  0.3069  0.0459  0.56797 0.0137
## 
## $perinc
##                                diff     lwr      upr  p adj
## 19,000-24,000-14,000-19,000 -0.4417 -0.8573 -0.02605 0.0342
## 9,000-14,000-14,000-19,000   0.3763  0.2411  0.51142 0.0000
## 9,000-14,000-19,000-24,000   0.8179  0.4056  1.23025 0.0000
## 
## $`mlda:perinc`
##                                       diff      lwr      upr  p adj
## 19:14,000-19,000-18:14,000-19,000  0.09931 -1.64955  1.84817 1.0000
## 20:14,000-19,000-18:14,000-19,000 -0.04001 -1.79335  1.71333 1.0000
## 21:14,000-19,000-18:14,000-19,000  0.21264 -1.48452  1.90979 1.0000
## 18:19,000-24,000-18:14,000-19,000       NA       NA       NA     NA
## 19:19,000-24,000-18:14,000-19,000       NA       NA       NA     NA
## 20:19,000-24,000-18:14,000-19,000 -0.23548 -2.62488  2.15392 1.0000
## 21:19,000-24,000-18:14,000-19,000 -0.30072 -2.09277  1.49133 1.0000
## 18:9,000-14,000-18:14,000-19,000   0.38004 -1.36882  2.12890 0.9999
## 19:9,000-14,000-18:14,000-19,000   0.76006 -0.94777  2.46788 0.9488
## 20:9,000-14,000-18:14,000-19,000   0.19358 -1.55139  1.93855 1.0000
## 21:9,000-14,000-18:14,000-19,000   0.59465 -1.10237  2.29167 0.9918
## 20:14,000-19,000-19:14,000-19,000 -0.13932 -0.79008  0.51144 0.9999
## 21:14,000-19,000-19:14,000-19,000  0.11333 -0.36586  0.59251 0.9998
## 18:19,000-24,000-19:14,000-19,000       NA       NA       NA     NA
## 19:19,000-24,000-19:14,000-19,000       NA       NA       NA     NA
## 20:19,000-24,000-19:14,000-19,000 -0.33479 -2.08365  1.41407 1.0000
## 21:19,000-24,000-19:14,000-19,000 -0.40003 -1.14885  0.34879 0.8389
## 18:9,000-14,000-19:14,000-19,000   0.28073 -0.35787  0.91932 0.9530
## 19:9,000-14,000-19:14,000-19,000   0.66075  0.14504  1.17646 0.0019
## 20:9,000-14,000-19:14,000-19,000   0.09427 -0.53359  0.72213 1.0000
## 21:9,000-14,000-19:14,000-19,000   0.49534  0.01663  0.97405 0.0352
## 21:14,000-19,000-20:14,000-19,000  0.25265 -0.24263  0.74793 0.8765
## 18:19,000-24,000-20:14,000-19,000       NA       NA       NA     NA
## 19:19,000-24,000-20:14,000-19,000       NA       NA       NA     NA
## 20:19,000-24,000-20:14,000-19,000 -0.19547 -1.94881  1.55787 1.0000
## 21:19,000-24,000-20:14,000-19,000 -0.26071 -1.01993  0.49851 0.9931
## 18:9,000-14,000-20:14,000-19,000   0.42005 -0.23071  1.07081 0.6053
## 19:9,000-14,000-20:14,000-19,000   0.80007  0.26937  1.33077 0.0001
## 20:9,000-14,000-20:14,000-19,000   0.23359 -0.40664  0.87382 0.9886
## 21:9,000-14,000-20:14,000-19,000   0.63466  0.13984  1.12948 0.0018
## 18:19,000-24,000-21:14,000-19,000       NA       NA       NA     NA
## 19:19,000-24,000-21:14,000-19,000       NA       NA       NA     NA
## 20:19,000-24,000-21:14,000-19,000 -0.44812 -2.14527  1.24904 0.9993
## 21:19,000-24,000-21:14,000-19,000 -0.51336 -1.13186  0.10514 0.2153
## 18:9,000-14,000-21:14,000-19,000   0.16740 -0.31179  0.64658 0.9920
## 19:9,000-14,000-21:14,000-19,000   0.54742  0.25115  0.84369 0.0000
## 20:9,000-14,000-21:14,000-19,000  -0.01906 -0.48385  0.44572 1.0000
## 21:9,000-14,000-21:14,000-19,000   0.38201  0.15622  0.60779 0.0000
## 19:19,000-24,000-18:19,000-24,000       NA       NA       NA     NA
## 20:19,000-24,000-18:19,000-24,000       NA       NA       NA     NA
## 21:19,000-24,000-18:19,000-24,000       NA       NA       NA     NA
## 18:9,000-14,000-18:19,000-24,000        NA       NA       NA     NA
## 19:9,000-14,000-18:19,000-24,000        NA       NA       NA     NA
## 20:9,000-14,000-18:19,000-24,000        NA       NA       NA     NA
## 21:9,000-14,000-18:19,000-24,000        NA       NA       NA     NA
## 20:19,000-24,000-19:19,000-24,000       NA       NA       NA     NA
## 21:19,000-24,000-19:19,000-24,000       NA       NA       NA     NA
## 18:9,000-14,000-19:19,000-24,000        NA       NA       NA     NA
## 19:9,000-14,000-19:19,000-24,000        NA       NA       NA     NA
## 20:9,000-14,000-19:19,000-24,000        NA       NA       NA     NA
## 21:9,000-14,000-19:19,000-24,000        NA       NA       NA     NA
## 21:19,000-24,000-20:19,000-24,000 -0.06524 -1.85729  1.72681 1.0000
## 18:9,000-14,000-20:19,000-24,000   0.61552 -1.13334  2.36438 0.9915
## 19:9,000-14,000-20:19,000-24,000   0.99554 -0.71229  2.70336 0.7466
## 20:9,000-14,000-20:19,000-24,000   0.42906 -1.31591  2.17403 0.9997
## 21:9,000-14,000-20:19,000-24,000   0.83013 -0.86689  2.52715 0.9044
## 18:9,000-14,000-21:19,000-24,000   0.68076 -0.06806  1.42957 0.1154
## 19:9,000-14,000-21:19,000-24,000   1.06078  0.41356  1.70799 0.0000
## 20:9,000-14,000-21:19,000-24,000   0.49430 -0.24539  1.23398 0.5509
## 21:9,000-14,000-21:19,000-24,000   0.89537  0.27723  1.51350 0.0002
## 19:9,000-14,000-18:9,000-14,000    0.38002 -0.13569  0.89573 0.3917
## 20:9,000-14,000-18:9,000-14,000   -0.18646 -0.81432  0.44140 0.9981
## 21:9,000-14,000-18:9,000-14,000    0.21461 -0.26410  0.69332 0.9462
## 20:9,000-14,000-19:9,000-14,000   -0.56648 -1.06884 -0.06412 0.0127
## 21:9,000-14,000-19:9,000-14,000   -0.16541 -0.46091  0.13009 0.7930
## 21:9,000-14,000-20:9,000-14,000    0.40107 -0.06322  0.86537 0.1670
#Tukey HSD Test for combination Mandatory Jail Time and Per Capita Personal Income model
TukeyHSD(model23, ordered = FALSE, conf.level = 0.95)
##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov(formula = mrall ~ jaild * perinc, data = data)
## 
## $jaild
##          diff    lwr    upr p adj
## yes-no 0.3529 0.2328 0.4729     0
## 
## $perinc
##                                diff     lwr     upr  p adj
## 19,000-24,000-14,000-19,000 -0.5578 -0.9644 -0.1513 0.0039
## 9,000-14,000-14,000-19,000   0.3614  0.2291  0.4936 0.0000
## 9,000-14,000-19,000-24,000   0.9192  0.5159  1.3225 0.0000
## 
## $`jaild:perinc`
##                                         diff       lwr      upr  p adj
## yes:14,000-19,000-no:14,000-19,000   0.52960  0.221039  0.83816 0.0000
## no:19,000-24,000-no:14,000-19,000   -0.42712 -1.084962  0.23073 0.4281
## yes:19,000-24,000-no:14,000-19,000  -0.29110 -1.023444  0.44125 0.8646
## no:9,000-14,000-no:14,000-19,000     0.44606  0.258807  0.63332 0.0000
## yes:9,000-14,000-no:14,000-19,000    0.64863  0.421975  0.87529 0.0000
## no:19,000-24,000-yes:14,000-19,000  -0.95672 -1.657419 -0.25601 0.0015
## yes:19,000-24,000-yes:14,000-19,000 -0.82070 -1.591771 -0.04963 0.0295
## no:9,000-14,000-yes:14,000-19,000   -0.08354 -0.388967  0.22189 0.9701
## yes:9,000-14,000-yes:14,000-19,000   0.11903 -0.212022  0.45008 0.9074
## yes:19,000-24,000-no:19,000-24,000   0.13602 -0.829438  1.10147 0.9986
## no:9,000-14,000-no:19,000-24,000     0.87318  0.216796  1.52956 0.0022
## yes:9,000-14,000-no:19,000-24,000    1.07575  0.407056  1.74444 0.0001
## no:9,000-14,000-yes:19,000-24,000    0.73716  0.006131  1.46819 0.0468
## yes:9,000-14,000-yes:19,000-24,000   0.93973  0.197628  1.68183 0.0044
## yes:9,000-14,000-no:9,000-14,000     0.20257 -0.019805  0.42494 0.0974
#Tukey HSD Test for the model with all three factors
TukeyHSD(model123, ordered = FALSE, conf.level = 0.95)
##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov(formula = mrall ~ mlda * jaild * perinc, data = data)
## 
## $mlda
##          diff     lwr      upr  p adj
## 19-18  0.2512 -0.1047  0.60709 0.2645
## 20-18 -0.2806 -0.6727  0.11148 0.2528
## 21-18  0.0263 -0.3022  0.35477 0.9969
## 20-19 -0.5318 -0.8107 -0.25298 0.0000
## 21-19 -0.2249 -0.4034 -0.04631 0.0069
## 21-20  0.3069  0.0641  0.54976 0.0066
## 
## $jaild
##          diff    lwr    upr p adj
## yes-no 0.3349 0.2208 0.4491     0
## 
## $perinc
##                                diff     lwr     upr  p adj
## 19,000-24,000-14,000-19,000 -0.5306 -0.9173 -0.1440 0.0039
## 9,000-14,000-14,000-19,000   0.3380  0.2122  0.4637 0.0000
## 9,000-14,000-19,000-24,000   0.8686  0.4850  1.2521 0.0000
## 
## $`mlda:jaild`
##                   diff      lwr     upr  p adj
## 19:no-18:no    0.14879 -0.41655  0.7141 0.9929
## 20:no-18:no   -0.16529 -0.76666  0.4361 0.9907
## 21:no-18:no    0.12802 -0.39873  0.6548 0.9956
## 18:yes-18:no   0.34372 -0.41014  1.0976 0.8609
## 19:yes-18:no   0.89962  0.29445  1.5048 0.0002
## 20:yes-18:no   0.02140 -0.73246  0.7753 1.0000
## 21:yes-18:no   0.35876 -0.19003  0.9075 0.4871
## 20:no-19:no   -0.31409 -0.70247  0.0743 0.2134
## 21:no-19:no   -0.02077 -0.27897  0.2374 1.0000
## 18:yes-19:no   0.19492 -0.40298  0.7928 0.9750
## 19:yes-19:no   0.75083  0.35658  1.1451 0.0000
## 20:yes-19:no  -0.12740 -0.72530  0.4705 0.9981
## 21:yes-19:no   0.20996 -0.09064  0.5106 0.3972
## 21:no-20:no    0.29332 -0.03638  0.6230 0.1220
## 18:yes-20:no   0.50901 -0.12308  1.1411 0.2180
## 19:yes-20:no   1.06492  0.62054  1.5093 0.0000
## 20:yes-20:no   0.18669 -0.44540  0.8188 0.9858
## 21:yes-20:no   0.52405  0.16018  0.8879 0.0004
## 18:yes-21:no   0.21569 -0.34587  0.7773 0.9393
## 19:yes-21:no   0.77160  0.43501  1.1082 0.0000
## 20:yes-21:no  -0.10663 -0.66819  0.4549 0.9991
## 21:yes-21:no   0.23073  0.01113  0.4503 0.0317
## 19:yes-18:yes  0.55591 -0.07980  1.1916 0.1363
## 20:yes-18:yes -0.32232 -1.10090  0.4563 0.9116
## 21:yes-18:yes  0.01504 -0.56724  0.5973 1.0000
## 20:yes-19:yes -0.87823 -1.51393 -0.2425 0.0008
## 21:yes-19:yes -0.54087 -0.91099 -0.1707 0.0003
## 21:yes-20:yes  0.33736 -0.24492  0.9196 0.6424
## 
## $`mlda:perinc`
##                                       diff       lwr      upr  p adj
## 19:14,000-19,000-18:14,000-19,000  0.15062 -1.476371  1.77762 1.0000
## 20:14,000-19,000-18:14,000-19,000 -0.14902 -1.780179  1.48214 1.0000
## 21:14,000-19,000-18:14,000-19,000  0.06277 -1.516118  1.64166 1.0000
## 18:19,000-24,000-18:14,000-19,000       NA        NA       NA     NA
## 19:19,000-24,000-18:14,000-19,000       NA        NA       NA     NA
## 20:19,000-24,000-18:14,000-19,000 -0.55297 -2.775867  1.66993 0.9996
## 21:19,000-24,000-18:14,000-19,000 -0.50139 -2.168558  1.16579 0.9978
## 18:9,000-14,000-18:14,000-19,000   0.22092 -1.406078  1.84791 1.0000
## 19:9,000-14,000-18:14,000-19,000   0.55073 -1.038089  2.13955 0.9925
## 20:9,000-14,000-18:14,000-19,000   0.02209 -1.601282  1.64547 1.0000
## 21:9,000-14,000-18:14,000-19,000   0.45116 -1.127607  2.02992 0.9986
## 20:14,000-19,000-19:14,000-19,000 -0.29964 -0.905052  0.30577 0.8970
## 21:14,000-19,000-19:14,000-19,000 -0.08785 -0.533644  0.35794 1.0000
## 18:19,000-24,000-19:14,000-19,000       NA        NA       NA     NA
## 19:19,000-24,000-19:14,000-19,000       NA        NA       NA     NA
## 20:19,000-24,000-19:14,000-19,000 -0.70359 -2.330587  0.92340 0.9581
## 21:19,000-24,000-19:14,000-19,000 -0.65201 -1.348646  0.04463 0.0913
## 18:9,000-14,000-19:14,000-19,000   0.07029 -0.523801  0.66439 1.0000
## 19:9,000-14,000-19:14,000-19,000   0.40011 -0.079668  0.87988 0.2091
## 20:9,000-14,000-19:14,000-19,000  -0.12853 -0.712639  0.45558 0.9999
## 21:9,000-14,000-19:14,000-19,000   0.30054 -0.144816  0.74589 0.5350
## 21:14,000-19,000-20:14,000-19,000  0.21179 -0.248977  0.67256 0.9361
## 18:19,000-24,000-20:14,000-19,000       NA        NA       NA     NA
## 19:19,000-24,000-20:14,000-19,000       NA        NA       NA     NA
## 20:19,000-24,000-20:14,000-19,000 -0.40395 -2.035113  1.22721 0.9996
## 21:19,000-24,000-20:14,000-19,000 -0.35237 -1.058681  0.35395 0.8919
## 18:9,000-14,000-20:14,000-19,000   0.36993 -0.235477  0.97535 0.6851
## 19:9,000-14,000-20:14,000-19,000   0.69975  0.206029  1.19347 0.0003
## 20:9,000-14,000-20:14,000-19,000   0.17111 -0.424505  0.76673 0.9986
## 21:9,000-14,000-20:14,000-19,000   0.60018  0.139836  1.06052 0.0014
## 18:19,000-24,000-21:14,000-19,000       NA        NA       NA     NA
## 19:19,000-24,000-21:14,000-19,000       NA        NA       NA     NA
## 20:19,000-24,000-21:14,000-19,000 -0.61574 -2.194633  0.96315 0.9806
## 21:19,000-24,000-21:14,000-19,000 -0.56416 -1.139560  0.01124 0.0605
## 18:9,000-14,000-21:14,000-19,000   0.15814 -0.287650  0.60394 0.9909
## 19:9,000-14,000-21:14,000-19,000   0.48796  0.212335  0.76358 0.0000
## 20:9,000-14,000-21:14,000-19,000  -0.04068 -0.473077  0.39172 1.0000
## 21:9,000-14,000-21:14,000-19,000   0.38839  0.178333  0.59844 0.0000
## 19:19,000-24,000-18:19,000-24,000       NA        NA       NA     NA
## 20:19,000-24,000-18:19,000-24,000       NA        NA       NA     NA
## 21:19,000-24,000-18:19,000-24,000       NA        NA       NA     NA
## 18:9,000-14,000-18:19,000-24,000        NA        NA       NA     NA
## 19:9,000-14,000-18:19,000-24,000        NA        NA       NA     NA
## 20:9,000-14,000-18:19,000-24,000        NA        NA       NA     NA
## 21:9,000-14,000-18:19,000-24,000        NA        NA       NA     NA
## 20:19,000-24,000-19:19,000-24,000       NA        NA       NA     NA
## 21:19,000-24,000-19:19,000-24,000       NA        NA       NA     NA
## 18:9,000-14,000-19:19,000-24,000        NA        NA       NA     NA
## 19:9,000-14,000-19:19,000-24,000        NA        NA       NA     NA
## 20:9,000-14,000-19:19,000-24,000        NA        NA       NA     NA
## 21:9,000-14,000-19:19,000-24,000        NA        NA       NA     NA
## 21:19,000-24,000-20:19,000-24,000  0.05158 -1.615589  1.71876 1.0000
## 18:9,000-14,000-20:19,000-24,000   0.77389 -0.853109  2.40088 0.9199
## 19:9,000-14,000-20:19,000-24,000   1.10370 -0.485120  2.69252 0.4879
## 20:9,000-14,000-20:19,000-24,000   0.57506 -1.048313  2.19844 0.9911
## 21:9,000-14,000-20:19,000-24,000   1.00413 -0.574638  2.58289 0.6274
## 18:9,000-14,000-21:19,000-24,000   0.72230  0.025664  1.41894 0.0344
## 19:9,000-14,000-21:19,000-24,000   1.05212  0.450003  1.65423 0.0000
## 20:9,000-14,000-21:19,000-24,000   0.52348 -0.164663  1.21162 0.3408
## 21:9,000-14,000-21:19,000-24,000   0.95254  0.377484  1.52760 0.0000
## 19:9,000-14,000-18:9,000-14,000    0.32981 -0.149961  0.80959 0.5050
## 20:9,000-14,000-18:9,000-14,000   -0.19882 -0.782933  0.38529 0.9936
## 21:9,000-14,000-18:9,000-14,000    0.23024 -0.215110  0.67559 0.8662
## 20:9,000-14,000-19:9,000-14,000   -0.52864 -0.995990 -0.06128 0.0122
## 21:9,000-14,000-19:9,000-14,000   -0.09957 -0.374478  0.17533 0.9893
## 21:9,000-14,000-20:9,000-14,000    0.42907 -0.002876  0.86101 0.0534
## 
## $`jaild:perinc`
##                                         diff      lwr      upr  p adj
## yes:14,000-19,000-no:14,000-19,000   0.60038  0.30693  0.89383 0.0000
## no:19,000-24,000-no:14,000-19,000   -0.43992 -1.06555  0.18571 0.3355
## yes:19,000-24,000-no:14,000-19,000  -0.18148 -0.87796  0.51500 0.9758
## no:9,000-14,000-no:14,000-19,000     0.44664  0.26856  0.62473 0.0000
## yes:9,000-14,000-no:14,000-19,000    0.58460  0.36904  0.80015 0.0000
## no:19,000-24,000-yes:14,000-19,000  -1.04030 -1.70668 -0.37391 0.0002
## yes:19,000-24,000-yes:14,000-19,000 -0.78186 -1.51517 -0.04855 0.0290
## no:9,000-14,000-yes:14,000-19,000   -0.15374 -0.44421  0.13673 0.6533
## yes:9,000-14,000-yes:14,000-19,000  -0.01578 -0.33062  0.29906 1.0000
## yes:19,000-24,000-no:19,000-24,000   0.25844 -0.65974  1.17661 0.9661
## no:9,000-14,000-no:19,000-24,000     0.88656  0.26232  1.51080 0.0008
## yes:9,000-14,000-no:19,000-24,000    1.02451  0.38857  1.66046 0.0001
## no:9,000-14,000-yes:19,000-24,000    0.62812 -0.06711  1.32335 0.1026
## yes:9,000-14,000-yes:19,000-24,000   0.76608  0.06032  1.47183 0.0246
## yes:9,000-14,000-no:9,000-14,000     0.13796 -0.07353  0.34944 0.4224
## 
## $`mlda:jaild:perinc`
##                                                diff       lwr        upr
## 19:no:14,000-19,000-18:no:14,000-19,000   -0.069529 -1.887533  1.7484748
## 20:no:14,000-19,000-18:no:14,000-19,000   -0.040010 -1.858014  1.7779940
## 21:no:14,000-19,000-18:no:14,000-19,000    0.113196 -1.648951  1.8753440
## 18:yes:14,000-19,000-18:no:14,000-19,000         NA        NA         NA
## 19:yes:14,000-19,000-18:no:14,000-19,000   2.294230 -0.183292  4.7717518
## 20:yes:14,000-19,000-18:no:14,000-19,000         NA        NA         NA
## 21:yes:14,000-19,000-18:no:14,000-19,000   0.537739 -1.247505  2.3229838
## 18:no:19,000-24,000-18:no:14,000-19,000          NA        NA         NA
## 19:no:19,000-24,000-18:no:14,000-19,000          NA        NA         NA
## 20:no:19,000-24,000-18:no:14,000-19,000          NA        NA         NA
## 21:no:19,000-24,000-18:no:14,000-19,000   -0.353922 -2.273002  1.5651582
## 18:yes:19,000-24,000-18:no:14,000-19,000         NA        NA         NA
## 19:yes:19,000-24,000-18:no:14,000-19,000         NA        NA         NA
## 20:yes:19,000-24,000-18:no:14,000-19,000  -0.235480 -2.713002  2.2420419
## 21:yes:19,000-24,000-18:no:14,000-19,000  -0.212047 -2.234935  1.8108415
## 18:no:9,000-14,000-18:no:14,000-19,000     0.213694 -1.659136  2.0865248
## 19:no:9,000-14,000-18:no:14,000-19,000     0.509847 -1.275398  2.2950915
## 20:no:9,000-14,000-18:no:14,000-19,000     0.093863 -1.752772  1.9404991
## 21:no:9,000-14,000-18:no:14,000-19,000     0.594102 -1.168292  2.3564963
## 18:yes:9,000-14,000-18:no:14,000-19,000    0.546379 -1.326452  2.4192091
## 19:yes:9,000-14,000-18:no:14,000-19,000    1.085332 -0.709803  2.8804672
## 20:yes:9,000-14,000-18:no:14,000-19,000    0.343147 -1.549092  2.2353852
## 21:yes:9,000-14,000-18:no:14,000-19,000    0.596157 -1.184674  2.3769884
## 20:no:14,000-19,000-19:no:14,000-19,000    0.029519 -0.657622  0.7166602
## 21:no:14,000-19,000-19:no:14,000-19,000    0.182726 -0.338991  0.7044420
## 18:yes:14,000-19,000-19:no:14,000-19,000         NA        NA         NA
## 19:yes:14,000-19,000-19:no:14,000-19,000   2.363759  0.545755  4.1817631
## 20:yes:14,000-19,000-19:no:14,000-19,000         NA        NA         NA
## 21:yes:14,000-19,000-19:no:14,000-19,000   0.607268  0.012187  1.2023500
## 18:no:19,000-24,000-19:no:14,000-19,000          NA        NA         NA
## 19:no:19,000-24,000-19:no:14,000-19,000          NA        NA         NA
## 20:no:19,000-24,000-19:no:14,000-19,000          NA        NA         NA
## 21:no:19,000-24,000-19:no:14,000-19,000   -0.284393 -1.206289  0.6375035
## 18:yes:19,000-24,000-19:no:14,000-19,000         NA        NA         NA
## 19:yes:19,000-24,000-19:no:14,000-19,000         NA        NA         NA
## 20:yes:19,000-24,000-19:no:14,000-19,000  -0.165951 -1.983955  1.6520533
## 21:yes:19,000-24,000-19:no:14,000-19,000  -0.142517 -1.264614  0.9795790
## 18:no:9,000-14,000-19:no:14,000-19,000     0.283224 -0.538067  1.1045140
## 19:no:9,000-14,000-19:no:14,000-19,000     0.579376 -0.015705  1.1744577
## 20:no:9,000-14,000-19:no:14,000-19,000     0.163393 -0.596270  0.9230555
## 21:no:9,000-14,000-19:no:14,000-19,000     0.663631  0.141082  1.1861806
## 18:yes:9,000-14,000-19:no:14,000-19,000    0.615908 -0.205383  1.4371983
## 19:yes:9,000-14,000-19:no:14,000-19,000    1.154861  0.530735  1.7789880
## 20:yes:9,000-14,000-19:no:14,000-19,000    0.412676 -0.451957  1.2773093
## 21:yes:9,000-14,000-19:no:14,000-19,000    0.665687  0.083979  1.2473937
## 21:no:14,000-19,000-20:no:14,000-19,000    0.153206 -0.368510  0.6749228
## 18:yes:14,000-19,000-20:no:14,000-19,000         NA        NA         NA
## 19:yes:14,000-19,000-20:no:14,000-19,000   2.334240  0.516236  4.1522439
## 20:yes:14,000-19,000-20:no:14,000-19,000         NA        NA         NA
## 21:yes:14,000-19,000-20:no:14,000-19,000   0.577749 -0.017332  1.1728308
## 18:no:19,000-24,000-20:no:14,000-19,000          NA        NA         NA
## 19:no:19,000-24,000-20:no:14,000-19,000          NA        NA         NA
## 20:no:19,000-24,000-20:no:14,000-19,000          NA        NA         NA
## 21:no:19,000-24,000-20:no:14,000-19,000   -0.313912 -1.235808  0.6079843
## 18:yes:19,000-24,000-20:no:14,000-19,000         NA        NA         NA
## 19:yes:19,000-24,000-20:no:14,000-19,000         NA        NA         NA
## 20:yes:19,000-24,000-20:no:14,000-19,000  -0.195470 -2.013474  1.6225340
## 21:yes:19,000-24,000-20:no:14,000-19,000  -0.172037 -1.294133  0.9500598
## 18:no:9,000-14,000-20:no:14,000-19,000     0.253704 -0.567586  1.0749948
## 19:no:9,000-14,000-20:no:14,000-19,000     0.549857 -0.045225  1.1449384
## 20:no:9,000-14,000-20:no:14,000-19,000     0.133873 -0.625790  0.8935362
## 21:no:9,000-14,000-20:no:14,000-19,000     0.634112  0.111562  1.1566614
## 18:yes:9,000-14,000-20:no:14,000-19,000    0.586389 -0.234902  1.4076791
## 19:yes:9,000-14,000-20:no:14,000-19,000    1.125342  0.501215  1.7494688
## 20:yes:9,000-14,000-20:no:14,000-19,000    0.383157 -0.481477  1.2477900
## 21:yes:9,000-14,000-20:no:14,000-19,000    0.636167  0.054460  1.2178745
## 18:yes:14,000-19,000-21:no:14,000-19,000         NA        NA         NA
## 19:yes:14,000-19,000-21:no:14,000-19,000   2.181033  0.418886  3.9431809
## 20:yes:14,000-19,000-21:no:14,000-19,000         NA        NA         NA
## 21:yes:14,000-19,000-21:no:14,000-19,000   0.424543  0.031927  0.8171587
## 18:no:19,000-24,000-21:no:14,000-19,000          NA        NA         NA
## 19:no:19,000-24,000-21:no:14,000-19,000          NA        NA         NA
## 20:no:19,000-24,000-21:no:14,000-19,000          NA        NA         NA
## 21:no:19,000-24,000-21:no:14,000-19,000   -0.467118 -1.273293  0.3390565
## 18:yes:19,000-24,000-21:no:14,000-19,000         NA        NA         NA
## 19:yes:19,000-24,000-21:no:14,000-19,000         NA        NA         NA
## 20:yes:19,000-24,000-21:no:14,000-19,000  -0.348676 -2.110824  1.4134710
## 21:yes:19,000-24,000-21:no:14,000-19,000  -0.325243 -1.354381  0.7038952
## 18:no:9,000-14,000-21:no:14,000-19,000     0.100498 -0.588373  0.7893689
## 19:no:9,000-14,000-21:no:14,000-19,000     0.396650  0.004035  0.7892663
## 20:no:9,000-14,000-21:no:14,000-19,000    -0.019333 -0.633428  0.5947621
## 21:no:9,000-14,000-21:no:14,000-19,000     0.480905  0.210567  0.7512444
## 18:yes:9,000-14,000-21:no:14,000-19,000    0.433182 -0.255689  1.1220532
## 19:yes:9,000-14,000-21:no:14,000-19,000    0.972136  0.536751  1.4075198
## 20:yes:9,000-14,000-21:no:14,000-19,000    0.229950 -0.510061  0.9699611
## 21:yes:9,000-14,000-21:no:14,000-19,000    0.482961  0.110928  0.8549937
## 19:yes:14,000-19,000-18:yes:14,000-19,000        NA        NA         NA
## 20:yes:14,000-19,000-18:yes:14,000-19,000        NA        NA         NA
## 21:yes:14,000-19,000-18:yes:14,000-19,000        NA        NA         NA
## 18:no:19,000-24,000-18:yes:14,000-19,000         NA        NA         NA
## 19:no:19,000-24,000-18:yes:14,000-19,000         NA        NA         NA
## 20:no:19,000-24,000-18:yes:14,000-19,000         NA        NA         NA
## 21:no:19,000-24,000-18:yes:14,000-19,000         NA        NA         NA
## 18:yes:19,000-24,000-18:yes:14,000-19,000        NA        NA         NA
## 19:yes:19,000-24,000-18:yes:14,000-19,000        NA        NA         NA
## 20:yes:19,000-24,000-18:yes:14,000-19,000        NA        NA         NA
## 21:yes:19,000-24,000-18:yes:14,000-19,000        NA        NA         NA
## 18:no:9,000-14,000-18:yes:14,000-19,000          NA        NA         NA
## 19:no:9,000-14,000-18:yes:14,000-19,000          NA        NA         NA
## 20:no:9,000-14,000-18:yes:14,000-19,000          NA        NA         NA
## 21:no:9,000-14,000-18:yes:14,000-19,000          NA        NA         NA
## 18:yes:9,000-14,000-18:yes:14,000-19,000         NA        NA         NA
## 19:yes:9,000-14,000-18:yes:14,000-19,000         NA        NA         NA
## 20:yes:9,000-14,000-18:yes:14,000-19,000         NA        NA         NA
## 21:yes:9,000-14,000-18:yes:14,000-19,000         NA        NA         NA
## 20:yes:14,000-19,000-19:yes:14,000-19,000        NA        NA         NA
## 21:yes:14,000-19,000-19:yes:14,000-19,000 -1.756491 -3.541735  0.0287539
## 18:no:19,000-24,000-19:yes:14,000-19,000         NA        NA         NA
## 19:no:19,000-24,000-19:yes:14,000-19,000         NA        NA         NA
## 20:no:19,000-24,000-19:yes:14,000-19,000         NA        NA         NA
## 21:no:19,000-24,000-19:yes:14,000-19,000  -2.648152 -4.567232 -0.7290717
## 18:yes:19,000-24,000-19:yes:14,000-19,000        NA        NA         NA
## 19:yes:19,000-24,000-19:yes:14,000-19,000        NA        NA         NA
## 20:yes:19,000-24,000-19:yes:14,000-19,000 -2.529710 -5.007232 -0.0521880
## 21:yes:19,000-24,000-19:yes:14,000-19,000 -2.506277 -4.529165 -0.4833884
## 18:no:9,000-14,000-19:yes:14,000-19,000   -2.080536 -3.953366 -0.2077051
## 19:no:9,000-14,000-19:yes:14,000-19,000   -1.784383 -3.569627  0.0008615
## 20:no:9,000-14,000-19:yes:14,000-19,000   -2.200367 -4.047002 -0.3537308
## 21:no:9,000-14,000-19:yes:14,000-19,000   -1.700128 -3.462522  0.0622664
## 18:yes:9,000-14,000-19:yes:14,000-19,000  -1.747851 -3.620682  0.1249792
## 19:yes:9,000-14,000-19:yes:14,000-19,000  -1.208898 -3.004033  0.5862373
## 20:yes:9,000-14,000-19:yes:14,000-19,000  -1.951083 -3.843322 -0.0588447
## 21:yes:9,000-14,000-19:yes:14,000-19,000  -1.698073 -3.478904  0.0827585
## 21:yes:14,000-19,000-20:yes:14,000-19,000        NA        NA         NA
## 18:no:19,000-24,000-20:yes:14,000-19,000         NA        NA         NA
## 19:no:19,000-24,000-20:yes:14,000-19,000         NA        NA         NA
## 20:no:19,000-24,000-20:yes:14,000-19,000         NA        NA         NA
## 21:no:19,000-24,000-20:yes:14,000-19,000         NA        NA         NA
## 18:yes:19,000-24,000-20:yes:14,000-19,000        NA        NA         NA
## 19:yes:19,000-24,000-20:yes:14,000-19,000        NA        NA         NA
## 20:yes:19,000-24,000-20:yes:14,000-19,000        NA        NA         NA
## 21:yes:19,000-24,000-20:yes:14,000-19,000        NA        NA         NA
## 18:no:9,000-14,000-20:yes:14,000-19,000          NA        NA         NA
## 19:no:9,000-14,000-20:yes:14,000-19,000          NA        NA         NA
## 20:no:9,000-14,000-20:yes:14,000-19,000          NA        NA         NA
## 21:no:9,000-14,000-20:yes:14,000-19,000          NA        NA         NA
## 18:yes:9,000-14,000-20:yes:14,000-19,000         NA        NA         NA
## 19:yes:9,000-14,000-20:yes:14,000-19,000         NA        NA         NA
## 20:yes:9,000-14,000-20:yes:14,000-19,000         NA        NA         NA
## 21:yes:9,000-14,000-20:yes:14,000-19,000         NA        NA         NA
## 18:no:19,000-24,000-21:yes:14,000-19,000         NA        NA         NA
## 19:no:19,000-24,000-21:yes:14,000-19,000         NA        NA         NA
## 20:no:19,000-24,000-21:yes:14,000-19,000         NA        NA         NA
## 21:no:19,000-24,000-21:yes:14,000-19,000  -0.891661 -1.747145 -0.0361777
## 18:yes:19,000-24,000-21:yes:14,000-19,000        NA        NA         NA
## 19:yes:19,000-24,000-21:yes:14,000-19,000        NA        NA         NA
## 20:yes:19,000-24,000-21:yes:14,000-19,000 -0.773219 -2.558464  1.0120253
## 21:yes:19,000-24,000-21:yes:14,000-19,000 -0.749786 -1.817990  0.3184181
## 18:no:9,000-14,000-21:yes:14,000-19,000   -0.324045 -1.070019  0.4219292
## 19:no:9,000-14,000-21:yes:14,000-19,000   -0.027892 -0.513774  0.4579897
## 20:no:9,000-14,000-21:yes:14,000-19,000   -0.443876 -1.121406  0.2336542
## 21:no:9,000-14,000-21:yes:14,000-19,000    0.056363 -0.337360  0.4500850
## 18:yes:9,000-14,000-21:yes:14,000-19,000   0.008639 -0.737335  0.7546135
## 19:yes:9,000-14,000-21:yes:14,000-19,000   0.547593  0.026542  1.0686432
## 20:yes:9,000-14,000-21:yes:14,000-19,000  -0.194593 -0.988035  0.5988494
## 21:yes:9,000-14,000-21:yes:14,000-19,000   0.058418 -0.410989  0.5278247
## 19:no:19,000-24,000-18:no:19,000-24,000          NA        NA         NA
## 20:no:19,000-24,000-18:no:19,000-24,000          NA        NA         NA
## 21:no:19,000-24,000-18:no:19,000-24,000          NA        NA         NA
## 18:yes:19,000-24,000-18:no:19,000-24,000         NA        NA         NA
## 19:yes:19,000-24,000-18:no:19,000-24,000         NA        NA         NA
## 20:yes:19,000-24,000-18:no:19,000-24,000         NA        NA         NA
## 21:yes:19,000-24,000-18:no:19,000-24,000         NA        NA         NA
## 18:no:9,000-14,000-18:no:19,000-24,000           NA        NA         NA
## 19:no:9,000-14,000-18:no:19,000-24,000           NA        NA         NA
## 20:no:9,000-14,000-18:no:19,000-24,000           NA        NA         NA
## 21:no:9,000-14,000-18:no:19,000-24,000           NA        NA         NA
## 18:yes:9,000-14,000-18:no:19,000-24,000          NA        NA         NA
## 19:yes:9,000-14,000-18:no:19,000-24,000          NA        NA         NA
## 20:yes:9,000-14,000-18:no:19,000-24,000          NA        NA         NA
## 21:yes:9,000-14,000-18:no:19,000-24,000          NA        NA         NA
## 20:no:19,000-24,000-19:no:19,000-24,000          NA        NA         NA
## 21:no:19,000-24,000-19:no:19,000-24,000          NA        NA         NA
## 18:yes:19,000-24,000-19:no:19,000-24,000         NA        NA         NA
## 19:yes:19,000-24,000-19:no:19,000-24,000         NA        NA         NA
## 20:yes:19,000-24,000-19:no:19,000-24,000         NA        NA         NA
## 21:yes:19,000-24,000-19:no:19,000-24,000         NA        NA         NA
## 18:no:9,000-14,000-19:no:19,000-24,000           NA        NA         NA
## 19:no:9,000-14,000-19:no:19,000-24,000           NA        NA         NA
## 20:no:9,000-14,000-19:no:19,000-24,000           NA        NA         NA
## 21:no:9,000-14,000-19:no:19,000-24,000           NA        NA         NA
## 18:yes:9,000-14,000-19:no:19,000-24,000          NA        NA         NA
## 19:yes:9,000-14,000-19:no:19,000-24,000          NA        NA         NA
## 20:yes:9,000-14,000-19:no:19,000-24,000          NA        NA         NA
## 21:yes:9,000-14,000-19:no:19,000-24,000          NA        NA         NA
## 21:no:19,000-24,000-20:no:19,000-24,000          NA        NA         NA
## 18:yes:19,000-24,000-20:no:19,000-24,000         NA        NA         NA
## 19:yes:19,000-24,000-20:no:19,000-24,000         NA        NA         NA
## 20:yes:19,000-24,000-20:no:19,000-24,000         NA        NA         NA
## 21:yes:19,000-24,000-20:no:19,000-24,000         NA        NA         NA
## 18:no:9,000-14,000-20:no:19,000-24,000           NA        NA         NA
## 19:no:9,000-14,000-20:no:19,000-24,000           NA        NA         NA
## 20:no:9,000-14,000-20:no:19,000-24,000           NA        NA         NA
## 21:no:9,000-14,000-20:no:19,000-24,000           NA        NA         NA
## 18:yes:9,000-14,000-20:no:19,000-24,000          NA        NA         NA
## 19:yes:9,000-14,000-20:no:19,000-24,000          NA        NA         NA
## 20:yes:9,000-14,000-20:no:19,000-24,000          NA        NA         NA
## 21:yes:9,000-14,000-20:no:19,000-24,000          NA        NA         NA
## 18:yes:19,000-24,000-21:no:19,000-24,000         NA        NA         NA
## 19:yes:19,000-24,000-21:no:19,000-24,000         NA        NA         NA
## 20:yes:19,000-24,000-21:no:19,000-24,000   0.118442 -1.800638  2.0375222
## 21:yes:19,000-24,000-21:no:19,000-24,000   0.141875 -1.137511  1.4212621
## 18:no:9,000-14,000-21:no:19,000-24,000     0.567616 -0.458175  1.5934078
## 19:no:9,000-14,000-21:no:19,000-24,000     0.863769  0.008285  1.7192525
## 20:no:9,000-14,000-21:no:19,000-24,000     0.447785 -0.529362  1.4249331
## 21:no:9,000-14,000-21:no:19,000-24,000     0.948024  0.141310  1.7547383
## 18:yes:9,000-14,000-21:no:19,000-24,000    0.900301 -0.125491  1.9260921
## 19:yes:9,000-14,000-21:no:19,000-24,000    1.439254  0.563318  2.3151903
## 20:yes:9,000-14,000-21:no:19,000-24,000    0.697069 -0.363743  1.7578801
## 21:yes:9,000-14,000-21:no:19,000-24,000    0.950079  0.103845  1.7963141
## 19:yes:19,000-24,000-18:yes:19,000-24,000        NA        NA         NA
## 20:yes:19,000-24,000-18:yes:19,000-24,000        NA        NA         NA
## 21:yes:19,000-24,000-18:yes:19,000-24,000        NA        NA         NA
## 18:no:9,000-14,000-18:yes:19,000-24,000          NA        NA         NA
## 19:no:9,000-14,000-18:yes:19,000-24,000          NA        NA         NA
## 20:no:9,000-14,000-18:yes:19,000-24,000          NA        NA         NA
## 21:no:9,000-14,000-18:yes:19,000-24,000          NA        NA         NA
## 18:yes:9,000-14,000-18:yes:19,000-24,000         NA        NA         NA
## 19:yes:9,000-14,000-18:yes:19,000-24,000         NA        NA         NA
## 20:yes:9,000-14,000-18:yes:19,000-24,000         NA        NA         NA
## 21:yes:9,000-14,000-18:yes:19,000-24,000         NA        NA         NA
## 20:yes:19,000-24,000-19:yes:19,000-24,000        NA        NA         NA
## 21:yes:19,000-24,000-19:yes:19,000-24,000        NA        NA         NA
## 18:no:9,000-14,000-19:yes:19,000-24,000          NA        NA         NA
## 19:no:9,000-14,000-19:yes:19,000-24,000          NA        NA         NA
## 20:no:9,000-14,000-19:yes:19,000-24,000          NA        NA         NA
## 21:no:9,000-14,000-19:yes:19,000-24,000          NA        NA         NA
## 18:yes:9,000-14,000-19:yes:19,000-24,000         NA        NA         NA
## 19:yes:9,000-14,000-19:yes:19,000-24,000         NA        NA         NA
## 20:yes:9,000-14,000-19:yes:19,000-24,000         NA        NA         NA
## 21:yes:9,000-14,000-19:yes:19,000-24,000         NA        NA         NA
## 21:yes:19,000-24,000-20:yes:19,000-24,000  0.023433 -1.999455  2.0463215
## 18:no:9,000-14,000-20:yes:19,000-24,000    0.449174 -1.423656  2.3220048
## 19:no:9,000-14,000-20:yes:19,000-24,000    0.745327 -1.039918  2.5305714
## 20:no:9,000-14,000-20:yes:19,000-24,000    0.329343 -1.517292  2.1759791
## 21:no:9,000-14,000-20:yes:19,000-24,000    0.829582 -0.932812  2.5919763
## 18:yes:9,000-14,000-20:yes:19,000-24,000   0.781859 -1.090972  2.6546891
## 19:yes:9,000-14,000-20:yes:19,000-24,000   1.320812 -0.474323  3.1159472
## 20:yes:9,000-14,000-20:yes:19,000-24,000   0.578627 -1.313612  2.4708652
## 21:yes:9,000-14,000-20:yes:19,000-24,000   0.831637 -0.949194  2.6124684
## 18:no:9,000-14,000-21:yes:19,000-24,000    0.425741 -0.783166  1.6346478
## 19:no:9,000-14,000-21:yes:19,000-24,000    0.721894 -0.346310  1.7900976
## 20:no:9,000-14,000-21:yes:19,000-24,000    0.305910 -0.862005  1.4738250
## 21:no:9,000-14,000-21:yes:19,000-24,000    0.806149 -0.223412  1.8357095
## 18:yes:9,000-14,000-21:yes:19,000-24,000   0.758425 -0.450482  1.9673321
## 19:yes:9,000-14,000-21:yes:19,000-24,000   1.297379  0.212726  2.3820316
## 20:yes:9,000-14,000-21:yes:19,000-24,000   0.555193 -0.683568  1.7939542
## 21:yes:9,000-14,000-21:yes:19,000-24,000   0.808204 -0.252607  1.8690155
## 19:no:9,000-14,000-18:no:9,000-14,000      0.296153 -0.449821  1.0421268
## 20:no:9,000-14,000-18:no:9,000-14,000     -0.119831 -1.002692  0.7630298
## 21:no:9,000-14,000-18:no:9,000-14,000      0.380408 -0.309095  1.0699099
## 18:yes:9,000-14,000-18:no:9,000-14,000     0.332684 -0.603731  1.2690995
## 19:yes:9,000-14,000-18:no:9,000-14,000     0.871638  0.102294  1.6409814
## 20:yes:9,000-14,000-18:no:9,000-14,000     0.129452 -0.845200  1.1041042
## 21:yes:9,000-14,000-18:no:9,000-14,000     0.382463 -0.352886  1.1178124
## 20:no:9,000-14,000-19:no:9,000-14,000     -0.415984 -1.093514  0.2615465
## 21:no:9,000-14,000-19:no:9,000-14,000      0.084255 -0.309467  0.4779773
## 18:yes:9,000-14,000-19:no:9,000-14,000     0.036532 -0.709442  0.7825058
## 19:yes:9,000-14,000-19:no:9,000-14,000     0.575485  0.054435  1.0965355
## 20:yes:9,000-14,000-19:no:9,000-14,000    -0.166700 -0.960142  0.6267417
## 21:yes:9,000-14,000-19:no:9,000-14,000     0.086310 -0.383096  0.5557170
## 21:no:9,000-14,000-20:no:9,000-14,000      0.500239 -0.114565  1.1150418
## 18:yes:9,000-14,000-20:no:9,000-14,000     0.452515 -0.430346  1.3353760
## 19:yes:9,000-14,000-20:no:9,000-14,000     0.991469  0.288291  1.6946466
## 20:yes:9,000-14,000-20:no:9,000-14,000     0.249283 -0.674035  1.1726012
## 21:yes:9,000-14,000-20:no:9,000-14,000     0.502294 -0.163520  1.1681080
## 18:yes:9,000-14,000-21:no:9,000-14,000    -0.047723 -0.737226  0.6417789
## 19:yes:9,000-14,000-21:no:9,000-14,000     0.491230  0.054848  0.9276123
## 20:yes:9,000-14,000-21:no:9,000-14,000    -0.250955 -0.991554  0.4896432
## 21:yes:9,000-14,000-21:no:9,000-14,000     0.002055 -0.371145  0.3752558
## 19:yes:9,000-14,000-18:yes:9,000-14,000    0.538953 -0.230390  1.3082971
## 20:yes:9,000-14,000-18:yes:9,000-14,000   -0.203232 -1.177884  0.7714199
## 21:yes:9,000-14,000-18:yes:9,000-14,000    0.049779 -0.685571  0.7851281
## 20:yes:9,000-14,000-19:yes:9,000-14,000   -0.742185 -1.557638  0.0732669
## 21:yes:9,000-14,000-19:yes:9,000-14,000   -0.489175 -0.994897  0.0165473
## 21:yes:9,000-14,000-20:yes:9,000-14,000    0.253011 -0.530451  1.0364719
##                                            p adj
## 19:no:14,000-19,000-18:no:14,000-19,000   1.0000
## 20:no:14,000-19,000-18:no:14,000-19,000   1.0000
## 21:no:14,000-19,000-18:no:14,000-19,000   1.0000
## 18:yes:14,000-19,000-18:no:14,000-19,000      NA
## 19:yes:14,000-19,000-18:no:14,000-19,000  0.1130
## 20:yes:14,000-19,000-18:no:14,000-19,000      NA
## 21:yes:14,000-19,000-18:no:14,000-19,000  1.0000
## 18:no:19,000-24,000-18:no:14,000-19,000       NA
## 19:no:19,000-24,000-18:no:14,000-19,000       NA
## 20:no:19,000-24,000-18:no:14,000-19,000       NA
## 21:no:19,000-24,000-18:no:14,000-19,000   1.0000
## 18:yes:19,000-24,000-18:no:14,000-19,000      NA
## 19:yes:19,000-24,000-18:no:14,000-19,000      NA
## 20:yes:19,000-24,000-18:no:14,000-19,000  1.0000
## 21:yes:19,000-24,000-18:no:14,000-19,000  1.0000
## 18:no:9,000-14,000-18:no:14,000-19,000    1.0000
## 19:no:9,000-14,000-18:no:14,000-19,000    1.0000
## 20:no:9,000-14,000-18:no:14,000-19,000    1.0000
## 21:no:9,000-14,000-18:no:14,000-19,000    0.9999
## 18:yes:9,000-14,000-18:no:14,000-19,000   1.0000
## 19:yes:9,000-14,000-18:no:14,000-19,000   0.8580
## 20:yes:9,000-14,000-18:no:14,000-19,000   1.0000
## 21:yes:9,000-14,000-18:no:14,000-19,000   0.9999
## 20:no:14,000-19,000-19:no:14,000-19,000   1.0000
## 21:no:14,000-19,000-19:no:14,000-19,000   0.9999
## 18:yes:14,000-19,000-19:no:14,000-19,000      NA
## 19:yes:14,000-19,000-19:no:14,000-19,000  0.0007
## 20:yes:14,000-19,000-19:no:14,000-19,000      NA
## 21:yes:14,000-19,000-19:no:14,000-19,000  0.0391
## 18:no:19,000-24,000-19:no:14,000-19,000       NA
## 19:no:19,000-24,000-19:no:14,000-19,000       NA
## 20:no:19,000-24,000-19:no:14,000-19,000       NA
## 21:no:19,000-24,000-19:no:14,000-19,000   1.0000
## 18:yes:19,000-24,000-19:no:14,000-19,000      NA
## 19:yes:19,000-24,000-19:no:14,000-19,000      NA
## 20:yes:19,000-24,000-19:no:14,000-19,000  1.0000
## 21:yes:19,000-24,000-19:no:14,000-19,000  1.0000
## 18:no:9,000-14,000-19:no:14,000-19,000    0.9999
## 19:no:9,000-14,000-19:no:14,000-19,000    0.0677
## 20:no:9,000-14,000-19:no:14,000-19,000    1.0000
## 21:no:9,000-14,000-19:no:14,000-19,000    0.0011
## 18:yes:9,000-14,000-19:no:14,000-19,000   0.4780
## 19:yes:9,000-14,000-19:no:14,000-19,000   0.0000
## 20:yes:9,000-14,000-19:no:14,000-19,000   0.9873
## 21:yes:9,000-14,000-19:no:14,000-19,000   0.0076
## 21:no:14,000-19,000-20:no:14,000-19,000   1.0000
## 18:yes:14,000-19,000-20:no:14,000-19,000      NA
## 19:yes:14,000-19,000-20:no:14,000-19,000  0.0009
## 20:yes:14,000-19,000-20:no:14,000-19,000      NA
## 21:yes:14,000-19,000-20:no:14,000-19,000  0.0699
## 18:no:19,000-24,000-20:no:14,000-19,000       NA
## 19:no:19,000-24,000-20:no:14,000-19,000       NA
## 20:no:19,000-24,000-20:no:14,000-19,000       NA
## 21:no:19,000-24,000-20:no:14,000-19,000   0.9999
## 18:yes:19,000-24,000-20:no:14,000-19,000      NA
## 19:yes:19,000-24,000-20:no:14,000-19,000      NA
## 20:yes:19,000-24,000-20:no:14,000-19,000  1.0000
## 21:yes:19,000-24,000-20:no:14,000-19,000  1.0000
## 18:no:9,000-14,000-20:no:14,000-19,000    1.0000
## 19:no:9,000-14,000-20:no:14,000-19,000    0.1154
## 20:no:9,000-14,000-20:no:14,000-19,000    1.0000
## 21:no:9,000-14,000-20:no:14,000-19,000    0.0027
## 18:yes:9,000-14,000-20:no:14,000-19,000   0.5818
## 19:yes:9,000-14,000-20:no:14,000-19,000   0.0000
## 20:yes:9,000-14,000-20:no:14,000-19,000   0.9951
## 21:yes:9,000-14,000-20:no:14,000-19,000   0.0153
## 18:yes:14,000-19,000-21:no:14,000-19,000      NA
## 19:yes:14,000-19,000-21:no:14,000-19,000  0.0019
## 20:yes:14,000-19,000-21:no:14,000-19,000      NA
## 21:yes:14,000-19,000-21:no:14,000-19,000  0.0181
## 18:no:19,000-24,000-21:no:14,000-19,000       NA
## 19:no:19,000-24,000-21:no:14,000-19,000       NA
## 20:no:19,000-24,000-21:no:14,000-19,000       NA
## 21:no:19,000-24,000-21:no:14,000-19,000   0.9009
## 18:yes:19,000-24,000-21:no:14,000-19,000      NA
## 19:yes:19,000-24,000-21:no:14,000-19,000      NA
## 20:yes:19,000-24,000-21:no:14,000-19,000  1.0000
## 21:yes:19,000-24,000-21:no:14,000-19,000  1.0000
## 18:no:9,000-14,000-21:no:14,000-19,000    1.0000
## 19:no:9,000-14,000-21:no:14,000-19,000    0.0443
## 20:no:9,000-14,000-21:no:14,000-19,000    1.0000
## 21:no:9,000-14,000-21:no:14,000-19,000    0.0000
## 18:yes:9,000-14,000-21:no:14,000-19,000   0.8079
## 19:yes:9,000-14,000-21:no:14,000-19,000   0.0000
## 20:yes:9,000-14,000-21:no:14,000-19,000   1.0000
## 21:yes:9,000-14,000-21:no:14,000-19,000   0.0007
## 19:yes:14,000-19,000-18:yes:14,000-19,000     NA
## 20:yes:14,000-19,000-18:yes:14,000-19,000     NA
## 21:yes:14,000-19,000-18:yes:14,000-19,000     NA
## 18:no:19,000-24,000-18:yes:14,000-19,000      NA
## 19:no:19,000-24,000-18:yes:14,000-19,000      NA
## 20:no:19,000-24,000-18:yes:14,000-19,000      NA
## 21:no:19,000-24,000-18:yes:14,000-19,000      NA
## 18:yes:19,000-24,000-18:yes:14,000-19,000     NA
## 19:yes:19,000-24,000-18:yes:14,000-19,000     NA
## 20:yes:19,000-24,000-18:yes:14,000-19,000     NA
## 21:yes:19,000-24,000-18:yes:14,000-19,000     NA
## 18:no:9,000-14,000-18:yes:14,000-19,000       NA
## 19:no:9,000-14,000-18:yes:14,000-19,000       NA
## 20:no:9,000-14,000-18:yes:14,000-19,000       NA
## 21:no:9,000-14,000-18:yes:14,000-19,000       NA
## 18:yes:9,000-14,000-18:yes:14,000-19,000      NA
## 19:yes:9,000-14,000-18:yes:14,000-19,000      NA
## 20:yes:9,000-14,000-18:yes:14,000-19,000      NA
## 21:yes:9,000-14,000-18:yes:14,000-19,000      NA
## 20:yes:14,000-19,000-19:yes:14,000-19,000     NA
## 21:yes:14,000-19,000-19:yes:14,000-19,000 0.0603
## 18:no:19,000-24,000-19:yes:14,000-19,000      NA
## 19:no:19,000-24,000-19:yes:14,000-19,000      NA
## 20:no:19,000-24,000-19:yes:14,000-19,000      NA
## 21:no:19,000-24,000-19:yes:14,000-19,000  0.0002
## 18:yes:19,000-24,000-19:yes:14,000-19,000     NA
## 19:yes:19,000-24,000-19:yes:14,000-19,000     NA
## 20:yes:19,000-24,000-19:yes:14,000-19,000 0.0389
## 21:yes:19,000-24,000-19:yes:14,000-19,000 0.0018
## 18:no:9,000-14,000-19:yes:14,000-19,000   0.0121
## 19:no:9,000-14,000-19:yes:14,000-19,000   0.0503
## 20:no:9,000-14,000-19:yes:14,000-19,000   0.0038
## 21:no:9,000-14,000-19:yes:14,000-19,000   0.0748
## 18:yes:9,000-14,000-19:yes:14,000-19,000  0.1049
## 19:yes:9,000-14,000-19:yes:14,000-19,000  0.6966
## 20:yes:9,000-14,000-19:yes:14,000-19,000  0.0344
## 21:yes:9,000-14,000-19:yes:14,000-19,000  0.0846
## 21:yes:14,000-19,000-20:yes:14,000-19,000     NA
## 18:no:19,000-24,000-20:yes:14,000-19,000      NA
## 19:no:19,000-24,000-20:yes:14,000-19,000      NA
## 20:no:19,000-24,000-20:yes:14,000-19,000      NA
## 21:no:19,000-24,000-20:yes:14,000-19,000      NA
## 18:yes:19,000-24,000-20:yes:14,000-19,000     NA
## 19:yes:19,000-24,000-20:yes:14,000-19,000     NA
## 20:yes:19,000-24,000-20:yes:14,000-19,000     NA
## 21:yes:19,000-24,000-20:yes:14,000-19,000     NA
## 18:no:9,000-14,000-20:yes:14,000-19,000       NA
## 19:no:9,000-14,000-20:yes:14,000-19,000       NA
## 20:no:9,000-14,000-20:yes:14,000-19,000       NA
## 21:no:9,000-14,000-20:yes:14,000-19,000       NA
## 18:yes:9,000-14,000-20:yes:14,000-19,000      NA
## 19:yes:9,000-14,000-20:yes:14,000-19,000      NA
## 20:yes:9,000-14,000-20:yes:14,000-19,000      NA
## 21:yes:9,000-14,000-20:yes:14,000-19,000      NA
## 18:no:19,000-24,000-21:yes:14,000-19,000      NA
## 19:no:19,000-24,000-21:yes:14,000-19,000      NA
## 20:no:19,000-24,000-21:yes:14,000-19,000      NA
## 21:no:19,000-24,000-21:yes:14,000-19,000  0.0299
## 18:yes:19,000-24,000-21:yes:14,000-19,000     NA
## 19:yes:19,000-24,000-21:yes:14,000-19,000     NA
## 20:yes:19,000-24,000-21:yes:14,000-19,000 0.9964
## 21:yes:19,000-24,000-21:yes:14,000-19,000 0.6167
## 18:no:9,000-14,000-21:yes:14,000-19,000   0.9963
## 19:no:9,000-14,000-21:yes:14,000-19,000   1.0000
## 20:no:9,000-14,000-21:yes:14,000-19,000   0.7448
## 21:no:9,000-14,000-21:yes:14,000-19,000   1.0000
## 18:yes:9,000-14,000-21:yes:14,000-19,000  1.0000
## 19:yes:9,000-14,000-21:yes:14,000-19,000  0.0268
## 20:yes:9,000-14,000-21:yes:14,000-19,000  1.0000
## 21:yes:9,000-14,000-21:yes:14,000-19,000  1.0000
## 19:no:19,000-24,000-18:no:19,000-24,000       NA
## 20:no:19,000-24,000-18:no:19,000-24,000       NA
## 21:no:19,000-24,000-18:no:19,000-24,000       NA
## 18:yes:19,000-24,000-18:no:19,000-24,000      NA
## 19:yes:19,000-24,000-18:no:19,000-24,000      NA
## 20:yes:19,000-24,000-18:no:19,000-24,000      NA
## 21:yes:19,000-24,000-18:no:19,000-24,000      NA
## 18:no:9,000-14,000-18:no:19,000-24,000        NA
## 19:no:9,000-14,000-18:no:19,000-24,000        NA
## 20:no:9,000-14,000-18:no:19,000-24,000        NA
## 21:no:9,000-14,000-18:no:19,000-24,000        NA
## 18:yes:9,000-14,000-18:no:19,000-24,000       NA
## 19:yes:9,000-14,000-18:no:19,000-24,000       NA
## 20:yes:9,000-14,000-18:no:19,000-24,000       NA
## 21:yes:9,000-14,000-18:no:19,000-24,000       NA
## 20:no:19,000-24,000-19:no:19,000-24,000       NA
## 21:no:19,000-24,000-19:no:19,000-24,000       NA
## 18:yes:19,000-24,000-19:no:19,000-24,000      NA
## 19:yes:19,000-24,000-19:no:19,000-24,000      NA
## 20:yes:19,000-24,000-19:no:19,000-24,000      NA
## 21:yes:19,000-24,000-19:no:19,000-24,000      NA
## 18:no:9,000-14,000-19:no:19,000-24,000        NA
## 19:no:9,000-14,000-19:no:19,000-24,000        NA
## 20:no:9,000-14,000-19:no:19,000-24,000        NA
## 21:no:9,000-14,000-19:no:19,000-24,000        NA
## 18:yes:9,000-14,000-19:no:19,000-24,000       NA
## 19:yes:9,000-14,000-19:no:19,000-24,000       NA
## 20:yes:9,000-14,000-19:no:19,000-24,000       NA
## 21:yes:9,000-14,000-19:no:19,000-24,000       NA
## 21:no:19,000-24,000-20:no:19,000-24,000       NA
## 18:yes:19,000-24,000-20:no:19,000-24,000      NA
## 19:yes:19,000-24,000-20:no:19,000-24,000      NA
## 20:yes:19,000-24,000-20:no:19,000-24,000      NA
## 21:yes:19,000-24,000-20:no:19,000-24,000      NA
## 18:no:9,000-14,000-20:no:19,000-24,000        NA
## 19:no:9,000-14,000-20:no:19,000-24,000        NA
## 20:no:9,000-14,000-20:no:19,000-24,000        NA
## 21:no:9,000-14,000-20:no:19,000-24,000        NA
## 18:yes:9,000-14,000-20:no:19,000-24,000       NA
## 19:yes:9,000-14,000-20:no:19,000-24,000       NA
## 20:yes:9,000-14,000-20:no:19,000-24,000       NA
## 21:yes:9,000-14,000-20:no:19,000-24,000       NA
## 18:yes:19,000-24,000-21:no:19,000-24,000      NA
## 19:yes:19,000-24,000-21:no:19,000-24,000      NA
## 20:yes:19,000-24,000-21:no:19,000-24,000  1.0000
## 21:yes:19,000-24,000-21:no:19,000-24,000  1.0000
## 18:no:9,000-14,000-21:no:19,000-24,000    0.9354
## 19:no:9,000-14,000-21:no:19,000-24,000    0.0446
## 20:no:9,000-14,000-21:no:19,000-24,000    0.9924
## 21:no:9,000-14,000-21:no:19,000-24,000    0.0049
## 18:yes:9,000-14,000-21:no:19,000-24,000   0.1809
## 19:yes:9,000-14,000-21:no:19,000-24,000   0.0000
## 20:yes:9,000-14,000-21:no:19,000-24,000   0.7397
## 21:yes:9,000-14,000-21:no:19,000-24,000   0.0103
## 19:yes:19,000-24,000-18:yes:19,000-24,000     NA
## 20:yes:19,000-24,000-18:yes:19,000-24,000     NA
## 21:yes:19,000-24,000-18:yes:19,000-24,000     NA
## 18:no:9,000-14,000-18:yes:19,000-24,000       NA
## 19:no:9,000-14,000-18:yes:19,000-24,000       NA
## 20:no:9,000-14,000-18:yes:19,000-24,000       NA
## 21:no:9,000-14,000-18:yes:19,000-24,000       NA
## 18:yes:9,000-14,000-18:yes:19,000-24,000      NA
## 19:yes:9,000-14,000-18:yes:19,000-24,000      NA
## 20:yes:9,000-14,000-18:yes:19,000-24,000      NA
## 21:yes:9,000-14,000-18:yes:19,000-24,000      NA
## 20:yes:19,000-24,000-19:yes:19,000-24,000     NA
## 21:yes:19,000-24,000-19:yes:19,000-24,000     NA
## 18:no:9,000-14,000-19:yes:19,000-24,000       NA
## 19:no:9,000-14,000-19:yes:19,000-24,000       NA
## 20:no:9,000-14,000-19:yes:19,000-24,000       NA
## 21:no:9,000-14,000-19:yes:19,000-24,000       NA
## 18:yes:9,000-14,000-19:yes:19,000-24,000      NA
## 19:yes:9,000-14,000-19:yes:19,000-24,000      NA
## 20:yes:9,000-14,000-19:yes:19,000-24,000      NA
## 21:yes:9,000-14,000-19:yes:19,000-24,000      NA
## 21:yes:19,000-24,000-20:yes:19,000-24,000 1.0000
## 18:no:9,000-14,000-20:yes:19,000-24,000   1.0000
## 19:no:9,000-14,000-20:yes:19,000-24,000   0.9979
## 20:no:9,000-14,000-20:yes:19,000-24,000   1.0000
## 21:no:9,000-14,000-20:yes:19,000-24,000   0.9893
## 18:yes:9,000-14,000-20:yes:19,000-24,000  0.9979
## 19:yes:9,000-14,000-20:yes:19,000-24,000  0.5186
## 20:yes:9,000-14,000-20:yes:19,000-24,000  1.0000
## 21:yes:9,000-14,000-20:yes:19,000-24,000  0.9904
## 18:no:9,000-14,000-21:yes:19,000-24,000   0.9998
## 19:no:9,000-14,000-21:yes:19,000-24,000   0.6901
## 20:no:9,000-14,000-21:yes:19,000-24,000   1.0000
## 21:no:9,000-14,000-21:yes:19,000-24,000   0.3869
## 18:yes:9,000-14,000-21:yes:19,000-24,000  0.8111
## 19:yes:9,000-14,000-21:yes:19,000-24,000  0.0036
## 20:yes:9,000-14,000-21:yes:19,000-24,000  0.9943
## 21:yes:9,000-14,000-21:yes:19,000-24,000  0.4443
## 19:no:9,000-14,000-18:no:9,000-14,000     0.9990
## 20:no:9,000-14,000-18:no:9,000-14,000     1.0000
## 21:no:9,000-14,000-18:no:9,000-14,000     0.9372
## 18:yes:9,000-14,000-18:no:9,000-14,000    0.9998
## 19:yes:9,000-14,000-18:no:9,000-14,000    0.0089
## 20:yes:9,000-14,000-18:no:9,000-14,000    1.0000
## 21:yes:9,000-14,000-18:no:9,000-14,000    0.9658
## 20:no:9,000-14,000-19:no:9,000-14,000     0.8396
## 21:no:9,000-14,000-19:no:9,000-14,000     1.0000
## 18:yes:9,000-14,000-19:no:9,000-14,000    1.0000
## 19:yes:9,000-14,000-19:no:9,000-14,000    0.0133
## 20:yes:9,000-14,000-19:no:9,000-14,000    1.0000
## 21:yes:9,000-14,000-19:no:9,000-14,000    1.0000
## 21:no:9,000-14,000-20:no:9,000-14,000     0.3100
## 18:yes:9,000-14,000-20:no:9,000-14,000    0.9709
## 19:yes:9,000-14,000-20:no:9,000-14,000    0.0001
## 20:yes:9,000-14,000-20:no:9,000-14,000    1.0000
## 21:yes:9,000-14,000-20:no:9,000-14,000    0.4653
## 18:yes:9,000-14,000-21:no:9,000-14,000    1.0000
## 19:yes:9,000-14,000-21:no:9,000-14,000    0.0099
## 20:yes:9,000-14,000-21:no:9,000-14,000    0.9999
## 21:yes:9,000-14,000-21:no:9,000-14,000    1.0000
## 19:yes:9,000-14,000-18:yes:9,000-14,000   0.6206
## 20:yes:9,000-14,000-18:yes:9,000-14,000   1.0000
## 21:yes:9,000-14,000-18:yes:9,000-14,000   1.0000
## 20:yes:9,000-14,000-19:yes:9,000-14,000   0.1327
## 21:yes:9,000-14,000-19:yes:9,000-14,000   0.0727
## 21:yes:9,000-14,000-20:yes:9,000-14,000   1.0000

We also use an interaction plot to visualize the interaction of the factors on the response variable.

# Interaction Plot of Age and Jail Time factors
interaction.plot(data$mlda,data$jaild,data$mrall)

plot of chunk unnamed-chunk-7

# Interaction Plot of Age and Income factors
interaction.plot(data$mlda,data$perinc,data$mrall)

plot of chunk unnamed-chunk-7

# Interaction Plot of Jail Time and Income factors
interaction.plot(data$jaild,data$perinc,data$mrall)

plot of chunk unnamed-chunk-7

We can see somewhat parrallelism and no intersections in each of our plots which implies there is no interaction effect.

Lastly we plot a fitted model against the residuals. We do not see a very large degree of variation among the plot.

#Plot of Fitted vs Residuals of the model with all factors
plot(fitted(model123), residuals(model123))

plot of chunk unnamed-chunk-8

Overrall the results of our models and our adequacy tests lead us to accept that each factor has an effect on the response variable. The results of our interaction tests lead us to reject the results of the test that indicate that certain interaction effects do have an effect on the response variable.

4. References to the literature

No literature was used

5. Appendices

The R package in which this data was found can be located at http://cran.r-project.org/web/packages/Ecdat/index.html

6. Contingencies

It is possible that the conclusions of our analysis are the results of chance. One concern is that the data is collected over several years. There are a large number of factors that fluctuate year to year that could effect the traffic fatality rate. In addition effects of factors like a mandatory jail sentence could be misleading since a having a harsh versus a a light sentence could have different effects.

Different states also have many different types and amounts of roadways. It would be raitonal to hypothesize that a more rural state, with less large cities and areas with a large number of roads, despite different laws, drinking ages and income would have less traffic fatalities in general.