Example of p-values from step and p-values from re-running the suggested final model

Trails A Regression Model 1

## Loading required package: Matrix
## KernSmooth 2.23 loaded
## Copyright M. P. Wand 1997-2009
## 
## Attaching package: 'lmerTest'
## 
## The following object is masked from 'package:lme4':
## 
##     lmer
## 
## The following object is masked from 'package:stats':
## 
##     step
## Warning: number of observations <= rank(Z); variance-covariance matrix
## will be unidentifiable
## Linear mixed model fit by maximum likelihood ['merModLmerTest']
## Formula: TrailsAtestSec ~ (Age + IPVstatus + Sex + Race)^4 + (Age | HNDid) +      (1 | subclass) 
##    Data: IPVandCognitionDataSet2 
##      AIC      BIC   logLik deviance 
##    962.6   1022.1   -460.3    920.6 
## Random effects:
##  Groups   Name        Std.Dev. Corr
##  HNDid    (Intercept) 7.04         
##           Age         0.20     1.00
##  subclass (Intercept) 1.85         
##  Residual             7.63         
## Number of obs: 126, groups: HNDid, 63; subclass, 21
## Fixed Effects:
##                     (Intercept)                              Age  
##                         29.0401                           0.3065  
##                      IPVstatus1                           SexMen  
##                         -0.0520                          -4.7692  
##                       RaceAfrAm                   Age:IPVstatus1  
##                         15.0198                          -0.0764  
##                      Age:SexMen                    Age:RaceAfrAm  
##                         -0.4153                           0.7334  
##               IPVstatus1:SexMen             IPVstatus1:RaceAfrAm  
##                         17.5731                           1.5937  
##                SexMen:RaceAfrAm            Age:IPVstatus1:SexMen  
##                         -2.5314                           0.9969  
##        Age:IPVstatus1:RaceAfrAm             Age:SexMen:RaceAfrAm  
##                          0.6192                           0.0895  
##     IPVstatus1:SexMen:RaceAfrAm  Age:IPVstatus1:SexMen:RaceAfrAm  
##                         -9.8200                          -1.0854
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Warning: 
##  model has been refitted with REML=TRUE 
## 
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## 
## Random effects:
##                         Chi.sq Chi.DF elim.num p.value
## (1 | subclass)            0.11      1        1  0.7382
## (Age | HNDid)             0.91      1        2  0.3398
##       (Age + 0 | HNDid)   0.00      1        3  1.0000
##       (1 | HNDid)        13.54      1     kept  0.0002
## 
## Fixed effects:
##                          Sum Sq  Mean Sq NumDF  DenDF F.value elim.num
## Age:IPVstatus:Sex:Race   32.410   32.410     1  97.30  0.5255        1
## IPVstatus:Sex:Race        1.375    1.375     1  52.92  0.0224        2
## Age:IPVstatus:Race        1.623    1.623     1  99.08  0.0320        3
## Age:Sex:Race              2.552    2.552     1  99.49  0.0331        4
## Age:IPVstatus:Sex        13.846   13.846     1  95.84  0.2420        5
## Age:Sex                  46.681   46.681     1 102.62  0.2731        6
## IPVstatus:Race           30.428   30.428     1  54.17  0.5885        7
## Sex:Race                 35.710   35.710     1  56.47  0.5556        8
## Age:IPVstatus           157.114  157.114     1  96.15  2.8413        9
## Age                    1855.248 1855.248     1 102.33 16.1443     kept
## IPVstatus                97.756   97.756     1  56.67  2.0191     kept
## Sex                      14.223   14.223     1  57.11  0.0285     kept
## Race                    433.600  433.600     1  81.23 10.8756     kept
## Age:Race                229.423  229.423     1 103.84  4.6166     kept
## IPVstatus:Sex           233.415  233.415     1  57.41  4.8137     kept
##                        Pr(>F)
## Age:IPVstatus:Sex:Race 0.4702
## IPVstatus:Sex:Race     0.8815
## Age:IPVstatus:Race     0.8584
## Age:Sex:Race           0.8559
## Age:IPVstatus:Sex      0.6239
## Age:Sex                0.6024
## IPVstatus:Race         0.4463
## Sex:Race               0.4591
## Age:IPVstatus          0.0951
## Age                    0.0001
## IPVstatus              0.1608
## Sex                    0.8664
## Race                   0.0014
## Age:Race               0.0340
## IPVstatus:Sex          0.0323
## 
## Least squares means:
##                        IPVstatus  Sex Race Estimate Standard Error   DF
## IPVstatus  0                 1.0   NA   NA    30.84           1.42 57.2
## IPVstatus  1                 2.0   NA   NA    34.20           1.96 56.9
## Sex  Women                    NA  2.0   NA    32.32           1.68 57.0
## Sex  Men                      NA  1.0   NA    32.72           1.76 57.4
## Race  White                   NA   NA  2.0    29.38           1.92 57.8
## Race  AfrAm                   NA   NA  1.0    35.66           1.45 56.9
## IPVstatus:Sex  0 Women       1.0  2.0   NA    33.27           2.14 56.9
## IPVstatus:Sex  1 Women       2.0  2.0   NA    31.36           2.55 57.1
## IPVstatus:Sex  0 Men         1.0  1.0   NA    28.40           1.85 58.8
## IPVstatus:Sex  1 Men         2.0  1.0   NA    37.04           2.95 56.7
##                        t-value Lower CI Upper CI p-value
## IPVstatus  0              21.7     28.0     33.7  <2e-16
## IPVstatus  1              17.4     30.3     38.1  <2e-16
## Sex  Women                19.2     29.0     35.7  <2e-16
## Sex  Men                  18.6     29.2     36.2  <2e-16
## Race  White               15.3     25.5     33.2  <2e-16
## Race  AfrAm               24.5     32.7     38.6  <2e-16
## IPVstatus:Sex  0 Women    15.6     29.0     37.6  <2e-16
## IPVstatus:Sex  1 Women    12.3     26.3     36.5  <2e-16
## IPVstatus:Sex  0 Men      15.3     24.7     32.1  <2e-16
## IPVstatus:Sex  1 Men      12.6     31.1     42.9  <2e-16
## 
##  Differences of LSMEANS:
##                                 Estimate Standard Error   DF t-value
## IPVstatus 0-1                       -3.4          2.368 56.7   -1.42
## Sex Women-Men                       -0.4          2.385 57.1   -0.17
## Race White-AfrAm                    -6.3          2.343 57.6   -2.68
## IPVstatus:Sex  0 Women- 1 Women      1.9          3.297 57.0    0.58
## IPVstatus:Sex  0 Women- 0 Men        4.9          2.814 58.2    1.73
## IPVstatus:Sex  0 Women- 1 Men       -3.8          3.589 56.8   -1.05
## IPVstatus:Sex  1 Women- 0 Men        3.0          3.116 57.0    0.95
## IPVstatus:Sex  1 Women- 1 Men       -5.7          3.877 56.8   -1.47
## IPVstatus:Sex  0 Men- 1 Men         -8.6          3.452 57.1   -2.50
##                                 Lower CI Upper CI p-value
## IPVstatus 0-1                     -8.108     1.38    0.16
## Sex Women-Men                     -5.179     4.37    0.87
## Race White-AfrAm                 -10.969    -1.59    0.01
## IPVstatus:Sex  0 Women- 1 Women   -4.691     8.51    0.56
## IPVstatus:Sex  0 Women- 0 Men     -0.758    10.51    0.09
## IPVstatus:Sex  0 Women- 1 Men    -10.956     3.42    0.30
## IPVstatus:Sex  1 Women- 0 Men     -3.278     9.20    0.35
## IPVstatus:Sex  1 Women- 1 Men    -13.443     2.08    0.15
## IPVstatus:Sex  0 Men- 1 Men      -15.554    -1.73    0.02
## 
## Final model:
## lme4::lmer(formula = TrailsAtestSec ~ Age + IPVstatus + Sex + 
##     Race + (1 | HNDid) + Age:Race + IPVstatus:Sex, data = IPVandCognitionDataSet2, 
##     REML = reml, contrasts = l)
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Linear mixed model fit by maximum likelihood ['merModLmerTest']
## Formula: TrailsAtestSec ~ (Age + IPVstatus + Sex + Race)^4 + (Age | HNDid) +      (1 | subclass) 
##    Data: IPVandCognitionDataSet2 
## 
##      AIC      BIC   logLik deviance 
##    962.6   1022.1   -460.3    920.6 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev. Corr
##  HNDid    (Intercept) 49.5674  7.04         
##           Age          0.0402  0.20     1.00
##  subclass (Intercept)  3.4234  1.85         
##  Residual             58.1751  7.63         
## Number of obs: 126, groups: HNDid, 63; subclass, 21
## 
## Fixed effects:
##                                 Estimate Std. Error       df t value
## (Intercept)                      29.0401     5.2087  21.6000    5.58
## Age                               0.3065     0.5302  71.3000    0.58
## IPVstatus1                       -0.0520     9.0277  44.2000   -0.01
## SexMen                           -4.7692     7.5427  40.7000   -0.63
## RaceAfrAm                        15.0198     6.2488  20.9000    2.40
## Age:IPVstatus1                   -0.0764     0.8783 100.8000   -0.09
## Age:SexMen                       -0.4153     0.7335 101.0000   -0.57
## Age:RaceAfrAm                     0.7334     0.5972  66.1000    1.23
## IPVstatus1:SexMen                17.5731    12.6419  32.7000    1.39
## IPVstatus1:RaceAfrAm              1.5937    10.9080  34.9000    0.15
## SexMen:RaceAfrAm                 -2.5314     8.8160  32.9000   -0.29
## Age:IPVstatus1:SexMen             0.9969     1.1750  91.5000    0.85
## Age:IPVstatus1:RaceAfrAm          0.6192     1.0533  93.0000    0.59
## Age:SexMen:RaceAfrAm              0.0895     0.8780  88.6000    0.10
## IPVstatus1:SexMen:RaceAfrAm      -9.8200    15.0816  29.6000   -0.65
## Age:IPVstatus1:SexMen:RaceAfrAm  -1.0854     1.4366  87.7000   -0.76
##                                 Pr(>|t|)
## (Intercept)                      1.4e-05
## Age                                0.565
## IPVstatus1                         0.995
## SexMen                             0.531
## RaceAfrAm                          0.026
## Age:IPVstatus1                     0.931
## Age:SexMen                         0.573
## Age:RaceAfrAm                      0.224
## IPVstatus1:SexMen                  0.174
## IPVstatus1:RaceAfrAm               0.885
## SexMen:RaceAfrAm                   0.776
## Age:IPVstatus1:SexMen              0.398
## Age:IPVstatus1:RaceAfrAm           0.558
## Age:SexMen:RaceAfrAm               0.919
## IPVstatus1:SexMen:RaceAfrAm        0.520
## Age:IPVstatus1:SexMen:RaceAfrAm    0.452
## 
## Correlation of Fixed Effects:
##             (Intr) Age    IPVst1 SexMen RcAfrA Ag:IPV1 Ag:SxM Ag:RAA
## Age          0.719                                                  
## IPVstatus1  -0.577 -0.412                                           
## SexMen      -0.690 -0.498  0.405                                    
## RaceAfrAm   -0.833 -0.599  0.490  0.578                             
## Ag:IPVstts1 -0.436 -0.591  0.831  0.304  0.371                      
## Age:SexMen  -0.520 -0.719  0.303  0.810  0.436  0.430               
## Age:RcAfrAm -0.638 -0.888  0.372  0.443  0.727  0.530   0.639       
## IPVstts1:SM  0.423  0.304 -0.724 -0.609 -0.359 -0.601  -0.492 -0.275
## IPVstt1:RAA  0.481  0.341 -0.834 -0.336 -0.578 -0.692  -0.252 -0.416
## SxMn:RcAfrA  0.598  0.430 -0.355 -0.859 -0.716 -0.267  -0.696 -0.522
## Ag:IPVs1:SM  0.335  0.447 -0.630 -0.515 -0.288 -0.754  -0.627 -0.399
## Ag:IPV1:RAA  0.365  0.493 -0.697 -0.254 -0.417 -0.836  -0.356 -0.556
## Ag:SxMn:RAA  0.442  0.606 -0.261 -0.681 -0.504 -0.365  -0.838 -0.682
## IPV1:SM:RAA -0.357 -0.255  0.611  0.512  0.427  0.506   0.414  0.309
## A:IPV1:SM:R -0.277 -0.367  0.519  0.423  0.318  0.619   0.511  0.415
##             IPVs1:SM IPV1:R SM:RAA Ag:IPV1:SM A:IPV1:R A:SM:R IPV1:SM:
## Age                                                                   
## IPVstatus1                                                            
## SexMen                                                                
## RaceAfrAm                                                             
## Ag:IPVstts1                                                           
## Age:SexMen                                                            
## Age:RcAfrAm                                                           
## IPVstts1:SM                                                           
## IPVstt1:RAA  0.603                                                    
## SxMn:RcAfrA  0.528    0.416                                           
## Ag:IPVs1:SM  0.819    0.526  0.448                                    
## Ag:IPV1:RAA  0.503    0.818  0.299  0.628                             
## Ag:SxMn:RAA  0.419    0.290  0.773  0.528      0.381                  
## IPV1:SM:RAA -0.842   -0.730 -0.594 -0.691     -0.596   -0.460         
## A:IPV1:SM:R -0.674   -0.608 -0.481 -0.819     -0.738   -0.612  0.794

Re-run final Model 1

(mm1 = lmer(TrailsAtestSec ~ Age + IPVstatus + Sex + Race + (Age | HNDid) + 
    (1 | subclass) + Age:Race + IPVstatus:Sex, data = IPVandCognitionDataSet2, 
    REML = F))
## Warning: number of observations <= rank(Z); variance-covariance matrix
## will be unidentifiable
## Linear mixed model fit by maximum likelihood ['merModLmerTest']
## Formula: TrailsAtestSec ~ Age + IPVstatus + Sex + Race + (Age | HNDid) +      (1 | subclass) + Age:Race + IPVstatus:Sex 
##    Data: IPVandCognitionDataSet2 
##      AIC      BIC   logLik deviance 
##    948.9    983.0   -462.5    924.9 
## Random effects:
##  Groups   Name        Std.Dev. Corr
##  HNDid    (Intercept) 7.968        
##           Age         0.316    0.98
##  subclass (Intercept) 1.541        
##  Residual             7.718        
## Number of obs: 126, groups: HNDid, 63; subclass, 21
## Fixed Effects:
##       (Intercept)                Age         IPVstatus1  
##            30.976              0.220             -1.928  
##            SexMen          RaceAfrAm      Age:RaceAfrAm  
##            -4.019             11.792              0.791  
## IPVstatus1:SexMen  
##             8.927

summary(mm1)
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Warning: number of observations <= rank(Z); variance-covariance matrix will be unidentifiable
## Linear mixed model fit by maximum likelihood ['merModLmerTest']
## Formula: TrailsAtestSec ~ Age + IPVstatus + Sex + Race + (Age | HNDid) +      (1 | subclass) + Age:Race + IPVstatus:Sex 
##    Data: IPVandCognitionDataSet2 
## 
##      AIC      BIC   logLik deviance 
##    948.9    983.0   -462.5    924.9 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev. Corr
##  HNDid    (Intercept) 63.49    7.968        
##           Age          0.10    0.316    0.98
##  subclass (Intercept)  2.37    1.541        
##  Residual             59.56    7.718        
## Number of obs: 126, groups: HNDid, 63; subclass, 21
## 
## Fixed effects:
##                   Estimate Std. Error     df t value Pr(>|t|)
## (Intercept)         30.976      3.577 28.600    8.66  1.7e-09
## Age                  0.220      0.295 31.000    0.75   0.4618
## IPVstatus1          -1.928      2.959 47.200   -0.65   0.5178
## SexMen              -4.019      2.641 58.000   -1.52   0.1335
## RaceAfrAm           11.792      3.731 18.200    3.16   0.0054
## Age:RaceAfrAm        0.791      0.353 22.700    2.24   0.0353
## IPVstatus1:SexMen    8.927      4.445 54.600    2.01   0.0496
## 
## Correlation of Fixed Effects:
##             (Intr) Age    IPVst1 SexMen RcAfrA Ag:RAA
## Age          0.720                                   
## IPVstatus1  -0.351  0.050                            
## SexMen      -0.412  0.085  0.501                     
## RaceAfrAm   -0.728 -0.725 -0.014 -0.056              
## Age:RcAfrAm -0.539 -0.839 -0.105 -0.206  0.818       
## IPVstts1:SM  0.266 -0.045 -0.701 -0.611  0.015  0.119