1. Run and interpret a null model with math scores gktmathss as the DV, with students clustered within classrooms and schools. How much variation in math scores is at the teacher level? The school level? Which level demonstrates more variability? And how much variability remains at the student level?

Variation in Math Scores ICC (Classroom) = 0.1260353 ICC (School) = 0.1673605

There is 13% variation at the teacher-level, 17% variation at the school level. The school level demonstrates more variability. About 71% of the variability remains at the student level.

  1. Now, run a conditional random intercept model with math scores (gktmathss) as the DV, and self-concept (gkselfconcraw) as a student-level IV, classroom type (gkclasstype) and teacher highest degree (gkthighdegree) as teacher-level IVs, and school average math scores as a school level IV. (Hint- you will need to calculate the last one using the procedures I demonstrated in the videos). Interpret the results and evaluate model fit using AIC/BIC and your choice of effect size.

The following predictors are significant: classroom type and self-concept. The highest degree is not showing signifance at the teacher-level, although there is slight significance for specialist over other high degrees. The average math scores at the school level show to be significant predictors as well. The AIC and BIC both improve with the model as we include additional predictors.

  1. Try adding a random slope for self-concept gkselfconcraw at both the teacher and school levels. You can do this as two separate models- one with the random slope at the teacher level, and one with the random slope at the school level. What do these random slopes “mean”? Use AIC/BIC to evaluate whether the slopes should be treated as random.

Without a Random Slope the AIC is 48768, the BIC is 48839.The Random Slope (self-concept) at teacher level : AIC is 48763, BIC is 48847. The Random Slope (self-concept) at the school level the AIC is 48762, the BIC is 48846.

Adding the random slopes allows variability for a given predictor across a higher-level clustering variable. It allows the direction of the predictor to change.Adding the random slope at the school level decreased the AIC, but not BIC. We should not include the random slope ‘gkselfconcraw’.

Load Some Packages to Help with the Analysis and Data Management:

suppressPackageStartupMessages(library(tidyverse))
suppressPackageStartupMessages(library(lme4))

Load and Explore the Data


projectSTAR <- haven::read_dta("projectSTAR.dta")
glimpse(projectSTAR)
Rows: 6,325
Columns: 27
$ stdntid       <dbl> 10001, 10133, 10246, 10263, 10266, 10275, 10281, 10282, 10285, 10286, 10...
$ race          <dbl+lbl> 1, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1,...
$ gender        <dbl+lbl> 1, 1, 2, 2, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 2, 1, 2, 2, 2, 2, 1, 1, 1,...
$ FLAGSGK       <dbl+lbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,...
$ flaggk        <dbl+lbl> 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1,...
$ gkclasstype   <dbl+lbl> 3, 3, 3, 1, 2, 3, 1, 3, 1, 2, 3, 3, 2, 3, 3, 3, 1, 2, 3, 3, 2, 1, 1,...
$ gkschid       <dbl> 169229, 169280, 218562, 205492, 257899, 161176, 189382, 189382, 201449, ...
$ gksurban      <dbl+lbl> 2, 2, 4, 2, 3, 3, 2, 2, 3, 3, 3, 2, 2, 2, 2, 3, 3, 3, 3, 3, 2, 1, 2,...
$ gktchid       <dbl> 16922904, 16928003, 21856202, 20549204, 25789904, 16117602, 18938204, 18...
$ gktgen        <dbl+lbl> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,...
$ gktrace       <dbl+lbl>  2,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  2,  1,  1,  1, ...
$ gkthighdegree <dbl+lbl>  2,  2,  3,  2,  3,  2,  3,  3,  3,  3,  2,  2,  3,  2,  2,  2,  2, ...
$ gktcareer     <dbl+lbl>  4,  4,  4,  4,  4, NA,  4,  4,  4,  4,  4,  4,  4,  4,  4,  4,  4, ...
$ gktyears      <dbl> 5, 7, 8, 3, 12, 2, 7, 14, 4, 6, 11, 16, 12, 5, 17, 10, 6, 10, 13, 9, 18,...
$ gkclasssize   <dbl> 24, 22, 17, 17, 24, 24, 13, 24, 14, 23, 23, 22, 20, 24, 23, 27, 17, 24, ...
$ gkfreelunch   <dbl+lbl> 2, 2, 2, 1, 2, 2, 2, 1, 1, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2, 1, 2, 2, 2,...
$ gkrepeat      <dbl+lbl> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,...
$ gkspeced      <dbl+lbl> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,...
$ gkspecin      <dbl+lbl> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,...
$ gkpresent     <dbl> 161, 175, 132, 178, 170, 94, 160, 154, 172, 95, 163, 172, 180, 149, 173,...
$ gkabsent      <dbl> 19, 5, 28, 2, 10, 3, 2, 7, 8, 2, 17, 8, 0, 31, 7, NA, 20, 7, 2, 19, 24, ...
$ gktreadss     <dbl> NA, 427, 450, 483, 456, 411, 443, 448, 463, 472, 428, 545, 408, 422, 472...
$ gktmathss     <dbl> NA, 478, 494, 513, 513, 468, 473, 449, 520, 536, 484, 626, 454, 439, 528...
$ gktlistss     <dbl> NA, 509, 549, 554, 520, 571, 595, 540, 565, 595, NA, 622, 474, 536, 578,...
$ gkwordskillss <dbl> NA, 418, 444, 431, 468, 396, 444, 444, 480, 486, 423, 524, 410, 423, 458...
$ gkmotivraw    <dbl> 23, 24, 28, 27, 25, 24, NA, NA, 26, 27, 24, 24, 23, 28, 24, NA, 26, 25, ...
$ gkselfconcraw <dbl> 52, 53, 56, 61, 54, 55, NA, NA, 52, 61, 55, 49, 49, 59, 50, NA, 58, 45, ...

Part One: Data Exploration and Management

Use str to Figure out the 2/1 Coding Scheme and Variable Labels

The str function (short for structure) will show us any variables labels or values that have been attached to a variable. You can use this to find our how variables have been coded.

str(projectSTAR)
tibble [6,325 x 27] (S3: tbl_df/tbl/data.frame)
 $ stdntid      : num [1:6325] 10001 10133 10246 10263 10266 ...
  ..- attr(*, "label")= chr "STUDENT ID"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ race         : dbl+lbl [1:6325] 1, 2, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, ...
   ..@ label       : chr "STUDENT RACE/ETHNICITY"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:6] 1 2 3 4 5 6
   .. ..- attr(*, "names")= chr [1:6] "WHITE" "BLACK" "ASIAN" "HISPANIC" ...
 $ gender       : dbl+lbl [1:6325] 1, 1, 2, 2, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 2, 1, 2, 2, 2, 2, 1, 1, 1, 1, ...
   ..@ label       : chr "STUDENT GENDER"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:2] 1 2
   .. ..- attr(*, "names")= chr [1:2] "MALE" "FEMALE"
 $ FLAGSGK      : dbl+lbl [1:6325] 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
   ..@ label       : chr "IN STAR IN KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:2] 0 1
   .. ..- attr(*, "names")= chr [1:2] "NO" "YES"
 $ flaggk       : dbl+lbl [1:6325] 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, ...
   ..@ label       : chr "ACHIEVEMENT DATA AVAILABLE GRADE K"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:2] 0 1
   .. ..- attr(*, "names")= chr [1:2] "NO" "YES"
 $ gkclasstype  : dbl+lbl [1:6325] 3, 3, 3, 1, 2, 3, 1, 3, 1, 2, 3, 3, 2, 3, 3, 3, 1, 2, 3, 3, 2, 1, 1, 3, ...
   ..@ label       : chr "CLASSROOM TYPE KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:3] 1 2 3
   .. ..- attr(*, "names")= chr [1:3] "SMALL CLASS" "REGULAR CLASS" "REGULAR + AIDE CLASS"
 $ gkschid      : num [1:6325] 169229 169280 218562 205492 257899 ...
  ..- attr(*, "label")= chr "KINDERGARTEN SCHOOL ID"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gksurban     : dbl+lbl [1:6325] 2, 2, 4, 2, 3, 3, 2, 2, 3, 3, 3, 2, 2, 2, 2, 3, 3, 3, 3, 3, 2, 1, 2, 3, ...
   ..@ label       : chr "SCHOOL URBANICITY KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:4] 1 2 3 4
   .. ..- attr(*, "names")= chr [1:4] "INNER CITY" "SUBURBAN" "RURAL" "URBAN"
 $ gktchid      : num [1:6325] 16922904 16928003 21856202 20549204 25789904 ...
  ..- attr(*, "label")= chr "KINDERGARTEN TEACHER ID"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gktgen       : dbl+lbl [1:6325] 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, ...
   ..@ label       : chr "TEACHER GENDER KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:2] 1 2
   .. ..- attr(*, "names")= chr [1:2] "MALE" "FEMALE"
 $ gktrace      : dbl+lbl [1:6325]  2,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  2,  1,  1,  1,  1, ...
   ..@ label       : chr "TEACHER RACE/ETHNICITY KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:6] 1 2 3 4 5 6
   .. ..- attr(*, "names")= chr [1:6] "WHITE" "BLACK" "ASIAN" "HISPANIC" ...
 $ gkthighdegree: dbl+lbl [1:6325]  2,  2,  3,  2,  3,  2,  3,  3,  3,  3,  2,  2,  3,  2,  2,  2,  2,  2, ...
   ..@ label       : chr "TEACHER HIGHEST DEGREE KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:6] 1 2 3 4 5 6
   .. ..- attr(*, "names")= chr [1:6] "ASSOCIATES" "BACHELORS" "MASTERS" "MASTERS +" ...
 $ gktcareer    : dbl+lbl [1:6325]  4,  4,  4,  4,  4, NA,  4,  4,  4,  4,  4,  4,  4,  4,  4,  4,  4,  4, ...
   ..@ label       : chr "TEACHER CAREER LADDER LEVEL KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:7] 1 2 3 4 5 6 7
   .. ..- attr(*, "names")= chr [1:7] "CHOSE NO TO BE ON CAREER LADDER" "APPRENTICE" "PROBATION" "LADDER LEVEL 1" ...
 $ gktyears     : num [1:6325] 5 7 8 3 12 2 7 14 4 6 ...
  ..- attr(*, "label")= chr "YEARS OF TOTAL TEACHING EXPERIENCE KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gkclasssize  : num [1:6325] 24 22 17 17 24 24 13 24 14 23 ...
  ..- attr(*, "label")= chr "CLASS SIZE KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gkfreelunch  : dbl+lbl [1:6325] 2, 2, 2, 1, 2, 2, 2, 1, 1, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2, 1, 2, 2, 2, 2, ...
   ..@ label       : chr "FREE/REDUCED LUNCH STATUS KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:2] 1 2
   .. ..- attr(*, "names")= chr [1:2] "FREE LUNCH" "NON-FREE LUNCH"
 $ gkrepeat     : dbl+lbl [1:6325] 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
   ..@ label       : chr "REPEATING KINDERGARTEN IN 1985-1986 SCHOOL YEAR"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:2] 1 2
   .. ..- attr(*, "names")= chr [1:2] "YES, PROMOTION RECOMMENDED" "NO, PROMOTION NOT RECOMMENDED"
 $ gkspeced     : dbl+lbl [1:6325] 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, ...
   ..@ label       : chr "SPECIAL EDUCATION STATUS KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:2] 1 2
   .. ..- attr(*, "names")= chr [1:2] "YES" "NO"
 $ gkspecin     : dbl+lbl [1:6325] 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, ...
   ..@ label       : chr "PULLED OUT FOR SPECIAL INSTRUCTION KINDERGARTEN"
   ..@ format.stata: chr "%10.0g"
   ..@ labels      : Named num [1:2] 1 2
   .. ..- attr(*, "names")= chr [1:2] "YES" "NO"
 $ gkpresent    : num [1:6325] 161 175 132 178 170 94 160 154 172 95 ...
  ..- attr(*, "label")= chr "DAYS PRESENT AT SCHOOL KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gkabsent     : num [1:6325] 19 5 28 2 10 3 2 7 8 2 ...
  ..- attr(*, "label")= chr "DAYS ABSENT FROM SCHOOL KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gktreadss    : num [1:6325] NA 427 450 483 456 411 443 448 463 472 ...
  ..- attr(*, "label")= chr "TOTAL READING SCALE SCORE SAT KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gktmathss    : num [1:6325] NA 478 494 513 513 468 473 449 520 536 ...
  ..- attr(*, "label")= chr "TOTAL MATH SCALE SCORE SAT KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gktlistss    : num [1:6325] NA 509 549 554 520 571 595 540 565 595 ...
  ..- attr(*, "label")= chr "TOTAL LISTENING SCALE SCORE SAT KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gkwordskillss: num [1:6325] NA 418 444 431 468 396 444 444 480 486 ...
  ..- attr(*, "label")= chr "WORD STUDY SKILLS SCALE SCORE SAT KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gkmotivraw   : num [1:6325] 23 24 28 27 25 24 NA NA 26 27 ...
  ..- attr(*, "label")= chr "MOTIVATION RAW SCORE SCAMIN KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"
 $ gkselfconcraw: num [1:6325] 52 53 56 61 54 55 NA NA 52 61 ...
  ..- attr(*, "label")= chr "SELF-CONCEPT RAW SCORE SCAMIN KINDERGARTEN"
  ..- attr(*, "format.stata")= chr "%10.0g"

Create a Clean Dataset Ready for Modeling

star.clean <- projectSTAR %>%
  mutate(.,
         schoolid = gkschid,
         classid = gktchid,
         read = gktreadss,
         selfcon = gkselfconcraw,
         high_degree = as_factor(gkthighdegree),
         classtype = as_factor(gkclasstype),
         years_exp = gktyears,
         urbanicity = as_factor(gksurban),
         math = gktmathss,
         stu_frl = as_factor(gkfreelunch)
         ) %>%
  group_by(classid) %>% # Create a new variable, which is the average FRL by classroom:
  mutate(.,
            class_frl = mean(gkfreelunch, na.rm = TRUE) - 1,
         class_math = mean(math, na.rm = TRUE) - 1
         ) %>%
  ungroup() %>%
  group_by(schoolid) %>% # Create a new variable, which is the average FRL by school:
  mutate(.,
            school_frl = mean(gkfreelunch, na.rm = TRUE) - 1,
         school_math = mean(math, na.rm = TRUE) - 1
         ) %>%
  ungroup() %>%
  select(.,
         schoolid,
         classid,
         read,
         math,
         classtype,
         years_exp,
         urbanicity,
         high_degree,
         selfcon,
         stu_frl,
         class_frl,
         school_frl,
         class_math,
         school_math
                 )

glimpse(star.clean)  
Rows: 6,325
Columns: 14
$ schoolid    <dbl> 169229, 169280, 218562, 205492, 257899, 161176, 189382, 189382, 201449, 23...
$ classid     <dbl> 16922904, 16928003, 21856202, 20549204, 25789904, 16117602, 18938204, 1893...
$ read        <dbl> NA, 427, 450, 483, 456, 411, 443, 448, 463, 472, 428, 545, 408, 422, 472, ...
$ math        <dbl> NA, 478, 494, 513, 513, 468, 473, 449, 520, 536, 484, 626, 454, 439, 528, ...
$ classtype   <fct> REGULAR + AIDE CLASS, REGULAR + AIDE CLASS, REGULAR + AIDE CLASS, SMALL CL...
$ years_exp   <dbl> 5, 7, 8, 3, 12, 2, 7, 14, 4, 6, 11, 16, 12, 5, 17, 10, 6, 10, 13, 9, 18, 1...
$ urbanicity  <fct> SUBURBAN, SUBURBAN, URBAN, SUBURBAN, RURAL, RURAL, SUBURBAN, SUBURBAN, RUR...
$ high_degree <fct> BACHELORS, BACHELORS, MASTERS, BACHELORS, MASTERS, BACHELORS, MASTERS, MAS...
$ selfcon     <dbl> 52, 53, 56, 61, 54, 55, NA, NA, 52, 61, 55, 49, 49, 59, 50, NA, 58, 45, NA...
$ stu_frl     <fct> NON-FREE LUNCH, NON-FREE LUNCH, NON-FREE LUNCH, FREE LUNCH, NON-FREE LUNCH...
$ class_frl   <dbl> 0.8750000, 0.9545455, 0.7647059, 0.2352941, 0.4583333, 0.6086957, 0.846153...
$ school_frl  <dbl> 0.86206897, 0.86206897, 0.71666667, 0.15189873, 0.62857143, 0.63291139, 0....
$ class_math  <dbl> 491.0476, 494.1000, 487.0000, 498.2500, 481.0000, 477.5833, 467.0000, 494....
$ school_math <dbl> 497.8768, 485.9811, 500.1071, 501.6133, 475.2800, 475.7821, 478.4062, 478....

Part Two: The Null 3-Level Model

Run a null 3-level model for reading scores


model.0 <- lmer(math ~ (1|schoolid) + (1|classid), REML = FALSE, data = star.clean)
summary(model.0)
Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method [lmerModLmerTest]
Formula: math ~ (1 | schoolid) + (1 | classid)
   Data: star.clean

     AIC      BIC   logLik deviance df.resid 
 60618.8  60645.5 -30305.4  60610.8     5867 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-6.2037 -0.6273 -0.0635  0.5846  3.9171 

Random effects:
 Groups   Name        Variance Std.Dev.
 classid  (Intercept)  287.6   16.96   
 schoolid (Intercept)  375.8   19.39   
 Residual             1612.4   40.15   
Number of obs: 5871, groups:  classid, 325; schoolid, 79

Fixed effects:
            Estimate Std. Error     df t value Pr(>|t|)    
(Intercept)   486.00       2.45  78.31   198.4   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Calculate L2 ICC (Classroom)


ICC.class <- 287.6/(1612.4 + 287.6 + 381.9)
ICC.class
[1] 0.1260353

Calculate L3 ICC (School)


ICC.school <- 381.9/(1612.4 + 287.6 + 381.9)
ICC.school
[1] 0.1673605

Part Three: Add Predictors at Each Level

Add a student-level predictor, FRL


model.1 <- lmer(math ~ selfcon + (1|schoolid) + (1|classid), REML = FALSE, data = star.clean)
summary(model.1)
Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method [lmerModLmerTest]
Formula: math ~ selfcon + (1 | schoolid) + (1 | classid)
   Data: star.clean

     AIC      BIC   logLik deviance df.resid 
 48969.7  49002.0 -24479.9  48959.7     4751 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-3.8498 -0.6473 -0.0640  0.5925  3.9068 

Random effects:
 Groups   Name        Variance Std.Dev.
 classid  (Intercept)  283.8   16.85   
 schoolid (Intercept)  395.7   19.89   
 Residual             1551.1   39.38   
Number of obs: 4756, groups:  classid, 301; schoolid, 73

Fixed effects:
             Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)  461.8349     7.1431 2421.8370  64.655  < 2e-16 ***
selfcon        0.4523     0.1188 4688.8553   3.807 0.000142 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
        (Intr)
selfcon -0.931

Add teacher-level predictor, yrs. of experience and class type


model.2 <- lmer(math ~ selfcon + classtype + high_degree + (1|schoolid) + (1|classid), REML = FALSE, data = star.clean)
summary(model.2)
Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method [lmerModLmerTest]
Formula: math ~ selfcon + classtype + high_degree + (1 | schoolid) + (1 |      classid)
   Data: star.clean

     AIC      BIC   logLik deviance df.resid 
 48961.2  49025.9 -24470.6  48941.2     4746 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-3.8266 -0.6442 -0.0610  0.5904  3.9341 

Random effects:
 Groups   Name        Variance Std.Dev.
 classid  (Intercept)  252.1   15.88   
 schoolid (Intercept)  407.1   20.18   
 Residual             1551.2   39.39   
Number of obs: 4756, groups:  classid, 301; schoolid, 73

Fixed effects:
                               Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)                    467.9764     7.3642 2501.3088  63.547  < 2e-16 ***
selfcon                          0.4363     0.1187 4692.9770   3.675 0.000241 ***
classtypeREGULAR CLASS          -8.1748     2.7014  227.9795  -3.026 0.002762 ** 
classtypeREGULAR + AIDE CLASS   -8.8767     2.6938  223.8718  -3.295 0.001143 ** 
high_degreeMASTERS               0.1572     2.7666  251.0161   0.057 0.954721    
high_degreeMASTERS +            -5.2330     7.5361  233.5241  -0.694 0.488128    
high_degreeSPECIALIST           22.2608    13.2338  316.8386   1.682 0.093530 .  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) selfcn cREGUC cRE+AC h_MAST h_MAS+
selfcon     -0.906                                   
cREGULARCLA -0.190  0.030                            
cREGULAR+AC -0.188  0.026  0.475                     
hgh_MASTERS -0.090 -0.018 -0.079 -0.075              
hg_MASTERS+ -0.026 -0.012 -0.032  0.001  0.131       
h_SPECIALIS -0.040 -0.019  0.119  0.118  0.099  0.015

Add a school-level predictor


model.3 <- lmer(math ~ selfcon + classtype + high_degree + school_math + (1|schoolid) + (1|classid), REML = FALSE, data = star.clean)
boundary (singular) fit: see ?isSingular
summary(model.3)
Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method [lmerModLmerTest]
Formula: math ~ selfcon + classtype + high_degree + school_math + (1 |  
    schoolid) + (1 | classid)
   Data: star.clean

     AIC      BIC   logLik deviance df.resid 
 48768.2  48839.3 -24373.1  48746.2     4745 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-3.7976 -0.6435 -0.0603  0.5904  4.0060 

Random effects:
 Groups   Name        Variance Std.Dev.
 classid  (Intercept)  174.4   13.21   
 schoolid (Intercept)    0.0    0.00   
 Residual             1552.8   39.41   
Number of obs: 4756, groups:  classid, 301; schoolid, 73

Fixed effects:
                                Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)                    -13.15882   22.04918  317.89068  -0.597 0.551070    
selfcon                          0.43544    0.11725 4755.84428   3.714 0.000206 ***
classtypeREGULAR CLASS          -8.20633    2.35152  293.16090  -3.490 0.000558 ***
classtypeREGULAR + AIDE CLASS   -8.69961    2.35099  288.49370  -3.700 0.000258 ***
high_degreeMASTERS              -0.33621    2.10936  280.44980  -0.159 0.873477    
high_degreeMASTERS +            -2.37697    5.96377  275.53134  -0.399 0.690520    
high_degreeSPECIALIST           19.25123   10.64910  405.76184   1.808 0.071381 .  
school_math                      0.99386    0.04374  275.01858  22.724  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) selfcn cREGUC cRE+AC h_MAST h_MAS+ h_SPEC
selfcon     -0.275                                          
cREGULARCLA -0.039  0.030                                   
cREGULAR+AC -0.059  0.027  0.479                            
hgh_MASTERS  0.019 -0.011 -0.072 -0.064                     
hg_MASTERS+  0.021 -0.007 -0.021  0.000  0.114              
h_SPECIALIS  0.003 -0.008  0.101  0.101  0.054  0.021       
school_math -0.951 -0.025 -0.019  0.002 -0.043 -0.030 -0.014
convergence code: 0
boundary (singular) fit: see ?isSingular

Part Four: Try Adding a Random Slope

Add a random slope for years experience at level 3 (school)


model.4 <- lmer(math ~ selfcon + classtype + high_degree + school_math + (selfcon|schoolid) + (1|classid), REML = FALSE, data = star.clean)
boundary (singular) fit: see ?isSingular
summary(model.4)
Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method [lmerModLmerTest]
Formula: math ~ selfcon + classtype + high_degree + school_math + (selfcon |  
    schoolid) + (1 | classid)
   Data: star.clean

     AIC      BIC   logLik deviance df.resid 
 48763.3  48847.4 -24368.7  48737.3     4743 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-3.7525 -0.6473 -0.0606  0.5866  4.0303 

Random effects:
 Groups   Name        Variance  Std.Dev. Corr 
 classid  (Intercept)  174.7209 13.2182       
 schoolid (Intercept) 1553.6697 39.4166       
          selfcon        0.4986  0.7061  -1.00
 Residual             1539.9033 39.2416       
Number of obs: 4756, groups:  classid, 301; schoolid, 73

Fixed effects:
                               Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)                   -13.26877   22.58528 318.28430  -0.587 0.557287    
selfcon                         0.41592    0.14666  66.04764   2.836 0.006059 ** 
classtypeREGULAR CLASS         -8.26954    2.35716 291.58465  -3.508 0.000522 ***
classtypeREGULAR + AIDE CLASS  -8.67336    2.35480 286.08960  -3.683 0.000275 ***
high_degreeMASTERS             -0.57626    2.11234 277.90787  -0.273 0.785204    
high_degreeMASTERS +           -2.53758    5.97821 273.87626  -0.424 0.671555    
high_degreeSPECIALIST          19.53847   10.67692 402.57088   1.830 0.067993 .  
school_math                     0.99664    0.04377 271.71372  22.769  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) selfcn cREGUC cRE+AC h_MAST h_MAS+ h_SPEC
selfcon     -0.343                                          
cREGULARCLA -0.040  0.027                                   
cREGULAR+AC -0.058  0.023  0.480                            
hgh_MASTERS  0.019 -0.011 -0.072 -0.064                     
hg_MASTERS+  0.020 -0.005 -0.021 -0.001  0.114              
h_SPECIALIS  0.004 -0.010  0.100  0.101  0.052  0.021       
school_math -0.928 -0.023 -0.019  0.002 -0.043 -0.030 -0.014
convergence code: 0
boundary (singular) fit: see ?isSingular
model.5 <- lmer(math ~ selfcon + classtype + high_degree + school_math + (1|schoolid) + (selfcon|classid), REML = FALSE, data = star.clean)
boundary (singular) fit: see ?isSingular
summary(model.5)
Linear mixed model fit by maximum likelihood . t-tests use Satterthwaite's method [lmerModLmerTest]
Formula: math ~ selfcon + classtype + high_degree + school_math + (1 |  
    schoolid) + (selfcon | classid)
   Data: star.clean

     AIC      BIC   logLik deviance df.resid 
 48762.3  48846.3 -24368.1  48736.3     4743 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-3.7905 -0.6407 -0.0613  0.5844  4.0037 

Random effects:
 Groups   Name        Variance  Std.Dev. Corr 
 classid  (Intercept) 2867.5732 53.5497       
          selfcon        0.8539  0.9241  -0.97
 schoolid (Intercept)    0.0000  0.0000       
 Residual             1532.8541 39.1517       
Number of obs: 4756, groups:  classid, 301; schoolid, 73

Fixed effects:
                               Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)                   -12.30357   22.15668 328.14697  -0.555 0.579069    
selfcon                         0.42743    0.13280 209.00804   3.219 0.001494 ** 
classtypeREGULAR CLASS         -8.20258    2.33735 293.21451  -3.509 0.000520 ***
classtypeREGULAR + AIDE CLASS  -8.69617    2.33513 287.62621  -3.724 0.000236 ***
high_degreeMASTERS             -0.38173    2.09456 279.05991  -0.182 0.855519    
high_degreeMASTERS +           -2.19900    5.93392 276.25113  -0.371 0.711234    
high_degreeSPECIALIST          19.46039   10.60443 400.37621   1.835 0.067230 .  
school_math                     0.99319    0.04347 274.83951  22.848  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) selfcn cREGUC cRE+AC h_MAST h_MAS+ h_SPEC
selfcon     -0.310                                          
cREGULARCLA -0.040  0.029                                   
cREGULAR+AC -0.059  0.027  0.480                            
hgh_MASTERS  0.019 -0.007 -0.072 -0.064                     
hg_MASTERS+  0.019 -0.005 -0.023 -0.001  0.114              
h_SPECIALIS  0.004 -0.007  0.101  0.101  0.053  0.021       
school_math -0.938 -0.029 -0.019  0.002 -0.044 -0.029 -0.014
convergence code: 0
boundary (singular) fit: see ?isSingular

Should We Include That Random Slope? Comparing model.4 with model.5:

anova(model.3, model.4, model.5)
Data: star.clean
Models:
model.3: math ~ selfcon + classtype + high_degree + school_math + (1 | 
model.3:     schoolid) + (1 | classid)
model.4: math ~ selfcon + classtype + high_degree + school_math + (selfcon | 
model.4:     schoolid) + (1 | classid)
model.5: math ~ selfcon + classtype + high_degree + school_math + (1 | 
model.5:     schoolid) + (selfcon | classid)
        npar   AIC   BIC logLik deviance  Chisq Df Pr(>Chisq)    
model.3   11 48768 48839 -24373    48746                         
model.4   13 48763 48847 -24369    48737 8.8308  2    0.01209 *  
model.5   13 48762 48846 -24368    48736 1.0750  0    < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Using the modelsummary and broom.mixed Packages to Organize Your Results:

library(modelsummary)
library(broom.mixed)
Registered S3 method overwritten by 'broom.mixed':
  method      from 
  tidy.gamlss broom
models <- list(model.0, model.1, model.2, model.3, model.4, model.5)
modelsummary(models, output = "default")
Model 1 Model 2 Model 3 Model 4 Model 5 Model 6
(Intercept) 486.000 461.835 467.976 -13.159 -13.269 -12.304
(2.450) (7.143) (7.364) (22.049) (22.585) (22.157)
sd__(Intercept) 16.958 16.846 15.877 13.205 13.218 53.550
sd__(Intercept) 16.958 16.846 15.877 13.205 13.218 0.000
sd__(Intercept) 16.958 16.846 15.877 13.205 39.417 53.550
sd__(Intercept) 16.958 16.846 15.877 13.205 39.417 0.000
sd__(Intercept) 16.958 16.846 15.877 0.000 13.218 53.550
sd__(Intercept) 16.958 16.846 15.877 0.000 13.218 0.000
sd__(Intercept) 16.958 16.846 15.877 0.000 39.417 53.550
sd__(Intercept) 16.958 16.846 15.877 0.000 39.417 0.000
sd__(Intercept) 16.958 16.846 20.176 13.205 13.218 53.550
sd__(Intercept) 16.958 16.846 20.176 13.205 13.218 0.000
sd__(Intercept) 16.958 16.846 20.176 13.205 39.417 53.550
sd__(Intercept) 16.958 16.846 20.176 13.205 39.417 0.000
sd__(Intercept) 16.958 16.846 20.176 0.000 13.218 53.550
sd__(Intercept) 16.958 16.846 20.176 0.000 13.218 0.000
sd__(Intercept) 16.958 16.846 20.176 0.000 39.417 53.550
sd__(Intercept) 16.958 16.846 20.176 0.000 39.417 0.000
sd__(Intercept) 16.958 19.893 15.877 13.205 13.218 53.550
sd__(Intercept) 16.958 19.893 15.877 13.205 13.218 0.000
sd__(Intercept) 16.958 19.893 15.877 13.205 39.417 53.550
sd__(Intercept) 16.958 19.893 15.877 13.205 39.417 0.000
sd__(Intercept) 16.958 19.893 15.877 0.000 13.218 53.550
sd__(Intercept) 16.958 19.893 15.877 0.000 13.218 0.000
sd__(Intercept) 16.958 19.893 15.877 0.000 39.417 53.550
sd__(Intercept) 16.958 19.893 15.877 0.000 39.417 0.000
sd__(Intercept) 16.958 19.893 20.176 13.205 13.218 53.550
sd__(Intercept) 16.958 19.893 20.176 13.205 13.218 0.000
sd__(Intercept) 16.958 19.893 20.176 13.205 39.417 53.550
sd__(Intercept) 16.958 19.893 20.176 13.205 39.417 0.000
sd__(Intercept) 16.958 19.893 20.176 0.000 13.218 53.550
sd__(Intercept) 16.958 19.893 20.176 0.000 13.218 0.000
sd__(Intercept) 16.958 19.893 20.176 0.000 39.417 53.550
sd__(Intercept) 16.958 19.893 20.176 0.000 39.417 0.000
sd__(Intercept) 19.386 16.846 15.877 13.205 13.218 53.550
sd__(Intercept) 19.386 16.846 15.877 13.205 13.218 0.000
sd__(Intercept) 19.386 16.846 15.877 13.205 39.417 53.550
sd__(Intercept) 19.386 16.846 15.877 13.205 39.417 0.000
sd__(Intercept) 19.386 16.846 15.877 0.000 13.218 53.550
sd__(Intercept) 19.386 16.846 15.877 0.000 13.218 0.000
sd__(Intercept) 19.386 16.846 15.877 0.000 39.417 53.550
sd__(Intercept) 19.386 16.846 15.877 0.000 39.417 0.000
sd__(Intercept) 19.386 16.846 20.176 13.205 13.218 53.550
sd__(Intercept) 19.386 16.846 20.176 13.205 13.218 0.000
sd__(Intercept) 19.386 16.846 20.176 13.205 39.417 53.550
sd__(Intercept) 19.386 16.846 20.176 13.205 39.417 0.000
sd__(Intercept) 19.386 16.846 20.176 0.000 13.218 53.550
sd__(Intercept) 19.386 16.846 20.176 0.000 13.218 0.000
sd__(Intercept) 19.386 16.846 20.176 0.000 39.417 53.550
sd__(Intercept) 19.386 16.846 20.176 0.000 39.417 0.000
sd__(Intercept) 19.386 19.893 15.877 13.205 13.218 53.550
sd__(Intercept) 19.386 19.893 15.877 13.205 13.218 0.000
sd__(Intercept) 19.386 19.893 15.877 13.205 39.417 53.550
sd__(Intercept) 19.386 19.893 15.877 13.205 39.417 0.000
sd__(Intercept) 19.386 19.893 15.877 0.000 13.218 53.550
sd__(Intercept) 19.386 19.893 15.877 0.000 13.218 0.000
sd__(Intercept) 19.386 19.893 15.877 0.000 39.417 53.550
sd__(Intercept) 19.386 19.893 15.877 0.000 39.417 0.000
sd__(Intercept) 19.386 19.893 20.176 13.205 13.218 53.550
sd__(Intercept) 19.386 19.893 20.176 13.205 13.218 0.000
sd__(Intercept) 19.386 19.893 20.176 13.205 39.417 53.550
sd__(Intercept) 19.386 19.893 20.176 13.205 39.417 0.000
sd__(Intercept) 19.386 19.893 20.176 0.000 13.218 53.550
sd__(Intercept) 19.386 19.893 20.176 0.000 13.218 0.000
sd__(Intercept) 19.386 19.893 20.176 0.000 39.417 53.550
sd__(Intercept) 19.386 19.893 20.176 0.000 39.417 0.000
sd__Observation 40.155 39.383 39.386 39.405 39.242 39.152
selfcon 0.452 0.436 0.435 0.416 0.427
(0.119) (0.119) (0.117) (0.147) (0.133)
classtypeREGULAR + AIDE CLASS -8.877 -8.700 -8.673 -8.696
(2.694) (2.351) (2.355) (2.335)
classtypeREGULAR CLASS -8.175 -8.206 -8.270 -8.203
(2.701) (2.352) (2.357) (2.337)
high_degreeMASTERS 0.157 -0.336 -0.576 -0.382
(2.767) (2.109) (2.112) (2.095)
high_degreeMASTERS + -5.233 -2.377 -2.538 -2.199
(7.536) (5.964) (5.978) (5.934)
high_degreeSPECIALIST 22.261 19.251 19.538 19.460
(13.234) (10.649) (10.677) (10.604)
school_math 0.994 0.997 0.993
(0.044) (0.044) (0.043)
cor__(Intercept).selfcon -1.000 -0.970
sd__selfcon 0.706 0.924
AIC 60618.8 48969.7 48961.2 48768.2 48763.3 48762.3
BIC 60645.5 49002.0 49025.9 48839.3 48847.4 48846.3
Log.Lik. -30305.389 -24479.852 -24470.610 -24373.084 -24368.669 -24368.131

HTML Version That You Can Open in Word:

modelsummary(models, output = 'msum.html', title = 'MLM Estimates')
[WARNING] This document format requires a nonempty <title> element.
  Please specify either 'title' or 'pagetitle' in the metadata,
  e.g. by using --metadata pagetitle="..." on the command line.
  Falling back to 'msum'
LS0tDQp0aXRsZTogJ01vZHVsZSA3OiBUaHJlZS1MZXZlbCBNb2RlbHMnDQphdXRob3I6ICdKYWtlIFJleW5vbGRzIC0gT2N0b2JlciA2LCAyMDIwJw0Kb3V0cHV0OiBodG1sX25vdGVib29rDQotLS0NCjEuIFJ1biBhbmQgaW50ZXJwcmV0IGEgbnVsbCBtb2RlbCB3aXRoIG1hdGggc2NvcmVzIGdrdG1hdGhzcyBhcyB0aGUgRFYsIHdpdGggc3R1ZGVudHMgY2x1c3RlcmVkIHdpdGhpbiBjbGFzc3Jvb21zIGFuZCBzY2hvb2xzLiBIb3cgbXVjaCB2YXJpYXRpb24gaW4gbWF0aCBzY29yZXMgaXMgYXQgdGhlIHRlYWNoZXIgbGV2ZWw/IFRoZSBzY2hvb2wgbGV2ZWw/IFdoaWNoIGxldmVsIGRlbW9uc3RyYXRlcyBtb3JlIHZhcmlhYmlsaXR5PyBBbmQgaG93IG11Y2ggdmFyaWFiaWxpdHkgcmVtYWlucyBhdCB0aGUgc3R1ZGVudCBsZXZlbD8NCg0KVmFyaWF0aW9uIGluIE1hdGggU2NvcmVzDQpJQ0MgKENsYXNzcm9vbSkgPSAwLjEyNjAzNTMgDQpJQ0MgKFNjaG9vbCkgPSAwLjE2NzM2MDUNCg0KVGhlcmUgaXMgMTMlIHZhcmlhdGlvbiBhdCB0aGUgdGVhY2hlci1sZXZlbCwgMTclIHZhcmlhdGlvbiBhdCB0aGUgc2Nob29sIGxldmVsLiBUaGUgc2Nob29sIGxldmVsIGRlbW9uc3RyYXRlcyBtb3JlIHZhcmlhYmlsaXR5LiBBYm91dCA3MSUgb2YgdGhlIHZhcmlhYmlsaXR5IHJlbWFpbnMgYXQgdGhlIHN0dWRlbnQgbGV2ZWwuDQoNCjIuIE5vdywgcnVuIGEgY29uZGl0aW9uYWwgcmFuZG9tIGludGVyY2VwdCBtb2RlbCB3aXRoIG1hdGggc2NvcmVzIChna3RtYXRoc3MpIGFzIHRoZSBEViwgYW5kIHNlbGYtY29uY2VwdCAoZ2tzZWxmY29uY3JhdykgYXMgYSBzdHVkZW50LWxldmVsIElWLCBjbGFzc3Jvb20gdHlwZSAoZ2tjbGFzc3R5cGUpIGFuZCB0ZWFjaGVyIGhpZ2hlc3QgZGVncmVlIChna3RoaWdoZGVncmVlKSBhcyB0ZWFjaGVyLWxldmVsIElWcywgYW5kIHNjaG9vbCBhdmVyYWdlIG1hdGggc2NvcmVzIGFzIGEgc2Nob29sIGxldmVsIElWLiAoSGludC0geW91IHdpbGwgbmVlZCB0byBjYWxjdWxhdGUgdGhlIGxhc3Qgb25lIHVzaW5nIHRoZSBwcm9jZWR1cmVzIEkgZGVtb25zdHJhdGVkIGluIHRoZSB2aWRlb3MpLiBJbnRlcnByZXQgdGhlIHJlc3VsdHMgYW5kIGV2YWx1YXRlIG1vZGVsIGZpdCB1c2luZyBBSUMvQklDIGFuZCB5b3VyIGNob2ljZSBvZiBlZmZlY3Qgc2l6ZS4NCg0KDQpUaGUgZm9sbG93aW5nIHByZWRpY3RvcnMgYXJlIHNpZ25pZmljYW50OiBjbGFzc3Jvb20gdHlwZSBhbmQgc2VsZi1jb25jZXB0LiBUaGUgaGlnaGVzdCBkZWdyZWUgaXMgbm90IHNob3dpbmcgc2lnbmlmYW5jZSBhdCB0aGUgdGVhY2hlci1sZXZlbCwgYWx0aG91Z2ggdGhlcmUgaXMgc2xpZ2h0IHNpZ25pZmljYW5jZSBmb3Igc3BlY2lhbGlzdCBvdmVyIG90aGVyIGhpZ2ggZGVncmVlcy4gVGhlIGF2ZXJhZ2UgbWF0aCBzY29yZXMgYXQgdGhlIHNjaG9vbCBsZXZlbCBzaG93IHRvIGJlIHNpZ25pZmljYW50IHByZWRpY3RvcnMgYXMgd2VsbC4gVGhlIEFJQyBhbmQgQklDIGJvdGggaW1wcm92ZSB3aXRoIHRoZSBtb2RlbCBhcyB3ZSBpbmNsdWRlIGFkZGl0aW9uYWwgcHJlZGljdG9ycy4gIA0KDQoNCg0KMy4gIFRyeSBhZGRpbmcgYSByYW5kb20gc2xvcGUgZm9yIHNlbGYtY29uY2VwdCBna3NlbGZjb25jcmF3IGF0IGJvdGggdGhlIHRlYWNoZXIgYW5kIHNjaG9vbCBsZXZlbHMuIFlvdSBjYW4gZG8gdGhpcyBhcyB0d28gc2VwYXJhdGUgbW9kZWxzLSBvbmUgd2l0aCB0aGUgcmFuZG9tIHNsb3BlIGF0IHRoZSB0ZWFjaGVyIGxldmVsLCBhbmQgb25lIHdpdGggdGhlIHJhbmRvbSBzbG9wZSBhdCB0aGUgc2Nob29sIGxldmVsLiBXaGF0IGRvIHRoZXNlIHJhbmRvbSBzbG9wZXMg4oCcbWVhbuKAnT8gVXNlIEFJQy9CSUMgdG8gZXZhbHVhdGUgd2hldGhlciB0aGUgc2xvcGVzIHNob3VsZCBiZSB0cmVhdGVkIGFzIHJhbmRvbS4NCg0KV2l0aG91dCBhIFJhbmRvbSBTbG9wZSB0aGUgQUlDIGlzIDQ4NzY4LCB0aGUgQklDIGlzIDQ4ODM5LlRoZSBSYW5kb20gU2xvcGUgKHNlbGYtY29uY2VwdCkgYXQgdGVhY2hlciBsZXZlbCA6IEFJQyBpcyA0ODc2MywgQklDIGlzIDQ4ODQ3LiBUaGUgUmFuZG9tIFNsb3BlIChzZWxmLWNvbmNlcHQpIGF0IHRoZSBzY2hvb2wgbGV2ZWwgdGhlIEFJQyBpcyA0ODc2MiwgdGhlIEJJQyBpcyA0ODg0Ni4NCg0KQWRkaW5nIHRoZSByYW5kb20gc2xvcGVzIGFsbG93cyB2YXJpYWJpbGl0eSBmb3IgYSBnaXZlbiBwcmVkaWN0b3IgYWNyb3NzIGEgaGlnaGVyLWxldmVsIGNsdXN0ZXJpbmcgdmFyaWFibGUuIEl0IGFsbG93cyB0aGUgZGlyZWN0aW9uIG9mIHRoZSBwcmVkaWN0b3IgdG8gY2hhbmdlLkFkZGluZyB0aGUgcmFuZG9tIHNsb3BlIGF0IHRoZSBzY2hvb2wgbGV2ZWwgZGVjcmVhc2VkIHRoZSBBSUMsIGJ1dCBub3QgQklDLiBXZSBzaG91bGQgbm90IGluY2x1ZGUgdGhlIHJhbmRvbSBzbG9wZSAnZ2tzZWxmY29uY3JhdycuIA0KDQoNCg0KDQoNCiMgTG9hZCBTb21lIFBhY2thZ2VzIHRvIEhlbHAgd2l0aCB0aGUgQW5hbHlzaXMgYW5kIERhdGEgTWFuYWdlbWVudDoNCmBgYHtyfQ0Kc3VwcHJlc3NQYWNrYWdlU3RhcnR1cE1lc3NhZ2VzKGxpYnJhcnkodGlkeXZlcnNlKSkNCnN1cHByZXNzUGFja2FnZVN0YXJ0dXBNZXNzYWdlcyhsaWJyYXJ5KGxtZTQpKQ0KYGBgDQoNCiMgTG9hZCBhbmQgRXhwbG9yZSB0aGUgRGF0YQ0KYGBge3J9DQoNCnByb2plY3RTVEFSIDwtIGhhdmVuOjpyZWFkX2R0YSgicHJvamVjdFNUQVIuZHRhIikNCmdsaW1wc2UocHJvamVjdFNUQVIpDQoNCmBgYA0KDQojIFBhcnQgT25lOiBEYXRhIEV4cGxvcmF0aW9uIGFuZCBNYW5hZ2VtZW50DQoNCiMjIFVzZSBgc3RyYCB0byBGaWd1cmUgb3V0IHRoZSAyLzEgQ29kaW5nIFNjaGVtZSBhbmQgVmFyaWFibGUgTGFiZWxzDQpUaGUgYHN0cmAgZnVuY3Rpb24gKHNob3J0IGZvciBzdHJ1Y3R1cmUpIHdpbGwgc2hvdyB1cyBhbnkgdmFyaWFibGVzIGxhYmVscyBvciB2YWx1ZXMgdGhhdCBoYXZlIGJlZW4gYXR0YWNoZWQgdG8gYSB2YXJpYWJsZS4gWW91IGNhbiB1c2UgdGhpcyB0byBmaW5kIG91ciBob3cgdmFyaWFibGVzIGhhdmUgYmVlbiBjb2RlZC4NCmBgYHtyfQ0Kc3RyKHByb2plY3RTVEFSKQ0KDQpgYGANCg0KIyMgQ3JlYXRlIGEgQ2xlYW4gRGF0YXNldCBSZWFkeSBmb3IgTW9kZWxpbmcNCmBgYHtyfQ0Kc3Rhci5jbGVhbiA8LSBwcm9qZWN0U1RBUiAlPiUNCiAgbXV0YXRlKC4sDQogICAgICAgICBzY2hvb2xpZCA9IGdrc2NoaWQsDQogICAgICAgICBjbGFzc2lkID0gZ2t0Y2hpZCwNCiAgICAgICAgIHJlYWQgPSBna3RyZWFkc3MsDQogICAgICAgICBzZWxmY29uID0gZ2tzZWxmY29uY3JhdywNCiAgICAgICAgIGhpZ2hfZGVncmVlID0gYXNfZmFjdG9yKGdrdGhpZ2hkZWdyZWUpLA0KICAgICAgICAgY2xhc3N0eXBlID0gYXNfZmFjdG9yKGdrY2xhc3N0eXBlKSwNCiAgICAgICAgIHllYXJzX2V4cCA9IGdrdHllYXJzLA0KICAgICAgICAgdXJiYW5pY2l0eSA9IGFzX2ZhY3Rvcihna3N1cmJhbiksDQogICAgICAgICBtYXRoID0gZ2t0bWF0aHNzLA0KICAgICAgICAgc3R1X2ZybCA9IGFzX2ZhY3Rvcihna2ZyZWVsdW5jaCkNCiAgICAgICAgICkgJT4lDQogIGdyb3VwX2J5KGNsYXNzaWQpICU+JSAjIENyZWF0ZSBhIG5ldyB2YXJpYWJsZSwgd2hpY2ggaXMgdGhlIGF2ZXJhZ2UgRlJMIGJ5IGNsYXNzcm9vbToNCiAgbXV0YXRlKC4sDQogICAgICAgICAgICBjbGFzc19mcmwgPSBtZWFuKGdrZnJlZWx1bmNoLCBuYS5ybSA9IFRSVUUpIC0gMSwNCiAgICAgICAgIGNsYXNzX21hdGggPSBtZWFuKG1hdGgsIG5hLnJtID0gVFJVRSkgLSAxDQogICAgICAgICApICU+JQ0KICB1bmdyb3VwKCkgJT4lDQogIGdyb3VwX2J5KHNjaG9vbGlkKSAlPiUgIyBDcmVhdGUgYSBuZXcgdmFyaWFibGUsIHdoaWNoIGlzIHRoZSBhdmVyYWdlIEZSTCBieSBzY2hvb2w6DQogIG11dGF0ZSguLA0KICAgICAgICAgICAgc2Nob29sX2ZybCA9IG1lYW4oZ2tmcmVlbHVuY2gsIG5hLnJtID0gVFJVRSkgLSAxLA0KICAgICAgICAgc2Nob29sX21hdGggPSBtZWFuKG1hdGgsIG5hLnJtID0gVFJVRSkgLSAxDQogICAgICAgICApICU+JQ0KICB1bmdyb3VwKCkgJT4lDQogIHNlbGVjdCguLA0KICAgICAgICAgc2Nob29saWQsDQogICAgICAgICBjbGFzc2lkLA0KICAgICAgICAgcmVhZCwNCiAgICAgICAgIG1hdGgsDQogICAgICAgICBjbGFzc3R5cGUsDQogICAgICAgICB5ZWFyc19leHAsDQogICAgICAgICB1cmJhbmljaXR5LA0KICAgICAgICAgaGlnaF9kZWdyZWUsDQogICAgICAgICBzZWxmY29uLA0KICAgICAgICAgc3R1X2ZybCwNCiAgICAgICAgIGNsYXNzX2ZybCwNCiAgICAgICAgIHNjaG9vbF9mcmwsDQogICAgICAgICBjbGFzc19tYXRoLA0KICAgICAgICAgc2Nob29sX21hdGgNCiAgICAgICAgICAgICAgICAgKQ0KDQpnbGltcHNlKHN0YXIuY2xlYW4pICANCmBgYA0KDQojIFBhcnQgVHdvOiBUaGUgTnVsbCAzLUxldmVsIE1vZGVsDQoNCiMjIFJ1biBhIG51bGwgMy1sZXZlbCBtb2RlbCBmb3IgcmVhZGluZyBzY29yZXMNCg0KYGBge3J9DQoNCm1vZGVsLjAgPC0gbG1lcihtYXRoIH4gKDF8c2Nob29saWQpICsgKDF8Y2xhc3NpZCksIFJFTUwgPSBGQUxTRSwgZGF0YSA9IHN0YXIuY2xlYW4pDQpzdW1tYXJ5KG1vZGVsLjApDQoNCmBgYA0KDQojIyBDYWxjdWxhdGUgTDIgSUNDIChDbGFzc3Jvb20pDQpgYGB7cn0NCg0KSUNDLmNsYXNzIDwtIDI4Ny42LygxNjEyLjQgKyAyODcuNiArIDM4MS45KQ0KSUNDLmNsYXNzDQoNCmBgYA0KDQoNCiMjIENhbGN1bGF0ZSBMMyBJQ0MgKFNjaG9vbCkNCmBgYHtyfQ0KDQpJQ0Muc2Nob29sIDwtIDM4MS45LygxNjEyLjQgKyAyODcuNiArIDM4MS45KQ0KSUNDLnNjaG9vbA0KDQpgYGANCg0KIyBQYXJ0IFRocmVlOiBBZGQgUHJlZGljdG9ycyBhdCBFYWNoIExldmVsDQoNCiMjIEFkZCBhIHN0dWRlbnQtbGV2ZWwgcHJlZGljdG9yLCBGUkwNCg0KYGBge3J9DQoNCm1vZGVsLjEgPC0gbG1lcihtYXRoIH4gc2VsZmNvbiArICgxfHNjaG9vbGlkKSArICgxfGNsYXNzaWQpLCBSRU1MID0gRkFMU0UsIGRhdGEgPSBzdGFyLmNsZWFuKQ0Kc3VtbWFyeShtb2RlbC4xKQ0KDQpgYGANCg0KIyMgQWRkIHRlYWNoZXItbGV2ZWwgcHJlZGljdG9yLCB5cnMuIG9mIGV4cGVyaWVuY2UgYW5kIGNsYXNzIHR5cGUNCg0KYGBge3J9DQoNCm1vZGVsLjIgPC0gbG1lcihtYXRoIH4gc2VsZmNvbiArIGNsYXNzdHlwZSArIGhpZ2hfZGVncmVlICsgKDF8c2Nob29saWQpICsgKDF8Y2xhc3NpZCksIFJFTUwgPSBGQUxTRSwgZGF0YSA9IHN0YXIuY2xlYW4pDQpzdW1tYXJ5KG1vZGVsLjIpDQoNCmBgYA0KDQojIyBBZGQgYSBzY2hvb2wtbGV2ZWwgcHJlZGljdG9yDQoNCmBgYHtyfQ0KDQptb2RlbC4zIDwtIGxtZXIobWF0aCB+IHNlbGZjb24gKyBjbGFzc3R5cGUgKyBoaWdoX2RlZ3JlZSArIHNjaG9vbF9tYXRoICsgKDF8c2Nob29saWQpICsgKDF8Y2xhc3NpZCksIFJFTUwgPSBGQUxTRSwgZGF0YSA9IHN0YXIuY2xlYW4pDQpzdW1tYXJ5KG1vZGVsLjMpDQoNCmBgYA0KDQoNCg0KIyBQYXJ0IEZvdXI6IFRyeSBBZGRpbmcgYSBSYW5kb20gU2xvcGUNCg0KIyMgQWRkIGEgcmFuZG9tIHNsb3BlIGZvciB5ZWFycyBleHBlcmllbmNlIGF0IGxldmVsIDMgKHNjaG9vbCkNCmBgYHtyfQ0KDQptb2RlbC40IDwtIGxtZXIobWF0aCB+IHNlbGZjb24gKyBjbGFzc3R5cGUgKyBoaWdoX2RlZ3JlZSArIHNjaG9vbF9tYXRoICsgKHNlbGZjb258c2Nob29saWQpICsgKDF8Y2xhc3NpZCksIFJFTUwgPSBGQUxTRSwgZGF0YSA9IHN0YXIuY2xlYW4pDQpzdW1tYXJ5KG1vZGVsLjQpDQoNCmBgYA0KDQpgYGB7cn0NCm1vZGVsLjUgPC0gbG1lcihtYXRoIH4gc2VsZmNvbiArIGNsYXNzdHlwZSArIGhpZ2hfZGVncmVlICsgc2Nob29sX21hdGggKyAoMXxzY2hvb2xpZCkgKyAoc2VsZmNvbnxjbGFzc2lkKSwgUkVNTCA9IEZBTFNFLCBkYXRhID0gc3Rhci5jbGVhbikNCnN1bW1hcnkobW9kZWwuNSkNCmBgYA0KDQojIFNob3VsZCBXZSBJbmNsdWRlIFRoYXQgUmFuZG9tIFNsb3BlPyBDb21wYXJpbmcgYG1vZGVsLjRgIHdpdGggYG1vZGVsLjVgOg0KYGBge3J9DQphbm92YShtb2RlbC4zLCBtb2RlbC40LCBtb2RlbC41KQ0KYGBgDQoNCiMgVXNpbmcgdGhlIGBtb2RlbHN1bW1hcnlgIGFuZCBgYnJvb20ubWl4ZWRgIFBhY2thZ2VzIHRvIE9yZ2FuaXplIFlvdXIgUmVzdWx0czoNCmBgYHtyfQ0KbGlicmFyeShtb2RlbHN1bW1hcnkpDQpsaWJyYXJ5KGJyb29tLm1peGVkKQ0KDQptb2RlbHMgPC0gbGlzdChtb2RlbC4wLCBtb2RlbC4xLCBtb2RlbC4yLCBtb2RlbC4zLCBtb2RlbC40LCBtb2RlbC41KQ0KbW9kZWxzdW1tYXJ5KG1vZGVscywgb3V0cHV0ID0gImRlZmF1bHQiKQ0KYGBgDQoNCiMgSFRNTCBWZXJzaW9uIFRoYXQgWW91IENhbiBPcGVuIGluIFdvcmQ6DQpgYGB7cn0NCm1vZGVsc3VtbWFyeShtb2RlbHMsIG91dHB1dCA9ICdtc3VtLmh0bWwnLCB0aXRsZSA9ICdNTE0gRXN0aW1hdGVzJykNCg0KYGBgDQoNCg==