What

Russia has many ethnic groups. Here we investigate the intelligence gaps between two ethnic minority groups and the majority Russian group using the Raven’s test with item-level data. We employ IRT to analyze the items and relate the item stats to the group differences.

Init

options(digits = 3)
library(pacman)
p_load(kirkegaard, readxl, rms, mirt)

#parallelization
mirtCluster()

#Inf to NA
INF_to_NA = function(x) {x[is.infinite(x)] = NA; x}

Data

#read data
d = readxl::read_xlsx("data/IRT Dataset.xlsx")

#sample size
nrow(d)
## [1] 1559
#groups
d$Region %>% table2()
#missing data?
miss_by_var(d)
##    Age    Sex Region     A1     A2     A3     A4     A5     A6     A7 
##      0      1      0      0      0      0      0      0      0      0 
##     A8     A9    A10    A11    A12     B1     B2     B3     B4     B5 
##      0      0      0      0      0      0      0      0      0      0 
##     B6     B7     B8     B9    B10    B11    B12     C1     C2     C3 
##      0      0      0      0      0      0      0      0      0      0 
##     C4     C5     C6     C7     C8     C9    C10    C11    C12     D1 
##      0      0      0      0      0      0      0      0      0      0 
##     D2     D3     D4     D5     D6     D7     D8     D9    D10    D11 
##      0      0      0      0      0      0      0      0      0      0 
##    D12     E1     E2     E3     E4     E5     E6     E7     E8     E9 
##      0      0      0      0      0      0      0      0      0      0 
##    E10    E11    E12 
##      0      0      0
#which are items
cog_items_names = str_subset(names(d), "[A-E]\\d{1,2}")
length(cog_items_names)
## [1] 60

IRT

#item stats
item_stats = data_frame(
  item_number = 1:length(cog_items_names),
  item = cog_items_names,
  
  #difficulty
  pass_rate = map_dbl(cog_items_names, function(v) {d[[v]] %>% mean(na.rm = T)})
)
## Warning: `data_frame()` is deprecated, use `tibble()`.
## This warning is displayed once per session.
#IRT full
irt_fa_all = irt.fa(d[cog_items_names])
## Warning in cor.smooth(mat): Matrix was not positive definite, smoothing was
## done
## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs
## = np.obs, : The estimated weights for the factor scores are probably
## incorrect. Try a different factor extraction method.

irt_fa_all
## Item Response Analysis using Factor Analysis 
## 
## Call: irt.fa(x = d[cog_items_names])
## Item Response Analysis using Factor Analysis  
## 
##  Summary information by factor and item
##  Factor =  1 
##               -3    -2    -1    0    1    2    3
## A1          0.08  0.08  0.07 0.06 0.04 0.02 0.02
## A2          0.28  0.25  0.14 0.06 0.02 0.01 0.00
## A3          0.15  0.17  0.13 0.08 0.04 0.02 0.01
## A4          0.51  0.41  0.15 0.04 0.01 0.00 0.00
## A5          0.27  0.31  0.19 0.08 0.03 0.01 0.00
## A6          0.24  0.25  0.16 0.08 0.03 0.01 0.00
## A7          0.26  0.36  0.25 0.10 0.04 0.01 0.00
## A8          0.25  0.41  0.31 0.13 0.04 0.01 0.00
## A9          0.65  0.44  0.12 0.03 0.01 0.00 0.00
## A10         0.27  0.44  0.31 0.12 0.04 0.01 0.00
## A11         0.11  0.22  0.28 0.21 0.10 0.04 0.01
## A12         0.08  0.16  0.23 0.21 0.13 0.06 0.03
## B1          0.17  0.19  0.14 0.08 0.04 0.02 0.01
## B2          0.36  0.35  0.18 0.06 0.02 0.01 0.00
## B3          0.65  0.42  0.12 0.02 0.00 0.00 0.00
## B4          0.30  0.64  0.42 0.12 0.03 0.01 0.00
## B5          0.70  0.84  0.22 0.03 0.00 0.00 0.00
## B6          0.15  0.37  0.44 0.22 0.07 0.02 0.01
## B7          0.13  0.29  0.36 0.23 0.09 0.03 0.01
## B8          0.17  0.61  0.70 0.22 0.04 0.01 0.00
## B9          0.19  0.54  0.56 0.20 0.05 0.01 0.00
## B10         0.35  0.97  0.49 0.09 0.01 0.00 0.00
## B11         0.12  0.28  0.38 0.25 0.10 0.03 0.01
## B12         0.08  0.25  0.45 0.36 0.14 0.04 0.01
## C1          0.43  0.88  0.38 0.07 0.01 0.00 0.00
## C2          0.29  0.57  0.39 0.12 0.03 0.01 0.00
## C3          0.11  0.25  0.35 0.25 0.11 0.04 0.01
## C4          0.52  0.81  0.30 0.06 0.01 0.00 0.00
## C5          0.08  0.21  0.37 0.32 0.15 0.05 0.02
## C6          0.21  0.63  0.59 0.18 0.04 0.01 0.00
## C7          0.05  0.11  0.19 0.23 0.18 0.10 0.04
## C8          0.06  0.22  0.52 0.45 0.17 0.04 0.01
## C9          0.02  0.06  0.12 0.20 0.23 0.17 0.09
## C11         0.03  0.05  0.09 0.13 0.15 0.13 0.08
## C12         0.02  0.05  0.10 0.18 0.22 0.18 0.10
## D1          0.06  0.22  0.47 0.42 0.17 0.05 0.01
## D2          0.05  0.18  0.49 0.52 0.21 0.05 0.01
## D3          0.03  0.10  0.25 0.37 0.28 0.12 0.04
## D4          0.03  0.07  0.14 0.20 0.21 0.14 0.07
## D5          0.02  0.05  0.11 0.21 0.26 0.20 0.10
## D6          0.03  0.06  0.09 0.11 0.12 0.10 0.07
## D7          0.03  0.07  0.13 0.18 0.18 0.12 0.07
## D8          0.02  0.05  0.09 0.14 0.16 0.14 0.09
## D10         0.03  0.05  0.08 0.11 0.12 0.10 0.07
## D11         0.02  0.05  0.10 0.17 0.20 0.16 0.09
## D12         0.02  0.04  0.07 0.11 0.13 0.13 0.10
## E1          0.04  0.08  0.14 0.20 0.19 0.12 0.06
## E2          0.03  0.06  0.12 0.18 0.20 0.15 0.08
## E3          0.03  0.05  0.08 0.11 0.11 0.09 0.07
## E4          0.02  0.05  0.10 0.15 0.18 0.15 0.09
## E5          0.02  0.04  0.07 0.10 0.12 0.11 0.09
## Test Info   8.85 14.31 12.74 8.58 5.23 3.03 1.59
## SEM         0.34  0.26  0.28 0.34 0.44 0.57 0.79
## Reliability 0.89  0.93  0.92 0.88 0.81 0.67 0.37
## 
## Factor analysis with Call: fa(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, 
##     fm = fm)
## 
## Test of the hypothesis that 1 factor is sufficient.
## The degrees of freedom for the model is 1710  and the objective function was  176 
## The number of observations was  1559  with Chi Square =  271097  with prob <  0 
## 
## The root mean square of the residuals (RMSA) is  0.1 
## The df corrected root mean square of the residuals is  0.11 
## 
## Tucker Lewis Index of factoring reliability =  0.06
## RMSEA index =  0.32  and the 10 % confidence intervals are  0.317 NA
## BIC =  258525
irt_fa_all$fa
## Factor Analysis using method =  minres
## Call: fa(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, 
##     fm = fm)
## Standardized loadings (pattern matrix) based upon correlation matrix
##       MR1     h2   u2 com
## A1   0.32 0.1027 0.90   1
## A2   0.53 0.2820 0.72   1
## A3   0.43 0.1883 0.81   1
## A4   0.65 0.4261 0.57   1
## A5   0.55 0.3028 0.70   1
## A6   0.52 0.2664 0.73   1
## A7   0.58 0.3333 0.67   1
## A8   0.60 0.3653 0.63   1
## A9   0.69 0.4791 0.52   1
## A10  0.61 0.3776 0.62   1
## A11  0.53 0.2810 0.72   1
## A12  0.49 0.2430 0.76   1
## B1   0.46 0.2079 0.79   1
## B2   0.59 0.3507 0.65   1
## B3   0.69 0.4778 0.52   1
## B4   0.69 0.4719 0.53   1
## B5   0.76 0.5751 0.42   1
## B6   0.62 0.3864 0.61   1
## B7   0.58 0.3347 0.67   1
## B8   0.72 0.5252 0.47   1
## B9   0.69 0.4705 0.53   1
## B10  0.76 0.5751 0.42   1
## B11  0.59 0.3457 0.65   1
## B12  0.63 0.3914 0.61   1
## C1   0.74 0.5482 0.45   1
## C2   0.67 0.4455 0.55   1
## C3   0.57 0.3240 0.68   1
## C4   0.73 0.5369 0.46   1
## C5   0.59 0.3483 0.65   1
## C6   0.71 0.5026 0.50   1
## C7   0.49 0.2417 0.76   1
## C8   0.66 0.4357 0.56   1
## C9   0.49 0.2410 0.76   1
## C10  0.05 0.0025 1.00   1
## C11  0.41 0.1712 0.83   1
## C12  0.49 0.2354 0.76   1
## D1   0.64 0.4098 0.59   1
## D2   0.67 0.4463 0.55   1
## D3   0.58 0.3405 0.66   1
## D4   0.48 0.2302 0.77   1
## D5   0.51 0.2648 0.74   1
## D6   0.37 0.1401 0.86   1
## D7   0.46 0.2073 0.79   1
## D8   0.43 0.1822 0.82   1
## D9   0.25 0.0613 0.94   1
## D10  0.37 0.1399 0.86   1
## D11  0.46 0.2161 0.78   1
## D12  0.40 0.1585 0.84   1
## E1   0.47 0.2191 0.78   1
## E2   0.47 0.2163 0.78   1
## E3   0.37 0.1355 0.86   1
## E4   0.45 0.1998 0.80   1
## E5   0.37 0.1401 0.86   1
## E6   0.06 0.0037 1.00   1
## E7   0.13 0.0177 0.98   1
## E8   0.20 0.0418 0.96   1
## E9   0.23 0.0526 0.95   1
## E10  0.04 0.0019 1.00   1
## E11  0.19 0.0348 0.97   1
## E12 -0.07 0.0048 1.00   1
## 
##                  MR1
## SS loadings    16.66
## Proportion Var  0.28
## 
## Mean item complexity =  1
## Test of the hypothesis that 1 factor is sufficient.
## 
## The degrees of freedom for the null model are  1770  and the objective function was  194 with Chi Square of  298603
## The degrees of freedom for the model are 1710  and the objective function was  176 
## 
## The root mean square of the residuals (RMSR) is  0.1 
## The df corrected root mean square of the residuals is  0.11 
## 
## The harmonic number of observations is  1559 with the empirical chi square  60354  with prob <  0 
## The total number of observations was  1559  with Likelihood Chi Square =  271097  with prob <  0 
## 
## Tucker Lewis Index of factoring reliability =  0.06
## RMSEA index =  0.32  and the 90 % confidence intervals are  0.317 NA
## BIC =  258525
## Fit based upon off diagonal values = 0.88
#extract info
item_stats$difficulty = irt_fa_all$irt$difficulty[[1]]
item_stats$discrimination = irt_fa_all$irt$discrimination[, 1]
item_stats$loading = irt_fa_all$fa$loadings[, 1]

#IRT scores
irt_fa_all_scores = scoreIrt(irt_fa_all, d[cog_items_names])
d$g_IRT = irt_fa_all_scores$theta1 %>% standardize()

#sum score
d$g_sumscore = rowSums(d[cog_items_names], na.rm = T)

#item whole - CTT metric
item_stats$item_whole = map_dbl(cog_items_names, function(v) {
  cor(d[c("g_sumscore", v)]) %>% .[1, 2]
  })

#relation
GG_scatter(d, "g_IRT", "g_sumscore")

Group differences

Gaps / simple

#mean score by region
describeBy(d[c("g_IRT", "g_sumscore")], d$Region, mat = T)
#plots
GG_group_means(d, "g_IRT", "Region")

GG_denhist(d, "g_IRT", "Region")
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

#sex
describeBy(d[c("g_IRT", "g_sumscore")], d$Sex, mat = T)
GG_denhist(d, "g_IRT", "Sex")
## Warning in GG_denhist(d, "g_IRT", "Sex"): Grouping variable contained
## missing values. These were removed. If you want an NA group, convert to
## explicit value.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

#sex gap by age
ggplot(d, aes(Age, g_IRT, color = Sex)) +
  geom_smooth() +
  theme_bw()
## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

#table suitable for meta-analysis fodder
d %>% plyr::ddply(c("Age", "Sex"), .fun = function(dd) {
  dd %$% 
    data_frame(
      mean = mean(g_IRT),
      n = nrow(dd)
    )
})

Low scoring minorities: Tungus, Khabarovsk

Grouping the 2 small low scoring groups.

minority_groups = c("Tungus", "Khabarovsk")
d_LSM = d %>% filter(Region %in% c("Tungus", "Khabarovsk", "Tomsk"))
d_LSM$minority = d_LSM$Region %in% minority_groups
simple_item_stats = describeBy(d_LSM[cog_items_names], d_LSM$minority)
item_stats$pass_rate_d = simple_item_stats$`FALSE`$mean - simple_item_stats$`TRUE`$mean
item_stats$minority_z = qnorm(simple_item_stats$`TRUE`$mean, mean = item_stats$difficulty) %>% INF_to_NA()
item_stats$Russian_z = qnorm(simple_item_stats$`FALSE`$mean, mean = item_stats$difficulty)
item_stats$latent_d = item_stats$Russian_z - item_stats$minority_z

#observed
describeBy(d_LSM[c("g_IRT", "g_sumscore")], d_LSM$minority, mat = T)
#Spearman law
#IRT version
GG_scatter(item_stats, "loading", "latent_d")

#CTT version
GG_scatter(item_stats, "item_whole", "pass_rate_d")

#regression
rms::ols(latent_d ~ difficulty, data = item_stats)
## Linear Regression Model
##  
##  rms::ols(formula = latent_d ~ difficulty, data = item_stats)
##  
##                 Model Likelihood     Discrimination    
##                    Ratio Test           Indexes        
##  Obs      60    LR chi2     22.10    R2       0.308    
##  sigma0.3307    d.f.            1    R2 adj   0.296    
##  d.f.     58    Pr(> chi2) 0.0000    g        0.252    
##  
##  Residuals
##  
##       Min       1Q   Median       3Q      Max 
##  -0.80531 -0.25223  0.02526  0.27613  0.60476 
##  
##  
##             Coef    S.E.   t     Pr(>|t|)
##  Intercept   0.5751 0.0453 12.69 <0.0001 
##  difficulty -0.1483 0.0292 -5.08 <0.0001 
## 
rms::ols(latent_d ~ loading, data = item_stats)
## Linear Regression Model
##  
##  rms::ols(formula = latent_d ~ loading, data = item_stats)
##  
##                 Model Likelihood     Discrimination    
##                    Ratio Test           Indexes        
##  Obs      60    LR chi2     66.68    R2       0.671    
##  sigma0.2281    d.f.            1    R2 adj   0.665    
##  d.f.     58    Pr(> chi2) 0.0000    g        0.355    
##  
##  Residuals
##  
##       Min       1Q   Median       3Q      Max 
##  -0.47698 -0.18429  0.02911  0.16202  0.40885 
##  
##  
##            Coef    S.E.   t     Pr(>|t|)
##  Intercept -0.1624 0.0805 -2.02 0.0483  
##  loading    1.6608 0.1527 10.87 <0.0001 
## 
rms::ols(latent_d ~ difficulty + loading, data = item_stats)
## Linear Regression Model
##  
##  rms::ols(formula = latent_d ~ difficulty + loading, data = item_stats)
##  
##                 Model Likelihood     Discrimination    
##                    Ratio Test           Indexes        
##  Obs      60    LR chi2     67.45    R2       0.675    
##  sigma0.2286    d.f.            2    R2 adj   0.664    
##  d.f.     57    Pr(> chi2) 0.0000    g        0.354    
##  
##  Residuals
##  
##       Min       1Q   Median       3Q      Max 
##  -0.43281 -0.15627  0.03241  0.16840  0.47548 
##  
##  
##             Coef    S.E.   t     Pr(>|t|)
##  Intercept  -0.2183 0.1037 -2.10 0.0397  
##  difficulty  0.0254 0.0296  0.86 0.3944  
##  loading     1.8017 0.2246  8.02 <0.0001 
## 
rms::ols(latent_d ~ difficulty + loading + item_number, data = item_stats)
## Linear Regression Model
##  
##  rms::ols(formula = latent_d ~ difficulty + loading + item_number, 
##      data = item_stats)
##  
##                 Model Likelihood     Discrimination    
##                    Ratio Test           Indexes        
##  Obs      60    LR chi2     67.49    R2       0.675    
##  sigma0.2306    d.f.            3    R2 adj   0.658    
##  d.f.     56    Pr(> chi2) 0.0000    g        0.353    
##  
##  Residuals
##  
##       Min       1Q   Median       3Q      Max 
##  -0.43508 -0.15642  0.02816  0.16082  0.46856 
##  
##  
##              Coef    S.E.   t     Pr(>|t|)
##  Intercept   -0.1914 0.1723 -1.11 0.2714  
##  difficulty   0.0353 0.0584  0.60 0.5481  
##  loading      1.8097 0.2300  7.87 <0.0001 
##  item_number -0.0008 0.0043 -0.20 0.8446  
## 
#cors among stats
wtd.cors(item_stats[-2])
##                item_number pass_rate difficulty discrimination loading
## item_number         1.0000    -0.915      0.913         -0.591  -0.619
## pass_rate          -0.9151     1.000     -0.985          0.762   0.759
## difficulty          0.9131    -0.985      1.000         -0.743  -0.732
## discrimination     -0.5914     0.762     -0.743          1.000   0.980
## loading            -0.6193     0.759     -0.732          0.980   1.000
## item_whole         -0.0355     0.192     -0.111          0.677   0.715
## pass_rate_d        -0.2221     0.414     -0.302          0.696   0.689
## minority_z          0.6141    -0.787      0.770         -0.932  -0.892
## Russian_z           0.5799    -0.723      0.783         -0.699  -0.613
## latent_d           -0.4696     0.614     -0.555          0.820   0.819
##                item_whole pass_rate_d minority_z Russian_z latent_d
## item_number       -0.0355      -0.222      0.614    0.5799   -0.470
## pass_rate          0.1924       0.414     -0.787   -0.7227    0.614
## difficulty        -0.1111      -0.302      0.770    0.7827   -0.555
## discrimination     0.6771       0.696     -0.932   -0.6991    0.820
## loading            0.7153       0.689     -0.892   -0.6128    0.819
## item_whole         1.0000       0.805     -0.532   -0.0801    0.658
## pass_rate_d        0.8048       1.000     -0.720   -0.1531    0.864
## minority_z        -0.5323      -0.720      1.000    0.7053   -0.907
## Russian_z         -0.0801      -0.153      0.705    1.0000   -0.341
## latent_d           0.6584       0.864     -0.907   -0.3406    1.000

Yakut vs. Russian

SH does not work here easily because power is low when group gap is small.

#Yakut vs. Russian
minority_groups = c("Yakutsk")
d2 = d %>% filter(Region %in% c("Yakutsk", "Tomsk"))
d2$minority = d2$Region %in% minority_groups
simple_item_stats = describeBy(d2[cog_items_names], d2$minority)
item_stats$pass_rate_d = simple_item_stats$`FALSE`$mean - simple_item_stats$`TRUE`$mean
item_stats$minority_z = qnorm(simple_item_stats$`TRUE`$mean, mean = item_stats$difficulty) %>% INF_to_NA()
item_stats$Russian_z = qnorm(simple_item_stats$`FALSE`$mean, mean = item_stats$difficulty)
item_stats$latent_d = item_stats$Russian_z - item_stats$minority_z

#observed
describeBy(d2[c("g_IRT", "g_sumscore")], d2$minority, mat = T)
#Spearman law
#IRT version
GG_scatter(item_stats, "loading", "latent_d")

#CTT version
GG_scatter(item_stats, "item_whole", "pass_rate_d")

#regression
rms::ols(latent_d ~ difficulty, data = item_stats)
## Linear Regression Model
##  
##  rms::ols(formula = latent_d ~ difficulty, data = item_stats)
##  
##                 Model Likelihood     Discrimination    
##                    Ratio Test           Indexes        
##  Obs      60    LR chi2      8.88    R2       0.138    
##  sigma0.1782    d.f.            1    R2 adj   0.123    
##  d.f.     58    Pr(> chi2) 0.0029    g        0.081    
##  
##  Residuals
##  
##       Min       1Q   Median       3Q      Max 
##  -0.48204 -0.09037 -0.02939  0.08336  0.64988 
##  
##  
##             Coef    S.E.   t     Pr(>|t|)
##  Intercept  -0.1297 0.0244 -5.32 <0.0001 
##  difficulty -0.0478 0.0157 -3.04 0.0035  
## 
rms::ols(latent_d ~ loading, data = item_stats)
## Linear Regression Model
##  
##  rms::ols(formula = latent_d ~ loading, data = item_stats)
##  
##                 Model Likelihood     Discrimination    
##                    Ratio Test           Indexes        
##  Obs      60    LR chi2      1.40    R2       0.023    
##  sigma0.1896    d.f.            1    R2 adj   0.006    
##  d.f.     58    Pr(> chi2) 0.2362    g        0.032    
##  
##  Residuals
##  
##       Min       1Q   Median       3Q      Max 
##  -0.38581 -0.10650 -0.03784  0.12176  0.75555 
##  
##  
##            Coef    S.E.   t     Pr(>|t|)
##  Intercept -0.1778 0.0669 -2.66 0.0101  
##  loading    0.1488 0.1270  1.17 0.2462  
## 
rms::ols(latent_d ~ difficulty + loading, data = item_stats)
## Linear Regression Model
##  
##  rms::ols(formula = latent_d ~ difficulty + loading, data = item_stats)
##  
##                 Model Likelihood     Discrimination    
##                    Ratio Test           Indexes        
##  Obs      60    LR chi2     11.05    R2       0.168    
##  sigma0.1765    d.f.            2    R2 adj   0.139    
##  d.f.     57    Pr(> chi2) 0.0040    g        0.090    
##  
##  Residuals
##  
##       Min       1Q   Median       3Q      Max 
##  -0.52361 -0.07909 -0.03908  0.10106  0.56648 
##  
##  
##             Coef    S.E.   t     Pr(>|t|)
##  Intercept  -0.0191 0.0801 -0.24 0.8124  
##  difficulty -0.0721 0.0229 -3.15 0.0026  
##  loading    -0.2512 0.1734 -1.45 0.1527  
## 
rms::ols(latent_d ~ difficulty + loading + item_number, data = item_stats)
## Linear Regression Model
##  
##  rms::ols(formula = latent_d ~ difficulty + loading + item_number, 
##      data = item_stats)
##  
##                 Model Likelihood     Discrimination    
##                    Ratio Test           Indexes        
##  Obs      60    LR chi2     11.89    R2       0.180    
##  sigma0.1768    d.f.            3    R2 adj   0.136    
##  d.f.     56    Pr(> chi2) 0.0078    g        0.092    
##  
##  Residuals
##  
##       Min       1Q   Median       3Q      Max 
##  -0.53474 -0.07878 -0.03496  0.08747  0.54264 
##  
##  
##              Coef    S.E.   t     Pr(>|t|)
##  Intercept    0.0738 0.1321  0.56 0.5787  
##  difficulty  -0.0380 0.0448 -0.85 0.3993  
##  loading     -0.2239 0.1764 -1.27 0.2096  
##  item_number -0.0029 0.0033 -0.88 0.3800  
## 
#cors among stats
wtd.cors(item_stats[-2])
##                item_number pass_rate difficulty discrimination loading
## item_number         1.0000    -0.915      0.913         -0.591  -0.619
## pass_rate          -0.9151     1.000     -0.985          0.762   0.759
## difficulty          0.9131    -0.985      1.000         -0.743  -0.732
## discrimination     -0.5914     0.762     -0.743          1.000   0.980
## loading            -0.6193     0.759     -0.732          0.980   1.000
## item_whole         -0.0355     0.192     -0.111          0.677   0.715
## pass_rate_d        -0.4772     0.446     -0.453          0.231   0.171
## minority_z          0.6969    -0.803      0.839         -0.663  -0.570
## Russian_z           0.5799    -0.723      0.783         -0.699  -0.613
## latent_d           -0.3942     0.387     -0.371          0.193   0.152
##                item_whole pass_rate_d minority_z Russian_z latent_d
## item_number       -0.0355      -0.477      0.697   0.57993 -0.39422
## pass_rate          0.1924       0.446     -0.803  -0.72270  0.38668
## difficulty        -0.1111      -0.453      0.839   0.78274 -0.37097
## discrimination     0.6771       0.231     -0.663  -0.69914  0.19301
## loading            0.7153       0.171     -0.570  -0.61285  0.15204
## item_whole         1.0000      -0.270      0.017  -0.08007 -0.12570
## pass_rate_d       -0.2696       1.000     -0.711  -0.26651  0.80330
## minority_z         0.0170      -0.711      1.000   0.77795 -0.63257
## Russian_z         -0.0801      -0.267      0.778   1.00000 -0.00547
## latent_d          -0.1257       0.803     -0.633  -0.00547  1.00000

mirt package

Regular DIF style analysis

#regular fit joint group
mirt_fit = mirt(d[cog_items_names], 1)
## 
Iteration: 1, Log-Lik: -32440.781, Max-Change: 1.77202
Iteration: 2, Log-Lik: -31859.831, Max-Change: 0.53407
Iteration: 3, Log-Lik: -31814.846, Max-Change: 0.28881
Iteration: 4, Log-Lik: -31800.473, Max-Change: 0.14300
Iteration: 5, Log-Lik: -31793.851, Max-Change: 0.10904
Iteration: 6, Log-Lik: -31790.276, Max-Change: 0.05652
Iteration: 7, Log-Lik: -31789.100, Max-Change: 0.03766
Iteration: 8, Log-Lik: -31787.807, Max-Change: 0.02850
Iteration: 9, Log-Lik: -31787.030, Max-Change: 0.02191
Iteration: 10, Log-Lik: -31786.393, Max-Change: 0.01301
Iteration: 11, Log-Lik: -31786.187, Max-Change: 0.00933
Iteration: 12, Log-Lik: -31786.061, Max-Change: 0.00768
Iteration: 13, Log-Lik: -31785.802, Max-Change: 0.00717
Iteration: 14, Log-Lik: -31785.773, Max-Change: 0.00312
Iteration: 15, Log-Lik: -31785.755, Max-Change: 0.00268
Iteration: 16, Log-Lik: -31785.734, Max-Change: 0.00201
Iteration: 17, Log-Lik: -31785.722, Max-Change: 0.00157
Iteration: 18, Log-Lik: -31785.712, Max-Change: 0.00205
Iteration: 19, Log-Lik: -31785.695, Max-Change: 0.00237
Iteration: 20, Log-Lik: -31785.687, Max-Change: 0.00077
Iteration: 21, Log-Lik: -31785.684, Max-Change: 0.00053
Iteration: 22, Log-Lik: -31785.678, Max-Change: 0.00041
Iteration: 23, Log-Lik: -31785.677, Max-Change: 0.00028
Iteration: 24, Log-Lik: -31785.676, Max-Change: 0.00017
Iteration: 25, Log-Lik: -31785.675, Max-Change: 0.00070
Iteration: 26, Log-Lik: -31785.673, Max-Change: 0.00113
Iteration: 27, Log-Lik: -31785.672, Max-Change: 0.00120
Iteration: 28, Log-Lik: -31785.672, Max-Change: 0.00043
Iteration: 29, Log-Lik: -31785.672, Max-Change: 0.00113
Iteration: 30, Log-Lik: -31785.671, Max-Change: 0.00138
Iteration: 31, Log-Lik: -31785.671, Max-Change: 0.00036
Iteration: 32, Log-Lik: -31785.671, Max-Change: 0.00093
Iteration: 33, Log-Lik: -31785.670, Max-Change: 0.00117
Iteration: 34, Log-Lik: -31785.670, Max-Change: 0.00029
Iteration: 35, Log-Lik: -31785.670, Max-Change: 0.00075
Iteration: 36, Log-Lik: -31785.670, Max-Change: 0.00096
Iteration: 37, Log-Lik: -31785.670, Max-Change: 0.00024
Iteration: 38, Log-Lik: -31785.670, Max-Change: 0.00061
Iteration: 39, Log-Lik: -31785.670, Max-Change: 0.00079
Iteration: 40, Log-Lik: -31785.670, Max-Change: 0.00019
Iteration: 41, Log-Lik: -31785.670, Max-Change: 0.00049
Iteration: 42, Log-Lik: -31785.670, Max-Change: 0.00065
Iteration: 43, Log-Lik: -31785.670, Max-Change: 0.00016
Iteration: 44, Log-Lik: -31785.669, Max-Change: 0.00041
Iteration: 45, Log-Lik: -31785.669, Max-Change: 0.00053
Iteration: 46, Log-Lik: -31785.669, Max-Change: 0.00013
Iteration: 47, Log-Lik: -31785.669, Max-Change: 0.00034
Iteration: 48, Log-Lik: -31785.669, Max-Change: 0.00045
Iteration: 49, Log-Lik: -31785.669, Max-Change: 0.00057
Iteration: 50, Log-Lik: -31785.669, Max-Change: 0.00017
Iteration: 51, Log-Lik: -31785.669, Max-Change: 0.00038
Iteration: 52, Log-Lik: -31785.669, Max-Change: 0.00023
Iteration: 53, Log-Lik: -31785.669, Max-Change: 0.00012
Iteration: 54, Log-Lik: -31785.669, Max-Change: 0.00029
Iteration: 55, Log-Lik: -31785.669, Max-Change: 0.00014
Iteration: 56, Log-Lik: -31785.669, Max-Change: 0.00031
Iteration: 57, Log-Lik: -31785.669, Max-Change: 0.00011
Iteration: 58, Log-Lik: -31785.669, Max-Change: 0.00010
mirt_fit
## 
## Call:
## mirt(data = d[cog_items_names], model = 1)
## 
## Full-information item factor analysis with 1 factor(s).
## Converged within 1e-04 tolerance after 58 EM iterations.
## mirt version: 1.30 
## M-step optimizer: BFGS 
## EM acceleration: Ramsay 
## Number of rectangular quadrature: 61
## Latent density type: Gaussian 
## 
## Log-likelihood = -31786
## Estimated parameters: 120 
## AIC = 63811; AICc = 63832
## BIC = 64454; SABIC = 64072
## G2 (1e+10) = 40673, p = 1
## RMSEA = 0, CFI = NaN, TLI = NaN
mirt_fit_sum = mirt_fit %>% summary()
##          F1      h2
## A1   0.3838 0.14728
## A2   0.6245 0.39001
## A3   0.4738 0.22444
## A4   0.7068 0.49954
## A5   0.5904 0.34860
## A6   0.5866 0.34411
## A7   0.5897 0.34778
## A8   0.6073 0.36875
## A9   0.7672 0.58864
## A10  0.6314 0.39862
## A11  0.5185 0.26880
## A12  0.4795 0.22993
## B1   0.4917 0.24179
## B2   0.6502 0.42271
## B3   0.7486 0.56033
## B4   0.7114 0.50608
## B5   0.7882 0.62128
## B6   0.6472 0.41890
## B7   0.5724 0.32763
## B8   0.7579 0.57445
## B9   0.7258 0.52678
## B10  0.8177 0.66861
## B11  0.5705 0.32546
## B12  0.6329 0.40056
## C1   0.7792 0.60715
## C2   0.7008 0.49113
## C3   0.5779 0.33392
## C4   0.7694 0.59197
## C5   0.6344 0.40241
## C6   0.7507 0.56362
## C7   0.5193 0.26962
## C8   0.7016 0.49221
## C9   0.5044 0.25442
## C10  0.2171 0.04715
## C11  0.4946 0.24462
## C12  0.5339 0.28506
## D1   0.7190 0.51698
## D2   0.7303 0.53336
## D3   0.6265 0.39251
## D4   0.5069 0.25690
## D5   0.5827 0.33949
## D6   0.4511 0.20353
## D7   0.5583 0.31172
## D8   0.4926 0.24263
## D9   0.3258 0.10617
## D10  0.5513 0.30396
## D11  0.6081 0.36977
## D12  0.4722 0.22298
## E1   0.5503 0.30283
## E2   0.5766 0.33245
## E3   0.5072 0.25721
## E4   0.6401 0.40967
## E5   0.5166 0.26689
## E6   0.1264 0.01598
## E7   0.2785 0.07754
## E8   0.3885 0.15090
## E9   0.3258 0.10617
## E10  0.0668 0.00446
## E11  0.2047 0.04188
## E12 -0.0926 0.00857
## 
## SS loadings:  20.1 
## Proportion Var:  0.335 
## 
## Factor correlations: 
## 
##    F1
## F1  1
item_stats$loading_mirt = mirt_fit_sum$rotF[, 1]

#compare loadings from mirt and psych
GG_scatter(item_stats, "loading", "loading_mirt")

## 2 group fits
d2 = d %>% filter(Region %in% c("Yakutsk", "Tomsk"))
mirt_fit_groups_2_step1 = multipleGroup(d2[cog_items_names], model = 1, group = d2$Region, invariance = "")
## 
Iteration: 1, Log-Lik: -30254.949, Max-Change: 1.11777
Iteration: 2, Log-Lik: -29669.539, Max-Change: 0.36889
Iteration: 3, Log-Lik: -29621.660, Max-Change: 0.21700
Iteration: 4, Log-Lik: -29605.414, Max-Change: 0.15699
Iteration: 5, Log-Lik: -29596.809, Max-Change: 0.08967
Iteration: 6, Log-Lik: -29591.686, Max-Change: 0.05856
Iteration: 7, Log-Lik: -29588.596, Max-Change: 0.03621
Iteration: 8, Log-Lik: -29586.660, Max-Change: 0.02484
Iteration: 9, Log-Lik: -29585.370, Max-Change: 0.02088
Iteration: 10, Log-Lik: -29582.707, Max-Change: 0.00821
Iteration: 11, Log-Lik: -29582.528, Max-Change: 0.00623
Iteration: 12, Log-Lik: -29582.392, Max-Change: 0.00534
Iteration: 13, Log-Lik: -29582.023, Max-Change: 0.01016
Iteration: 14, Log-Lik: -29581.982, Max-Change: 0.00273
Iteration: 15, Log-Lik: -29581.958, Max-Change: 0.00213
Iteration: 16, Log-Lik: -29581.878, Max-Change: 0.00517
Iteration: 17, Log-Lik: -29581.871, Max-Change: 0.00222
Iteration: 18, Log-Lik: -29581.866, Max-Change: 0.00111
Iteration: 19, Log-Lik: -29581.862, Max-Change: 0.00104
Iteration: 20, Log-Lik: -29581.859, Max-Change: 0.00096
Iteration: 21, Log-Lik: -29581.857, Max-Change: 0.00095
Iteration: 22, Log-Lik: -29581.850, Max-Change: 0.00158
Iteration: 23, Log-Lik: -29581.849, Max-Change: 0.00076
Iteration: 24, Log-Lik: -29581.849, Max-Change: 0.00070
Iteration: 25, Log-Lik: -29581.849, Max-Change: 0.00026
Iteration: 26, Log-Lik: -29581.848, Max-Change: 0.00026
Iteration: 27, Log-Lik: -29581.848, Max-Change: 0.00025
Iteration: 28, Log-Lik: -29581.848, Max-Change: 0.00055
Iteration: 29, Log-Lik: -29581.847, Max-Change: 0.00027
Iteration: 30, Log-Lik: -29581.847, Max-Change: 0.00024
Iteration: 31, Log-Lik: -29581.847, Max-Change: 0.00025
Iteration: 32, Log-Lik: -29581.847, Max-Change: 0.00116
Iteration: 33, Log-Lik: -29581.847, Max-Change: 0.00126
Iteration: 34, Log-Lik: -29581.847, Max-Change: 0.00044
Iteration: 35, Log-Lik: -29581.847, Max-Change: 0.00032
Iteration: 36, Log-Lik: -29581.847, Max-Change: 0.00108
Iteration: 37, Log-Lik: -29581.846, Max-Change: 0.00156
Iteration: 38, Log-Lik: -29581.846, Max-Change: 0.00096
Iteration: 39, Log-Lik: -29581.846, Max-Change: 0.00062
Iteration: 40, Log-Lik: -29581.846, Max-Change: 0.00101
Iteration: 41, Log-Lik: -29581.846, Max-Change: 0.00092
Iteration: 42, Log-Lik: -29581.846, Max-Change: 0.00059
Iteration: 43, Log-Lik: -29581.846, Max-Change: 0.00096
Iteration: 44, Log-Lik: -29581.846, Max-Change: 0.00096
Iteration: 45, Log-Lik: -29581.846, Max-Change: 0.00061
Iteration: 46, Log-Lik: -29581.846, Max-Change: 0.00091
Iteration: 47, Log-Lik: -29581.846, Max-Change: 0.00114
Iteration: 48, Log-Lik: -29581.846, Max-Change: 0.00073
Iteration: 49, Log-Lik: -29581.846, Max-Change: 0.00023
Iteration: 50, Log-Lik: -29581.846, Max-Change: 0.00086
Iteration: 51, Log-Lik: -29581.846, Max-Change: 0.00100
Iteration: 52, Log-Lik: -29581.846, Max-Change: 0.00022
Iteration: 53, Log-Lik: -29581.846, Max-Change: 0.00081
Iteration: 54, Log-Lik: -29581.846, Max-Change: 0.00096
Iteration: 55, Log-Lik: -29581.845, Max-Change: 0.00020
Iteration: 56, Log-Lik: -29581.845, Max-Change: 0.00077
Iteration: 57, Log-Lik: -29581.845, Max-Change: 0.00087
Iteration: 58, Log-Lik: -29581.845, Max-Change: 0.00018
Iteration: 59, Log-Lik: -29581.845, Max-Change: 0.00073
Iteration: 60, Log-Lik: -29581.845, Max-Change: 0.00080
Iteration: 61, Log-Lik: -29581.845, Max-Change: 0.00017
Iteration: 62, Log-Lik: -29581.845, Max-Change: 0.00070
Iteration: 63, Log-Lik: -29581.845, Max-Change: 0.00074
Iteration: 64, Log-Lik: -29581.845, Max-Change: 0.00015
Iteration: 65, Log-Lik: -29581.845, Max-Change: 0.00066
Iteration: 66, Log-Lik: -29581.845, Max-Change: 0.00068
Iteration: 67, Log-Lik: -29581.845, Max-Change: 0.00014
Iteration: 68, Log-Lik: -29581.845, Max-Change: 0.00063
Iteration: 69, Log-Lik: -29581.845, Max-Change: 0.00064
Iteration: 70, Log-Lik: -29581.845, Max-Change: 0.00013
Iteration: 71, Log-Lik: -29581.845, Max-Change: 0.00060
Iteration: 72, Log-Lik: -29581.845, Max-Change: 0.00059
Iteration: 73, Log-Lik: -29581.845, Max-Change: 0.00013
Iteration: 74, Log-Lik: -29581.845, Max-Change: 0.00057
Iteration: 75, Log-Lik: -29581.845, Max-Change: 0.00056
Iteration: 76, Log-Lik: -29581.845, Max-Change: 0.00012
Iteration: 77, Log-Lik: -29581.845, Max-Change: 0.00054
Iteration: 78, Log-Lik: -29581.845, Max-Change: 0.00052
Iteration: 79, Log-Lik: -29581.845, Max-Change: 0.00011
Iteration: 80, Log-Lik: -29581.845, Max-Change: 0.00051
Iteration: 81, Log-Lik: -29581.845, Max-Change: 0.00049
Iteration: 82, Log-Lik: -29581.845, Max-Change: 0.00010
Iteration: 83, Log-Lik: -29581.845, Max-Change: 0.00049
Iteration: 84, Log-Lik: -29581.845, Max-Change: 0.00046
Iteration: 85, Log-Lik: -29581.845, Max-Change: 0.00010
mirt_fit_groups_2_step2 = multipleGroup(d2[cog_items_names], model = 1, group = d2$Region, invariance = c("slopes", "intercepts"))
## 
Iteration: 1, Log-Lik: -30254.949, Max-Change: 1.90934
Iteration: 2, Log-Lik: -29800.482, Max-Change: 0.41487
Iteration: 3, Log-Lik: -29772.207, Max-Change: 0.13595
Iteration: 4, Log-Lik: -29763.310, Max-Change: 0.08725
Iteration: 5, Log-Lik: -29759.065, Max-Change: 0.05099
Iteration: 6, Log-Lik: -29756.866, Max-Change: 0.03749
Iteration: 7, Log-Lik: -29755.633, Max-Change: 0.02506
Iteration: 8, Log-Lik: -29755.016, Max-Change: 0.01566
Iteration: 9, Log-Lik: -29754.724, Max-Change: 0.01251
Iteration: 10, Log-Lik: -29754.390, Max-Change: 0.00959
Iteration: 11, Log-Lik: -29754.315, Max-Change: 0.00615
Iteration: 12, Log-Lik: -29754.267, Max-Change: 0.00487
Iteration: 13, Log-Lik: -29754.216, Max-Change: 0.00261
Iteration: 14, Log-Lik: -29754.202, Max-Change: 0.00225
Iteration: 15, Log-Lik: -29754.190, Max-Change: 0.00246
Iteration: 16, Log-Lik: -29754.185, Max-Change: 0.00313
Iteration: 17, Log-Lik: -29754.174, Max-Change: 0.00246
Iteration: 18, Log-Lik: -29754.168, Max-Change: 0.00160
Iteration: 19, Log-Lik: -29754.166, Max-Change: 0.00141
Iteration: 20, Log-Lik: -29754.163, Max-Change: 0.00109
Iteration: 21, Log-Lik: -29754.161, Max-Change: 0.00099
Iteration: 22, Log-Lik: -29754.160, Max-Change: 0.00077
Iteration: 23, Log-Lik: -29754.159, Max-Change: 0.00057
Iteration: 24, Log-Lik: -29754.159, Max-Change: 0.00089
Iteration: 25, Log-Lik: -29754.158, Max-Change: 0.00118
Iteration: 26, Log-Lik: -29754.158, Max-Change: 0.00041
Iteration: 27, Log-Lik: -29754.158, Max-Change: 0.00108
Iteration: 28, Log-Lik: -29754.158, Max-Change: 0.00046
Iteration: 29, Log-Lik: -29754.158, Max-Change: 0.00112
Iteration: 30, Log-Lik: -29754.158, Max-Change: 0.00037
Iteration: 31, Log-Lik: -29754.158, Max-Change: 0.00033
Iteration: 32, Log-Lik: -29754.158, Max-Change: 0.00017
Iteration: 33, Log-Lik: -29754.158, Max-Change: 0.00047
Iteration: 34, Log-Lik: -29754.157, Max-Change: 0.00018
Iteration: 35, Log-Lik: -29754.157, Max-Change: 0.00041
Iteration: 36, Log-Lik: -29754.157, Max-Change: 0.00015
Iteration: 37, Log-Lik: -29754.157, Max-Change: 0.00013
Iteration: 38, Log-Lik: -29754.157, Max-Change: 0.00036
Iteration: 39, Log-Lik: -29754.157, Max-Change: 0.00042
Iteration: 40, Log-Lik: -29754.157, Max-Change: 0.00049
Iteration: 41, Log-Lik: -29754.157, Max-Change: 0.00013
Iteration: 42, Log-Lik: -29754.157, Max-Change: 0.00031
Iteration: 43, Log-Lik: -29754.157, Max-Change: 0.00018
Iteration: 44, Log-Lik: -29754.157, Max-Change: 0.00009
mirt_fit_groups_2_step3 = multipleGroup(d2[cog_items_names], model = 1, group = d2$Region, invariance = c('intercepts','slopes', 'free_means', 'free_var'))
## 
Iteration: 1, Log-Lik: -30254.949, Max-Change: 1.34550
Iteration: 2, Log-Lik: -29785.130, Max-Change: 0.23934
Iteration: 3, Log-Lik: -29759.702, Max-Change: 0.10238
Iteration: 4, Log-Lik: -29754.554, Max-Change: 0.05486
Iteration: 5, Log-Lik: -29751.898, Max-Change: 0.03522
Iteration: 6, Log-Lik: -29750.043, Max-Change: 0.02984
Iteration: 7, Log-Lik: -29746.584, Max-Change: 0.06741
Iteration: 8, Log-Lik: -29745.747, Max-Change: 0.02440
Iteration: 9, Log-Lik: -29745.238, Max-Change: 0.01617
Iteration: 10, Log-Lik: -29743.860, Max-Change: 0.05776
Iteration: 11, Log-Lik: -29743.466, Max-Change: 0.01230
Iteration: 12, Log-Lik: -29743.388, Max-Change: 0.00545
Iteration: 13, Log-Lik: -29743.216, Max-Change: 0.01078
Iteration: 14, Log-Lik: -29743.177, Max-Change: 0.00352
Iteration: 15, Log-Lik: -29743.152, Max-Change: 0.00232
Iteration: 16, Log-Lik: -29743.066, Max-Change: 0.00865
Iteration: 17, Log-Lik: -29743.049, Max-Change: 0.00190
Iteration: 18, Log-Lik: -29743.043, Max-Change: 0.00130
Iteration: 19, Log-Lik: -29743.024, Max-Change: 0.00294
Iteration: 20, Log-Lik: -29743.020, Max-Change: 0.00065
Iteration: 21, Log-Lik: -29743.019, Max-Change: 0.00066
Iteration: 22, Log-Lik: -29743.014, Max-Change: 0.00148
Iteration: 23, Log-Lik: -29743.013, Max-Change: 0.00036
Iteration: 24, Log-Lik: -29743.012, Max-Change: 0.00035
Iteration: 25, Log-Lik: -29743.011, Max-Change: 0.00080
Iteration: 26, Log-Lik: -29743.011, Max-Change: 0.00017
Iteration: 27, Log-Lik: -29743.011, Max-Change: 0.00018
Iteration: 28, Log-Lik: -29743.010, Max-Change: 0.00042
Iteration: 29, Log-Lik: -29743.010, Max-Change: 0.00009
mirt_fit_groups_2_step4 = multipleGroup(d2[cog_items_names], model = 1, group = d2$Region, SE = TRUE,
                           invariance = c('free_means', 'free_var'))
## Warning: Multiple-group model may not be identified without providing
## anchor items
## 
Iteration: 1, Log-Lik: -30254.949, Max-Change: 1.36373
Iteration: 2, Log-Lik: -29631.857, Max-Change: 0.28203
Iteration: 3, Log-Lik: -29601.688, Max-Change: 0.13619
Iteration: 4, Log-Lik: -29593.898, Max-Change: 0.08377
Iteration: 5, Log-Lik: -29589.957, Max-Change: 0.05821
Iteration: 6, Log-Lik: -29587.544, Max-Change: 0.04150
Iteration: 7, Log-Lik: -29584.418, Max-Change: 0.01991
Iteration: 8, Log-Lik: -29583.802, Max-Change: 0.01729
Iteration: 9, Log-Lik: -29583.352, Max-Change: 0.01373
Iteration: 10, Log-Lik: -29582.340, Max-Change: 0.00642
Iteration: 11, Log-Lik: -29582.246, Max-Change: 0.00527
Iteration: 12, Log-Lik: -29582.172, Max-Change: 0.00467
Iteration: 13, Log-Lik: -29581.929, Max-Change: 0.00286
Iteration: 14, Log-Lik: -29581.916, Max-Change: 0.00242
Iteration: 15, Log-Lik: -29581.906, Max-Change: 0.00219
Iteration: 16, Log-Lik: -29581.868, Max-Change: 0.00098
Iteration: 17, Log-Lik: -29581.867, Max-Change: 0.00078
Iteration: 18, Log-Lik: -29581.865, Max-Change: 0.00071
Iteration: 19, Log-Lik: -29581.861, Max-Change: 0.00095
Iteration: 20, Log-Lik: -29581.861, Max-Change: 0.00026
Iteration: 21, Log-Lik: -29581.861, Max-Change: 0.00025
Iteration: 22, Log-Lik: -29581.860, Max-Change: 0.00095
Iteration: 23, Log-Lik: -29581.860, Max-Change: 0.00024
Iteration: 24, Log-Lik: -29581.860, Max-Change: 0.00021
Iteration: 25, Log-Lik: -29581.860, Max-Change: 0.00080
Iteration: 26, Log-Lik: -29581.860, Max-Change: 0.00024
Iteration: 27, Log-Lik: -29581.860, Max-Change: 0.00020
Iteration: 28, Log-Lik: -29581.860, Max-Change: 0.00092
Iteration: 29, Log-Lik: -29581.860, Max-Change: 0.00024
Iteration: 30, Log-Lik: -29581.860, Max-Change: 0.00020
Iteration: 31, Log-Lik: -29581.860, Max-Change: 0.00091
Iteration: 32, Log-Lik: -29581.859, Max-Change: 0.00024
Iteration: 33, Log-Lik: -29581.859, Max-Change: 0.00020
Iteration: 34, Log-Lik: -29581.859, Max-Change: 0.00090
Iteration: 35, Log-Lik: -29581.859, Max-Change: 0.00024
Iteration: 36, Log-Lik: -29581.859, Max-Change: 0.00018
Iteration: 37, Log-Lik: -29581.859, Max-Change: 0.00054
Iteration: 38, Log-Lik: -29581.859, Max-Change: 0.00021
Iteration: 39, Log-Lik: -29581.859, Max-Change: 0.00019
Iteration: 40, Log-Lik: -29581.859, Max-Change: 0.00089
Iteration: 41, Log-Lik: -29581.859, Max-Change: 0.00022
Iteration: 42, Log-Lik: -29581.859, Max-Change: 0.00020
Iteration: 43, Log-Lik: -29581.859, Max-Change: 0.00086
Iteration: 44, Log-Lik: -29581.859, Max-Change: 0.00023
Iteration: 45, Log-Lik: -29581.859, Max-Change: 0.00019
Iteration: 46, Log-Lik: -29581.859, Max-Change: 0.00085
Iteration: 47, Log-Lik: -29581.859, Max-Change: 0.00023
Iteration: 48, Log-Lik: -29581.859, Max-Change: 0.00019
Iteration: 49, Log-Lik: -29581.859, Max-Change: 0.00089
Iteration: 50, Log-Lik: -29581.859, Max-Change: 0.00024
Iteration: 51, Log-Lik: -29581.859, Max-Change: 0.00018
Iteration: 52, Log-Lik: -29581.859, Max-Change: 0.00067
Iteration: 53, Log-Lik: -29581.859, Max-Change: 0.00021
Iteration: 54, Log-Lik: -29581.858, Max-Change: 0.00018
Iteration: 55, Log-Lik: -29581.858, Max-Change: 0.00057
Iteration: 56, Log-Lik: -29581.858, Max-Change: 0.00019
Iteration: 57, Log-Lik: -29581.858, Max-Change: 0.00018
Iteration: 58, Log-Lik: -29581.858, Max-Change: 0.00074
Iteration: 59, Log-Lik: -29581.858, Max-Change: 0.00023
Iteration: 60, Log-Lik: -29581.858, Max-Change: 0.00018
Iteration: 61, Log-Lik: -29581.858, Max-Change: 0.00062
Iteration: 62, Log-Lik: -29581.858, Max-Change: 0.00020
Iteration: 63, Log-Lik: -29581.858, Max-Change: 0.00018
Iteration: 64, Log-Lik: -29581.858, Max-Change: 0.00084
Iteration: 65, Log-Lik: -29581.858, Max-Change: 0.00021
Iteration: 66, Log-Lik: -29581.858, Max-Change: 0.00018
Iteration: 67, Log-Lik: -29581.858, Max-Change: 0.00057
Iteration: 68, Log-Lik: -29581.858, Max-Change: 0.00021
Iteration: 69, Log-Lik: -29581.858, Max-Change: 0.00017
Iteration: 70, Log-Lik: -29581.858, Max-Change: 0.00044
Iteration: 71, Log-Lik: -29581.858, Max-Change: 0.00019
Iteration: 72, Log-Lik: -29581.858, Max-Change: 0.00018
Iteration: 73, Log-Lik: -29581.858, Max-Change: 0.00081
Iteration: 74, Log-Lik: -29581.858, Max-Change: 0.00021
Iteration: 75, Log-Lik: -29581.858, Max-Change: 0.00017
Iteration: 76, Log-Lik: -29581.858, Max-Change: 0.00066
Iteration: 77, Log-Lik: -29581.858, Max-Change: 0.00020
Iteration: 78, Log-Lik: -29581.858, Max-Change: 0.00017
Iteration: 79, Log-Lik: -29581.858, Max-Change: 0.00079
Iteration: 80, Log-Lik: -29581.858, Max-Change: 0.00022
Iteration: 81, Log-Lik: -29581.857, Max-Change: 0.00018
Iteration: 82, Log-Lik: -29581.857, Max-Change: 0.00080
Iteration: 83, Log-Lik: -29581.857, Max-Change: 0.00020
Iteration: 84, Log-Lik: -29581.857, Max-Change: 0.00017
Iteration: 85, Log-Lik: -29581.857, Max-Change: 0.00078
Iteration: 86, Log-Lik: -29581.857, Max-Change: 0.00020
Iteration: 87, Log-Lik: -29581.857, Max-Change: 0.00016
Iteration: 88, Log-Lik: -29581.857, Max-Change: 0.00040
Iteration: 89, Log-Lik: -29581.857, Max-Change: 0.00018
Iteration: 90, Log-Lik: -29581.857, Max-Change: 0.00016
Iteration: 91, Log-Lik: -29581.857, Max-Change: 0.00063
Iteration: 92, Log-Lik: -29581.857, Max-Change: 0.00021
Iteration: 93, Log-Lik: -29581.857, Max-Change: 0.00015
Iteration: 94, Log-Lik: -29581.857, Max-Change: 0.00065
Iteration: 95, Log-Lik: -29581.857, Max-Change: 0.00017
Iteration: 96, Log-Lik: -29581.857, Max-Change: 0.00016
Iteration: 97, Log-Lik: -29581.857, Max-Change: 0.00073
Iteration: 98, Log-Lik: -29581.857, Max-Change: 0.00021
Iteration: 99, Log-Lik: -29581.857, Max-Change: 0.00016
Iteration: 100, Log-Lik: -29581.857, Max-Change: 0.00047
Iteration: 101, Log-Lik: -29581.857, Max-Change: 0.00018
Iteration: 102, Log-Lik: -29581.857, Max-Change: 0.00015
Iteration: 103, Log-Lik: -29581.857, Max-Change: 0.00076
Iteration: 104, Log-Lik: -29581.857, Max-Change: 0.00019
Iteration: 105, Log-Lik: -29581.857, Max-Change: 0.00015
Iteration: 106, Log-Lik: -29581.857, Max-Change: 0.00051
Iteration: 107, Log-Lik: -29581.857, Max-Change: 0.00019
Iteration: 108, Log-Lik: -29581.857, Max-Change: 0.00016
Iteration: 109, Log-Lik: -29581.857, Max-Change: 0.00048
Iteration: 110, Log-Lik: -29581.857, Max-Change: 0.00017
Iteration: 111, Log-Lik: -29581.857, Max-Change: 0.00016
Iteration: 112, Log-Lik: -29581.856, Max-Change: 0.00071
Iteration: 113, Log-Lik: -29581.856, Max-Change: 0.00019
Iteration: 114, Log-Lik: -29581.856, Max-Change: 0.00017
Iteration: 115, Log-Lik: -29581.856, Max-Change: 0.00073
Iteration: 116, Log-Lik: -29581.856, Max-Change: 0.00019
Iteration: 117, Log-Lik: -29581.856, Max-Change: 0.00014
Iteration: 118, Log-Lik: -29581.856, Max-Change: 0.00062
Iteration: 119, Log-Lik: -29581.856, Max-Change: 0.00017
Iteration: 120, Log-Lik: -29581.856, Max-Change: 0.00016
Iteration: 121, Log-Lik: -29581.856, Max-Change: 0.00070
Iteration: 122, Log-Lik: -29581.856, Max-Change: 0.00018
Iteration: 123, Log-Lik: -29581.856, Max-Change: 0.00015
Iteration: 124, Log-Lik: -29581.856, Max-Change: 0.00035
Iteration: 125, Log-Lik: -29581.856, Max-Change: 0.00016
Iteration: 126, Log-Lik: -29581.856, Max-Change: 0.00014
Iteration: 127, Log-Lik: -29581.856, Max-Change: 0.00072
Iteration: 128, Log-Lik: -29581.856, Max-Change: 0.00017
Iteration: 129, Log-Lik: -29581.856, Max-Change: 0.00015
Iteration: 130, Log-Lik: -29581.856, Max-Change: 0.00038
Iteration: 131, Log-Lik: -29581.856, Max-Change: 0.00015
Iteration: 132, Log-Lik: -29581.856, Max-Change: 0.00017
Iteration: 133, Log-Lik: -29581.856, Max-Change: 0.00051
Iteration: 134, Log-Lik: -29581.856, Max-Change: 0.00019
Iteration: 135, Log-Lik: -29581.856, Max-Change: 0.00014
Iteration: 136, Log-Lik: -29581.856, Max-Change: 0.00059
Iteration: 137, Log-Lik: -29581.856, Max-Change: 0.00017
Iteration: 138, Log-Lik: -29581.856, Max-Change: 0.00015
Iteration: 139, Log-Lik: -29581.856, Max-Change: 0.00048
Iteration: 140, Log-Lik: -29581.856, Max-Change: 0.00016
Iteration: 141, Log-Lik: -29581.856, Max-Change: 0.00015
Iteration: 142, Log-Lik: -29581.856, Max-Change: 0.00066
Iteration: 143, Log-Lik: -29581.856, Max-Change: 0.00017
Iteration: 144, Log-Lik: -29581.856, Max-Change: 0.00014
Iteration: 145, Log-Lik: -29581.856, Max-Change: 0.00038
Iteration: 146, Log-Lik: -29581.856, Max-Change: 0.00015
Iteration: 147, Log-Lik: -29581.856, Max-Change: 0.00014
Iteration: 148, Log-Lik: -29581.855, Max-Change: 0.00068
Iteration: 149, Log-Lik: -29581.855, Max-Change: 0.00016
Iteration: 150, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 151, Log-Lik: -29581.855, Max-Change: 0.00064
Iteration: 152, Log-Lik: -29581.855, Max-Change: 0.00018
Iteration: 153, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 154, Log-Lik: -29581.855, Max-Change: 0.00067
Iteration: 155, Log-Lik: -29581.855, Max-Change: 0.00017
Iteration: 156, Log-Lik: -29581.855, Max-Change: 0.00014
Iteration: 157, Log-Lik: -29581.855, Max-Change: 0.00030
Iteration: 158, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 159, Log-Lik: -29581.855, Max-Change: 0.00013
Iteration: 160, Log-Lik: -29581.855, Max-Change: 0.00050
Iteration: 161, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 162, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 163, Log-Lik: -29581.855, Max-Change: 0.00061
Iteration: 164, Log-Lik: -29581.855, Max-Change: 0.00018
Iteration: 165, Log-Lik: -29581.855, Max-Change: 0.00013
Iteration: 166, Log-Lik: -29581.855, Max-Change: 0.00034
Iteration: 167, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 168, Log-Lik: -29581.855, Max-Change: 0.00014
Iteration: 169, Log-Lik: -29581.855, Max-Change: 0.00043
Iteration: 170, Log-Lik: -29581.855, Max-Change: 0.00014
Iteration: 171, Log-Lik: -29581.855, Max-Change: 0.00014
Iteration: 172, Log-Lik: -29581.855, Max-Change: 0.00063
Iteration: 173, Log-Lik: -29581.855, Max-Change: 0.00017
Iteration: 174, Log-Lik: -29581.855, Max-Change: 0.00013
Iteration: 175, Log-Lik: -29581.855, Max-Change: 0.00033
Iteration: 176, Log-Lik: -29581.855, Max-Change: 0.00014
Iteration: 177, Log-Lik: -29581.855, Max-Change: 0.00013
Iteration: 178, Log-Lik: -29581.855, Max-Change: 0.00064
Iteration: 179, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 180, Log-Lik: -29581.855, Max-Change: 0.00013
Iteration: 181, Log-Lik: -29581.855, Max-Change: 0.00043
Iteration: 182, Log-Lik: -29581.855, Max-Change: 0.00016
Iteration: 183, Log-Lik: -29581.855, Max-Change: 0.00014
Iteration: 184, Log-Lik: -29581.855, Max-Change: 0.00045
Iteration: 185, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 186, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 187, Log-Lik: -29581.855, Max-Change: 0.00032
Iteration: 188, Log-Lik: -29581.855, Max-Change: 0.00015
Iteration: 189, Log-Lik: -29581.855, Max-Change: 0.00012
Iteration: 190, Log-Lik: -29581.855, Max-Change: 0.00044
Iteration: 191, Log-Lik: -29581.855, Max-Change: 0.00014
Iteration: 192, Log-Lik: -29581.855, Max-Change: 0.00013
Iteration: 193, Log-Lik: -29581.854, Max-Change: 0.00054
Iteration: 194, Log-Lik: -29581.854, Max-Change: 0.00016
Iteration: 195, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 196, Log-Lik: -29581.854, Max-Change: 0.00036
Iteration: 197, Log-Lik: -29581.854, Max-Change: 0.00014
Iteration: 198, Log-Lik: -29581.854, Max-Change: 0.00014
Iteration: 199, Log-Lik: -29581.854, Max-Change: 0.00049
Iteration: 200, Log-Lik: -29581.854, Max-Change: 0.00015
Iteration: 201, Log-Lik: -29581.854, Max-Change: 0.00013
Iteration: 202, Log-Lik: -29581.854, Max-Change: 0.00058
Iteration: 203, Log-Lik: -29581.854, Max-Change: 0.00015
Iteration: 204, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 205, Log-Lik: -29581.854, Max-Change: 0.00032
Iteration: 206, Log-Lik: -29581.854, Max-Change: 0.00013
Iteration: 207, Log-Lik: -29581.854, Max-Change: 0.00014
Iteration: 208, Log-Lik: -29581.854, Max-Change: 0.00048
Iteration: 209, Log-Lik: -29581.854, Max-Change: 0.00015
Iteration: 210, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 211, Log-Lik: -29581.854, Max-Change: 0.00057
Iteration: 212, Log-Lik: -29581.854, Max-Change: 0.00015
Iteration: 213, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 214, Log-Lik: -29581.854, Max-Change: 0.00031
Iteration: 215, Log-Lik: -29581.854, Max-Change: 0.00013
Iteration: 216, Log-Lik: -29581.854, Max-Change: 0.00014
Iteration: 217, Log-Lik: -29581.854, Max-Change: 0.00047
Iteration: 218, Log-Lik: -29581.854, Max-Change: 0.00015
Iteration: 219, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 220, Log-Lik: -29581.854, Max-Change: 0.00048
Iteration: 221, Log-Lik: -29581.854, Max-Change: 0.00015
Iteration: 222, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 223, Log-Lik: -29581.854, Max-Change: 0.00030
Iteration: 224, Log-Lik: -29581.854, Max-Change: 0.00013
Iteration: 225, Log-Lik: -29581.854, Max-Change: 0.00013
Iteration: 226, Log-Lik: -29581.854, Max-Change: 0.00046
Iteration: 227, Log-Lik: -29581.854, Max-Change: 0.00015
Iteration: 228, Log-Lik: -29581.854, Max-Change: 0.00013
Iteration: 229, Log-Lik: -29581.854, Max-Change: 0.00044
Iteration: 230, Log-Lik: -29581.854, Max-Change: 0.00013
Iteration: 231, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 232, Log-Lik: -29581.854, Max-Change: 0.00054
Iteration: 233, Log-Lik: -29581.854, Max-Change: 0.00014
Iteration: 234, Log-Lik: -29581.854, Max-Change: 0.00011
Iteration: 235, Log-Lik: -29581.854, Max-Change: 0.00026
Iteration: 236, Log-Lik: -29581.854, Max-Change: 0.00014
Iteration: 237, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 238, Log-Lik: -29581.854, Max-Change: 0.00023
Iteration: 239, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 240, Log-Lik: -29581.854, Max-Change: 0.00011
Iteration: 241, Log-Lik: -29581.854, Max-Change: 0.00055
Iteration: 242, Log-Lik: -29581.854, Max-Change: 0.00014
Iteration: 243, Log-Lik: -29581.854, Max-Change: 0.00011
Iteration: 244, Log-Lik: -29581.854, Max-Change: 0.00026
Iteration: 245, Log-Lik: -29581.854, Max-Change: 0.00014
Iteration: 246, Log-Lik: -29581.854, Max-Change: 0.00012
Iteration: 247, Log-Lik: -29581.854, Max-Change: 0.00036
Iteration: 248, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 249, Log-Lik: -29581.853, Max-Change: 0.00011
Iteration: 250, Log-Lik: -29581.853, Max-Change: 0.00037
Iteration: 251, Log-Lik: -29581.853, Max-Change: 0.00014
Iteration: 252, Log-Lik: -29581.853, Max-Change: 0.00011
Iteration: 253, Log-Lik: -29581.853, Max-Change: 0.00040
Iteration: 254, Log-Lik: -29581.853, Max-Change: 0.00014
Iteration: 255, Log-Lik: -29581.853, Max-Change: 0.00011
Iteration: 256, Log-Lik: -29581.853, Max-Change: 0.00029
Iteration: 257, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 258, Log-Lik: -29581.853, Max-Change: 0.00011
Iteration: 259, Log-Lik: -29581.853, Max-Change: 0.00038
Iteration: 260, Log-Lik: -29581.853, Max-Change: 0.00012
Iteration: 261, Log-Lik: -29581.853, Max-Change: 0.00011
Iteration: 262, Log-Lik: -29581.853, Max-Change: 0.00052
Iteration: 263, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 264, Log-Lik: -29581.853, Max-Change: 0.00011
Iteration: 265, Log-Lik: -29581.853, Max-Change: 0.00028
Iteration: 266, Log-Lik: -29581.853, Max-Change: 0.00012
Iteration: 267, Log-Lik: -29581.853, Max-Change: 0.00010
Iteration: 268, Log-Lik: -29581.853, Max-Change: 0.00041
Iteration: 269, Log-Lik: -29581.853, Max-Change: 0.00014
Iteration: 270, Log-Lik: -29581.853, Max-Change: 0.00010
Iteration: 271, Log-Lik: -29581.853, Max-Change: 0.00039
Iteration: 272, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 273, Log-Lik: -29581.853, Max-Change: 0.00010
Iteration: 274, Log-Lik: -29581.853, Max-Change: 0.00028
Iteration: 275, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 276, Log-Lik: -29581.853, Max-Change: 0.00010
Iteration: 277, Log-Lik: -29581.853, Max-Change: 0.00036
Iteration: 278, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 279, Log-Lik: -29581.853, Max-Change: 0.00010
Iteration: 280, Log-Lik: -29581.853, Max-Change: 0.00043
Iteration: 281, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 282, Log-Lik: -29581.853, Max-Change: 0.00010
Iteration: 283, Log-Lik: -29581.853, Max-Change: 0.00035
Iteration: 284, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 285, Log-Lik: -29581.853, Max-Change: 0.00010
Iteration: 286, Log-Lik: -29581.853, Max-Change: 0.00040
Iteration: 287, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 288, Log-Lik: -29581.853, Max-Change: 0.00010
Iteration: 289, Log-Lik: -29581.853, Max-Change: 0.00035
Iteration: 290, Log-Lik: -29581.853, Max-Change: 0.00013
Iteration: 291, Log-Lik: -29581.853, Max-Change: 0.00010
## 
## 
## Calculating information matrix...
#anova
anova(mirt_fit_groups_2_step1, mirt_fit_groups_2_step2)
## 
## Model 1: multipleGroup(data = d2[cog_items_names], model = 1, group = d2$Region, 
##     invariance = c("slopes", "intercepts"))
## Model 2: multipleGroup(data = d2[cog_items_names], model = 1, group = d2$Region, 
##     invariance = "")
anova(mirt_fit_groups_2_step2, mirt_fit_groups_2_step3)
## 
## Model 1: multipleGroup(data = d2[cog_items_names], model = 1, group = d2$Region, 
##     invariance = c("slopes", "intercepts"))
## Model 2: multipleGroup(data = d2[cog_items_names], model = 1, group = d2$Region, 
##     invariance = c("intercepts", "slopes", "free_means", "free_var"))
anova(mirt_fit_groups_2_step1, mirt_fit_groups_2_step3)
## 
## Model 1: multipleGroup(data = d2[cog_items_names], model = 1, group = d2$Region, 
##     invariance = c("intercepts", "slopes", "free_means", "free_var"))
## Model 2: multipleGroup(data = d2[cog_items_names], model = 1, group = d2$Region, 
##     invariance = "")
#fit
# M2(mirt_fit_groups_2_step1)
# M2(mirt_fit_groups_2_step2)
# M2(mirt_fit_groups_2_step3)

#plots
plot(mirt_fit_groups_2_step1)

plot(mirt_fit_groups_2_step2)

plot(mirt_fit_groups_2_step3)

plot(mirt_fit_groups_2_step4)

plot(mirt_fit_groups_2_step1, type = 'info')

#DIFs
DIFs = DIF(mirt_fit_groups_2_step3, c('a1', 'd'), scheme = 'drop')
DIFs = DIFs %>% rownames_to_column("item")
DIFs$number = 1:nrow(DIFs)
DIFs_detected = DIFs %>% filter(p < .05/60) #with significant DIF
DIFs_detected
#get specific items
DIFs2 = DIF(mirt_fit_groups_2_step3, c('a1', 'd'), scheme = 'drop', items2test = DIFs_detected$number, return_models = TRUE)

#plot the detected DIFs
for (i in seq_along(DIFs_detected$item)) {
  itemplot(DIFs2[[i]], item = DIFs_detected$number[i]) %>% print()
}

#fit without DIF items
mirt_fit_groups_2_step5 = multipleGroup(d2[setdiff(cog_items_names, DIFs_detected$item)], model = 1, group = d2$Region, SE=TRUE,
                           invariance = c(""))
## 
Iteration: 1, Log-Lik: -26533.822, Max-Change: 1.58634
Iteration: 2, Log-Lik: -26027.973, Max-Change: 0.43158
Iteration: 3, Log-Lik: -25977.066, Max-Change: 0.23966
Iteration: 4, Log-Lik: -25961.071, Max-Change: 0.13590
Iteration: 5, Log-Lik: -25953.190, Max-Change: 0.07089
Iteration: 6, Log-Lik: -25948.697, Max-Change: 0.05038
Iteration: 7, Log-Lik: -25942.057, Max-Change: 0.01739
Iteration: 8, Log-Lik: -25941.504, Max-Change: 0.01005
Iteration: 9, Log-Lik: -25941.158, Max-Change: 0.00890
Iteration: 10, Log-Lik: -25940.237, Max-Change: 0.01056
Iteration: 11, Log-Lik: -25940.195, Max-Change: 0.00417
Iteration: 12, Log-Lik: -25940.157, Max-Change: 0.00330
Iteration: 13, Log-Lik: -25940.106, Max-Change: 0.00320
Iteration: 14, Log-Lik: -25940.089, Max-Change: 0.00192
Iteration: 15, Log-Lik: -25940.076, Max-Change: 0.00178
Iteration: 16, Log-Lik: -25940.029, Max-Change: 0.00128
Iteration: 17, Log-Lik: -25940.027, Max-Change: 0.00214
Iteration: 18, Log-Lik: -25940.025, Max-Change: 0.00183
Iteration: 19, Log-Lik: -25940.021, Max-Change: 0.00089
Iteration: 20, Log-Lik: -25940.021, Max-Change: 0.00127
Iteration: 21, Log-Lik: -25940.020, Max-Change: 0.00055
Iteration: 22, Log-Lik: -25940.020, Max-Change: 0.00029
Iteration: 23, Log-Lik: -25940.020, Max-Change: 0.00026
Iteration: 24, Log-Lik: -25940.019, Max-Change: 0.00025
Iteration: 25, Log-Lik: -25940.018, Max-Change: 0.00047
Iteration: 26, Log-Lik: -25940.018, Max-Change: 0.00035
Iteration: 27, Log-Lik: -25940.018, Max-Change: 0.00118
Iteration: 28, Log-Lik: -25940.018, Max-Change: 0.00172
Iteration: 29, Log-Lik: -25940.018, Max-Change: 0.00103
Iteration: 30, Log-Lik: -25940.018, Max-Change: 0.00068
Iteration: 31, Log-Lik: -25940.018, Max-Change: 0.00109
Iteration: 32, Log-Lik: -25940.017, Max-Change: 0.00090
Iteration: 33, Log-Lik: -25940.017, Max-Change: 0.00058
Iteration: 34, Log-Lik: -25940.017, Max-Change: 0.00103
Iteration: 35, Log-Lik: -25940.017, Max-Change: 0.00077
Iteration: 36, Log-Lik: -25940.017, Max-Change: 0.00050
Iteration: 37, Log-Lik: -25940.017, Max-Change: 0.00098
Iteration: 38, Log-Lik: -25940.017, Max-Change: 0.00061
Iteration: 39, Log-Lik: -25940.017, Max-Change: 0.00039
Iteration: 40, Log-Lik: -25940.017, Max-Change: 0.00463
Iteration: 41, Log-Lik: -25940.017, Max-Change: 0.00191
Iteration: 42, Log-Lik: -25940.016, Max-Change: 0.00120
Iteration: 43, Log-Lik: -25940.016, Max-Change: 0.00043
Iteration: 44, Log-Lik: -25940.016, Max-Change: 0.00027
Iteration: 45, Log-Lik: -25940.016, Max-Change: 0.00017
Iteration: 46, Log-Lik: -25940.016, Max-Change: 0.00360
Iteration: 47, Log-Lik: -25940.016, Max-Change: 0.00071
Iteration: 48, Log-Lik: -25940.016, Max-Change: 0.00044
Iteration: 49, Log-Lik: -25940.016, Max-Change: 0.00014
Iteration: 50, Log-Lik: -25940.016, Max-Change: 0.00057
Iteration: 51, Log-Lik: -25940.016, Max-Change: 0.00064
Iteration: 52, Log-Lik: -25940.016, Max-Change: 0.00011
Iteration: 53, Log-Lik: -25940.016, Max-Change: 0.00054
Iteration: 54, Log-Lik: -25940.016, Max-Change: 0.00051
Iteration: 55, Log-Lik: -25940.016, Max-Change: 0.00011
Iteration: 56, Log-Lik: -25940.016, Max-Change: 0.00052
Iteration: 57, Log-Lik: -25940.016, Max-Change: 0.00049
Iteration: 58, Log-Lik: -25940.016, Max-Change: 0.00010
Iteration: 59, Log-Lik: -25940.016, Max-Change: 0.00049
Iteration: 60, Log-Lik: -25940.016, Max-Change: 0.00046
Iteration: 61, Log-Lik: -25940.016, Max-Change: 0.00010
## 
## 
## Calculating information matrix...
plot(mirt_fit_groups_2_step5)

So, removing 6 diff items did nothing much to the score difference. This DIF does not have real life importance.