Latent Variables & Data Science

Background :

The following exam is the Spring 2024, Midterm Performance. Particularly, an Achievement Test. Intended to measure learned knowledge or skills aquired across 4 weeks and reviewed the day before the exam.

The 3 primary topics assessed are :

1. Probability

2. Design

3. Regression

The following analysis & modeling isn’t meant to be an exhuastive sample meant to generalize across Stats 10 Classes–but rather as a demonstration of the utility & comparison between Latent Variable modeling & typical Data Science Methods s.t. we illuminate underlying assumptions and gain a fundamental understanding of the two.

Research Question(s) :

Psychometrics Q :

The present study seeks to investigate the latent dimensional structure of the Spring 2024 midterm examination after qualitative research I have determined three instructional domains assessed for the exam: Probability, Design, and Regression.

Specifically :

• Does a unidimensional IRT model adequately represent student performance on the midterm?

  • ie. Are all Probability, Design, and Regression items essentially measuring one single underlying ability — general “Statistics 10 competence”?

• Does a correlated three-factor multidimensional IRT model, reflecting Probability, Design, and Regression, provide improved model–data fit?

  • ie. Do Probability, Design, and Regression function as three distinct but related abilities?

• Does a hierarchical or bifactor structure better account for shared and domain-specific variance in item responses?

  • ie. A general statistics ability (shared across all items),

  • ie. PLUS domain-specific skills (Probability, Design, Regression)?

• Do domain-level subscores demonstrate sufficient reliability and distinctiveness to justify reporting them separately from a total score?

Data Science Q :

• Which individual exam items exert the greatest influence on overall exam performance, and do these align with the highest discrimination parameters from the latent variable model?

• Does the dimensional structure suggested by IRT (unidimensional vs. multidimensional vs. bifactor) correspond to the structure recovered by data-driven methods such as PCA, clustering, or predictive modeling?

• Do the manually assigned instructional domains (Probability, Design, Regression) from qualitative research improve predictive accuracy of student performance beyond item-level models, thereby demonstrating incremental utility in a data science framework?

Data Structure & Cleaning

Population : Stats 10 Std
Sample : Spring, 2024 Stats 10 Std taught by Thomas, with particular TA’s

df <- read.csv("/Users/isaiahmireles/Desktop/Misconceptions/24S-STATS-10-LEC-4_Midterm/S24_Midterm_student_responses copy.csv")
library(tidyverse)

Anonymization :

Lets hide the identity of students :

df$Student.ID <- paste0("Student", seq_len(nrow(df)))

# grab relevant col : 
df <- df |> select(
  Student.ID, Sections, Max.Points, Total.Score, 
  matches(
      "^Question\\.[0-9]+\\.Score$|
       ^Question\\.[0-9]+\\.Student\\.Response\\.s\\.$|
       ^Question [0-9]+ Correct Response$"
    )
  )

Data Structure & Basics :

How many unique students?

nrow(df) = 155?

unique(length(df$Student.ID))

## [1] 155

yes. So, there are 155 unique students – 1 student per row

Whats the max score a student can get?

df$Max.Points[1]

## [1] "34.0"

There are 34 questions, each weighted equally (ie. 1) – a max score is 34/34

Whats one question worth?

cat(paste0("Each Q, is worth about: ", round(1/34, 2)*100),"%")

## Each Q, is worth about: 3 %

Basic summary Statistics :

Recall Sample average formula :

\[ \bar{x}=\frac{1}{n}\sum_{\forall i}x_i, \ i=\{1,...,n\} \]

Where \(x_i\) are sampled measures and \(n\) is sample size.

df$Total.Score <- as.numeric(df$Total.Score)

## Warning: NAs introduced by coercion

summary(df$Total.Score)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##   13.00   23.00   27.00   26.33   30.00   34.00       6

As we can see, the IQR (Interquartile Range) is from 23-30, meaning the central 50% of our data lies between 23 to 30

Typical Spread (S.D.) :

\[ \text{S.D.}_{\text{Sample}} = \sqrt{\frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n - 1}} \]

sd(df$Total.Score, na.rm = T)

## [1] 4.477335

Whats the grade dist. look like?

library(patchwork)
total_score_hist1 <-
  df |>
  ggplot(aes(x = Total.Score)) +
  geom_histogram(binwidth = 4, fill = "lightblue", color = "black") +
  scale_x_continuous(
    breaks = seq(
      floor(min(df$Total.Score, na.rm = T)),
      ceiling(max(df$Total.Score, na.rm = T)),
      by = 1
    )
  ) +
  labs(title = "Bin width = S.D. -- basic look")

total_score_hist2 <-
  df |>
  ggplot(aes(x = Total.Score)) +
  geom_histogram(binwidth = 1, fill = "lightblue", color = "black") +
  labs(title = "Bin width = 1 (ie. 1 point buckets)")


bx_plt <-
  df |>
  ggplot(aes(x = Total.Score)) +
  geom_boxplot()

total_score_hist1 / total_score_hist2 / bx_plt

Citations :

• Boxplots are Awesome!!!

Approximately Normal?

df |>
  ggplot(aes(x = Total.Score)) +
  geom_histogram(aes(y = after_stat(density)),
                 binwidth = 2,
                 fill = "lightblue",
                 color = "black") +
  stat_function(fun = dnorm,
                args = list(
                  mean = mean(df$Total.Score, na.rm = TRUE),
                  sd   = sd(df$Total.Score, na.rm = TRUE)
                ),
                color = "red",
                linewidth = 1) +
  labs(title = "Histogram with Normal Curve Overlay")

• Just taking a look, it doesn’t seem terrible but there appears to be skew.

• Also consider because our scores are bounded (0–34), you have a ceiling effect–and our normal model stretches to infinity and our scores don’t.

QQplot of Total Score :

df |>
  ggplot(aes(sample = Total.Score)) +
  stat_qq() +
  stat_qq_line(color = "red") +
  labs(title = "Normal Q-Q Plot", x = "Theoretical Normal Quantiles", y = "Std. Total Score Quantile")

• okay, as we can see – more evidence of skew as the data isnt perfectly following a normal dist.

• Things above the mean (normalized, above 0) are poor estimates.

  • Specifically, students who Notice that those about 1 S.D. above the mean ( \(\bar{x}\) ) are being underestimated based on the normal model–ie. our largest exam scores are not as extreme as the normal model predicts with parameters ( \(\bar{x}\approx 26, \sigma \approx 4\) ), we would expect higher values

One SD above the mean is roughly :

round(mean(df$Total.Score, na.rm=T) + sd(df$Total.Score, na.rm = T), 0)

## [1] 31

citations :

• Quantile-Quantile Plots (QQ plots), Clearly Explained!!!

  • used to briefly review QQ-Plots basics

• Quantiles and Percentiles, Clearly Explained!!!

  • used to briefly review Quantiles

• Interpreting the normal QQ-plot

  • Applied examples

• Normal Quantile-Quantile Plots

  • Theoretical Review

Approximately Binomial?

\[ P(X = k) = \binom{n}{k} p^k (1 - p)^{n - k} \]

\[ \binom{n}{k} \quad \text{counts how many different ways we can get } k \text{ successes in } n \text{ trials.} \]

\[ p^k \quad \text{is the probability of getting } k \text{ successes.} \]

\[ (1 - p)^{n - k} \quad \text{is the probability of getting the remaining } n-k \text{ failures.} \]

Since we know our data isnt actually continuous and we are experiencing a ceiling effect, we can maybe try its discrete partner ( Binomial dist. )

\[ \hat{p}=\frac{\bar{x}}{n} \]

n <- 34
p_hat <- mean(df$Total.Score, na.rm = TRUE) / n

k_vals <- 0:n
binom_probs <- dbinom(k_vals, size = n, prob = p_hat)

df |>
  ggplot(aes(x = Total.Score)) +
  geom_histogram(aes(y = ..density..),
                 binwidth = 1,
                 fill = "lightblue",
                 color = "black") +
  geom_point(
    data = data.frame(
      Total.Score = k_vals,
      density = binom_probs
    ),
    aes(x = Total.Score, y = density),
    color = "red",
    size = 2
  ) +
  labs(
    title = "Observed Scores with Fitted Binomial Model",
    x = "Total Score",
    y = "Density"
  )

• as we can see, this model did terribly.

Fundamental Test Evaluation

Before latent variable modeling, we ask basic psychometric questions to understand the exam.

• Score Reliability

  • How consistent is the total test score?

  • How much of the score variance reflects true differences between students versus random error?

  • How precise are individual students’ scores (what is the standard error of measurement)?

• Item Quality

  • Are items appropriately difficult (not too easy or too hard)?

  • Do items discriminate between high- and low-performing students?

  • Does removing any item improve the test?

Score Reliability :

Question 1 : How consistent is the total test score?

Latent Variables Analysis :

The core general focus on the Latent variable analysis is Dimensionality, to investigate this :

The first question we will focus on is :

Does a unidimensional IRT model adequately represent student performance on the midterm?

• ie. Are all Probability, Design, and Regression items essentially measuring one single underlying ability — general “Statistics 10 competence”?

binary_matrix <- 
  df |> select(
  matches("^Question\\.[0-9]+\\.Score$")
)

Correlation Matrix :

So, there as with any model–there are assumptions. The pearson correlation coef. has 4 underlying assumptions:

1. The two variables (the variables of interest) need to be using a continuous scale.
2. The two variables of interest should have a linear relationship, which you can check with a scatterplot.
3. There should be no spurious outliers.
4. For inference, the variables should be normally or near-to-normally distributed.

If you notice, we are violating this assumption as our values are {0,1} only (binary)–therefore, a more apt. version is tetrachoric correlations.

This assumes :

• each binary variable reflects an underlying continuous latent variable

Therefore, this correlation is more appropriate for dichotomous test items (0,1 scoring).

binary_matrix

# incomplete obs " cor(binary_matrix) didn work
# whats missing? 
library(naniar)
miss_var_summary(binary_matrix) |> head()

Okay so, we are missing 6 student for each question about 4% of the data. Who are those six students?

# unique(binary_matrix$Question.1.Score)
lapply(as.list(binary_matrix), function(x){which(is.na(x))}) |>
  as.data.frame() 

Yeh, so its the same 6 students

binary_matrix

cor_matrix <- cor(binary_matrix, use = "complete.obs")
library(corrplot)

## corrplot 0.95 loaded

corrplot(
  cor_matrix,
  method = "color",
  type = "upper",
  tl.col = "black",
  tl.cex = 0.7
)

Exam as a whole^

Except notice there is a problem with this – out data is one of two values : 0,1

– Pearsons Correlation assumes continuous, linear and ideally normally distributed variables (Of which – mist are ).

Therefore the The tetrachoric correlation is more appropriate.

library(psych)

## 
## Attaching package: 'psych'

## The following objects are masked from 'package:ggplot2':
## 
##     %+%, alpha

# Suppose dat is a data frame or matrix of 0/1 variables
tetra <- tetrachoric(binary_matrix)

## Warning in cor.smooth(mat): Matrix was not positive definite, smoothing was
## done

# Extract the correlation matrix
R_tetra <- tetra$rho
R_tetra

##                   Question.1.Score Question.2.Score Question.3.Score
## Question.1.Score        1.00000000      0.497121399       0.19771922
## Question.2.Score        0.49712140      1.000000000       0.51509384
## Question.3.Score        0.19771922      0.515093836       1.00000000
## Question.4.Score        0.15142336     -0.048191692       0.02389296
## Question.5.Score        0.03036706      0.274061682       0.15917512
## Question.6.Score       -0.08099854      0.109949940       0.38700424
## Question.7.Score        0.52114593      0.306486922       0.24499054
## Question.8.Score        0.20600707      0.231947507       0.27902704
## Question.9.Score        0.17910356      0.424931567       0.30108836
## Question.10.Score       0.34694799      0.499635144       0.52130894
## Question.11.Score       0.11522300      0.192666933       0.13930365
## Question.12.Score       0.18837765      0.297425961       0.15962672
## Question.13.Score      -0.03665167     -0.009375441       0.33056686
## Question.14.Score       0.01180035      0.247690598       0.26951068
## Question.15.Score       0.01787841     -0.086017358       0.05224854
## Question.16.Score       0.20271100      0.148994113      -0.08858171
## Question.17.Score       0.23959405      0.301125387       0.09065888
## Question.18.Score      -0.03738325     -0.034359822       0.36734530
## Question.19.Score       0.19238529     -0.079292501       0.21892485
## Question.20.Score       0.14239707      0.135698836       0.29456595
## Question.21.Score      -0.11719143      0.126475839       0.21126401
## Question.22.Score       0.04692191      0.317877825       0.02085874
## Question.23.Score       0.12618331      0.068237269       0.33875798
## Question.24.Score       0.31965528      0.169507424       0.21081647
## Question.25.Score       0.03899115     -0.055744270       0.27445451
## Question.26.Score      -0.11937734      0.149803621       0.15472927
## Question.27.Score      -0.03821766      0.235606288       0.43936634
## Question.28.Score       0.30261688      0.233286533       0.32178135
## Question.29.Score      -0.06879692      0.155131455       0.25111410
## Question.30.Score       0.31886476      0.517352667       0.35679282
## Question.31.Score       0.30137864      0.311818688       0.31792361
## Question.32.Score       0.30858303      0.215141385      -0.12381963
## Question.33.Score       0.17788721      0.287407472       0.07171567
## Question.34.Score       0.05049106      0.278294536       0.20153377
##                   Question.4.Score Question.5.Score Question.6.Score
## Question.1.Score      1.514234e-01       0.03036706      -0.08099854
## Question.2.Score     -4.819169e-02       0.27406168       0.10994994
## Question.3.Score      2.389296e-02       0.15917512       0.38700424
## Question.4.Score      1.000000e+00      -0.09625325      -0.12042262
## Question.5.Score     -9.625325e-02       1.00000000       0.30713467
## Question.6.Score     -1.204226e-01       0.30713467       1.00000000
## Question.7.Score      8.719649e-05       0.19313253       0.35715664
## Question.8.Score      2.617658e-01       0.20285520       0.02715386
## Question.9.Score      1.618841e-01       0.18304102       0.15034286
## Question.10.Score     3.592071e-01       0.25299949       0.11844535
## Question.11.Score     2.614627e-02       0.30323913       0.09496295
## Question.12.Score     2.243169e-01       0.16535437       0.23338124
## Question.13.Score     4.670877e-01       0.06934029       0.39411337
## Question.14.Score     2.415456e-02       0.24576768       0.18041822
## Question.15.Score     5.330034e-02       0.02291785       0.24101323
## Question.16.Score     1.548412e-01       0.05474114       0.06065028
## Question.17.Score     3.952093e-01      -0.12057818      -0.02699511
## Question.18.Score     3.463172e-01       0.25548357       0.22477782
## Question.19.Score     3.138069e-01       0.29918794       0.22505413
## Question.20.Score     2.704820e-01       0.42725584       0.23143691
## Question.21.Score     2.927847e-01       0.48105725       0.11568460
## Question.22.Score     1.411515e-02       0.09194347      -0.13113417
## Question.23.Score     2.601261e-01       0.30419256       0.42428224
## Question.24.Score    -2.780949e-01       0.15358902       0.01575079
## Question.25.Score     9.775865e-02       0.21857841       0.40959903
## Question.26.Score    -1.397472e-01       0.45834428       0.33243798
## Question.27.Score     1.511456e-01       0.43805319       0.62995888
## Question.28.Score     2.891444e-01       0.18183714       0.02052230
## Question.29.Score     1.807577e-01       0.06175462       0.46102711
## Question.30.Score     2.215398e-01       0.07512057       0.12139140
## Question.31.Score     6.879205e-02       0.37821807       0.30680604
## Question.32.Score     4.587003e-02       0.08632457       0.20472001
## Question.33.Score     1.726794e-01       0.34664171       0.34853678
## Question.34.Score     2.251077e-01       0.22936889       0.21471874
##                   Question.7.Score Question.8.Score Question.9.Score
## Question.1.Score      5.211459e-01       0.20600707      0.179103559
## Question.2.Score      3.064869e-01       0.23194751      0.424931567
## Question.3.Score      2.449905e-01       0.27902704      0.301088358
## Question.4.Score      8.719649e-05       0.26176579      0.161884131
## Question.5.Score      1.931325e-01       0.20285520      0.183041019
## Question.6.Score      3.571566e-01       0.02715386      0.150342855
## Question.7.Score      1.000000e+00       0.23146414      0.357444220
## Question.8.Score      2.314641e-01       1.00000000      0.277658850
## Question.9.Score      3.574442e-01       0.27765885      1.000000000
## Question.10.Score     1.132152e-01       0.40220884      0.482837801
## Question.11.Score     3.641499e-01       0.42741651      0.487912141
## Question.12.Score     1.480021e-01       0.34339936      0.386943739
## Question.13.Score     1.345201e-01       0.26222092      0.088218811
## Question.14.Score     1.202932e-01       0.22683586      0.345294931
## Question.15.Score     3.317890e-01      -0.04675071     -0.006833803
## Question.16.Score     1.831295e-01       0.19969789      0.528600159
## Question.17.Score     1.672933e-01       0.31319776      0.140528433
## Question.18.Score     2.810188e-02       0.37007178      0.274711285
## Question.19.Score     1.293068e-01       0.28562884      0.183273040
## Question.20.Score     3.078032e-01       0.53346649      0.489174686
## Question.21.Score    -7.213607e-02       0.10513610      0.060955423
## Question.22.Score     1.085616e-01       0.08389676      0.065821556
## Question.23.Score    -8.959554e-02       0.18625785      0.004666897
## Question.24.Score     9.339659e-02      -0.11781096      0.102826899
## Question.25.Score     3.593297e-01       0.32336827      0.216073695
## Question.26.Score     1.430373e-01       0.16476638      0.234591166
## Question.27.Score     3.125290e-01       0.30503232      0.525043467
## Question.28.Score     6.927903e-03       0.05845763      0.155578735
## Question.29.Score     1.288425e-01       0.05497390      0.356590572
## Question.30.Score     1.621723e-01       0.31419648      0.269551795
## Question.31.Score     2.311656e-01       0.54101706      0.225721268
## Question.32.Score     1.790953e-01       0.03743699     -0.044841465
## Question.33.Score     4.906197e-02       0.02844253      0.178582442
## Question.34.Score    -1.318744e-02       0.21991728      0.014559247
##                   Question.10.Score Question.11.Score Question.12.Score
## Question.1.Score        0.346947987        0.11522300       0.188377647
## Question.2.Score        0.499635144        0.19266693       0.297425961
## Question.3.Score        0.521308942        0.13930365       0.159626723
## Question.4.Score        0.359207123        0.02614627       0.224316879
## Question.5.Score        0.252999488        0.30323913       0.165354375
## Question.6.Score        0.118445347        0.09496295       0.233381244
## Question.7.Score        0.113215235        0.36414992       0.148002127
## Question.8.Score        0.402208840        0.42741651       0.343399362
## Question.9.Score        0.482837801        0.48791214       0.386943739
## Question.10.Score       1.000000000        0.27029730       0.422043208
## Question.11.Score       0.270297296        1.00000000       0.396093772
## Question.12.Score       0.422043208        0.39609377       1.000000000
## Question.13.Score       0.194822926        0.22743872       0.454917051
## Question.14.Score       0.051511668        0.32803618       0.160468889
## Question.15.Score      -0.147764743       -0.06028382       0.127891104
## Question.16.Score       0.235513494        0.46926889       0.465707707
## Question.17.Score       0.292493941        0.08643898       0.405494980
## Question.18.Score       0.361748260        0.26329516       0.278037996
## Question.19.Score       0.247167311        0.18534267       0.358226771
## Question.20.Score       0.493835165        0.48210678       0.512144084
## Question.21.Score       0.250876301        0.24126475       0.340913914
## Question.22.Score      -0.002716641       -0.14492240       0.352014167
## Question.23.Score       0.341481992       -0.01620926       0.309869295
## Question.24.Score      -0.011891664        0.13931471      -0.055448191
## Question.25.Score       0.219150274        0.17750760       0.303664325
## Question.26.Score       0.169133158        0.41714651       0.005345905
## Question.27.Score       0.499477323        0.15470145       0.233986017
## Question.28.Score       0.453702808       -0.09479722      -0.068866062
## Question.29.Score       0.321158780        0.13747252       0.441620963
## Question.30.Score       0.469052438        0.02666856       0.162270740
## Question.31.Score       0.385730532        0.07146000       0.318390219
## Question.32.Score       0.250500038       -0.06177103       0.239880233
## Question.33.Score       0.338043785       -0.15543021       0.353416309
## Question.34.Score       0.365565606        0.19823509       0.441593022
##                   Question.13.Score Question.14.Score Question.15.Score
## Question.1.Score       -0.036651672       0.011800349       0.017878409
## Question.2.Score       -0.009375441       0.247690598      -0.086017358
## Question.3.Score        0.330566864       0.269510684       0.052248542
## Question.4.Score        0.467087684       0.024154559       0.053300337
## Question.5.Score        0.069340285       0.245767675       0.022917853
## Question.6.Score        0.394113369       0.180418221       0.241013233
## Question.7.Score        0.134520124       0.120293184       0.331788963
## Question.8.Score        0.262220919       0.226835862      -0.046750713
## Question.9.Score        0.088218811       0.345294931      -0.006833803
## Question.10.Score       0.194822926       0.051511668      -0.147764743
## Question.11.Score       0.227438717       0.328036182      -0.060283817
## Question.12.Score       0.454917051       0.160468889       0.127891104
## Question.13.Score       1.000000000       0.133228155       0.248729075
## Question.14.Score       0.133228155       1.000000000       0.332731835
## Question.15.Score       0.248729075       0.332731835       1.000000000
## Question.16.Score       0.211472198       0.080796297       0.054785935
## Question.17.Score      -0.065245588       0.119495183      -0.026704255
## Question.18.Score       0.384984337      -0.052732162       0.070919476
## Question.19.Score       0.427804652      -0.008159582      -0.081745024
## Question.20.Score       0.486176301       0.055592980      -0.072472628
## Question.21.Score       0.338990886       0.223895742       0.043799781
## Question.22.Score      -0.019308758       0.200807650       0.075923215
## Question.23.Score       0.433421484       0.340791761       0.132746155
## Question.24.Score      -0.218693069       0.271227274       0.050637746
## Question.25.Score       0.377313812       0.131553787       0.129135785
## Question.26.Score       0.124666936      -0.045140584      -0.177884684
## Question.27.Score       0.314755094       0.285639879       0.143549108
## Question.28.Score      -0.024300358       0.076110074       0.065765910
## Question.29.Score       0.157612190       0.225363611       0.125970517
## Question.30.Score      -0.187062335       0.206122868      -0.249774454
## Question.31.Score       0.168034856       0.092513719       0.066082181
## Question.32.Score       0.101677274       0.029700966       0.217744916
## Question.33.Score       0.245467147       0.051767958       0.331441269
## Question.34.Score       0.199809517       0.056413080       0.074888732
##                   Question.16.Score Question.17.Score Question.18.Score
## Question.1.Score         0.20271100       0.239594050       -0.03738325
## Question.2.Score         0.14899411       0.301125387       -0.03435982
## Question.3.Score        -0.08858171       0.090658882        0.36734530
## Question.4.Score         0.15484117       0.395209345        0.34631718
## Question.5.Score         0.05474114      -0.120578183        0.25548357
## Question.6.Score         0.06065028      -0.026995114        0.22477782
## Question.7.Score         0.18312951       0.167293296        0.02810188
## Question.8.Score         0.19969789       0.313197757        0.37007178
## Question.9.Score         0.52860016       0.140528433        0.27471128
## Question.10.Score        0.23551349       0.292493941        0.36174826
## Question.11.Score        0.46926889       0.086438975        0.26329516
## Question.12.Score        0.46570771       0.405494980        0.27803800
## Question.13.Score        0.21147220      -0.065245588        0.38498434
## Question.14.Score        0.08079630       0.119495183       -0.05273216
## Question.15.Score        0.05478593      -0.026704255        0.07091948
## Question.16.Score        1.00000000      -0.078883424        0.35608002
## Question.17.Score       -0.07888342       1.000000000        0.04440541
## Question.18.Score        0.35608002       0.044405406        1.00000000
## Question.19.Score        0.40916119       0.133909203        0.66479648
## Question.20.Score        0.48316366       0.100558918        0.65176828
## Question.21.Score        0.26895183       0.008277553        0.39338564
## Question.22.Score        0.20144464       0.260360128       -0.19198143
## Question.23.Score        0.13603399       0.030726816        0.34263725
## Question.24.Score        0.21619057      -0.214946388        0.02125778
## Question.25.Score        0.11286504       0.040414981        0.38392862
## Question.26.Score        0.31196528      -0.172947747        0.21667734
## Question.27.Score        0.12918056       0.077056442        0.45426365
## Question.28.Score        0.24081464      -0.063767634        0.42385936
## Question.29.Score        0.27240330       0.219448258        0.06325467
## Question.30.Score       -0.03387101       0.634408405        0.20392562
## Question.31.Score        0.29814697      -0.024107157        0.37310613
## Question.32.Score        0.26725055       0.199747943        0.06764699
## Question.33.Score        0.26791034       0.207471385        0.31002902
## Question.34.Score        0.04765819       0.390440397        0.34003938
##                   Question.19.Score Question.20.Score Question.21.Score
## Question.1.Score        0.192385290        0.14239707      -0.117191434
## Question.2.Score       -0.079292501        0.13569884       0.126475839
## Question.3.Score        0.218924853        0.29456595       0.211264015
## Question.4.Score        0.313806881        0.27048197       0.292784726
## Question.5.Score        0.299187940        0.42725584       0.481057249
## Question.6.Score        0.225054126        0.23143691       0.115684605
## Question.7.Score        0.129306819        0.30780320      -0.072136066
## Question.8.Score        0.285628842        0.53346649       0.105136100
## Question.9.Score        0.183273040        0.48917469       0.060955423
## Question.10.Score       0.247167311        0.49383517       0.250876301
## Question.11.Score       0.185342668        0.48210678       0.241264752
## Question.12.Score       0.358226771        0.51214408       0.340913914
## Question.13.Score       0.427804652        0.48617630       0.338990886
## Question.14.Score      -0.008159582        0.05559298       0.223895742
## Question.15.Score      -0.081745024       -0.07247263       0.043799781
## Question.16.Score       0.409161185        0.48316366       0.268951832
## Question.17.Score       0.133909203        0.10055892       0.008277553
## Question.18.Score       0.664796477        0.65176828       0.393385637
## Question.19.Score       1.000000000        0.72930231       0.272534687
## Question.20.Score       0.729302313        1.00000000       0.208329731
## Question.21.Score       0.272534687        0.20832973       1.000000000
## Question.22.Score       0.094513563        0.11198990       0.149644399
## Question.23.Score       0.363926622        0.31785455       0.430527251
## Question.24.Score       0.044314736        0.01318417       0.117212209
## Question.25.Score       0.482931826        0.55574353       0.091820622
## Question.26.Score       0.260950285        0.39897003       0.039467139
## Question.27.Score       0.333635888        0.45852548       0.198845223
## Question.28.Score       0.323947354        0.07723316       0.233892898
## Question.29.Score       0.059694238        0.06278774       0.274105762
## Question.30.Score       0.165710218        0.02307938       0.171879062
## Question.31.Score       0.248773824        0.37532526       0.327306308
## Question.32.Score       0.221043423        0.11884377      -0.018852770
## Question.33.Score       0.401274991        0.30305384       0.200515163
## Question.34.Score       0.160484384        0.27678966       0.270744658
##                   Question.22.Score Question.23.Score Question.24.Score
## Question.1.Score        0.046921911       0.126183314       0.319655279
## Question.2.Score        0.317877825       0.068237269       0.169507424
## Question.3.Score        0.020858736       0.338757978       0.210816472
## Question.4.Score        0.014115150       0.260126108      -0.278094873
## Question.5.Score        0.091943473       0.304192556       0.153589023
## Question.6.Score       -0.131134167       0.424282241       0.015750794
## Question.7.Score        0.108561575      -0.089595538       0.093396588
## Question.8.Score        0.083896759       0.186257851      -0.117810960
## Question.9.Score        0.065821556       0.004666897       0.102826899
## Question.10.Score      -0.002716641       0.341481992      -0.011891664
## Question.11.Score      -0.144922400      -0.016209257       0.139314714
## Question.12.Score       0.352014167       0.309869295      -0.055448191
## Question.13.Score      -0.019308758       0.433421484      -0.218693069
## Question.14.Score       0.200807650       0.340791761       0.271227274
## Question.15.Score       0.075923215       0.132746155       0.050637746
## Question.16.Score       0.201444645       0.136033985       0.216190569
## Question.17.Score       0.260360128       0.030726816      -0.214946388
## Question.18.Score      -0.191981427       0.342637247       0.021257783
## Question.19.Score       0.094513563       0.363926622       0.044314736
## Question.20.Score       0.111989896       0.317854551       0.013184167
## Question.21.Score       0.149644399       0.430527251       0.117212209
## Question.22.Score       1.000000000       0.009206076       0.017839346
## Question.23.Score       0.009206076       1.000000000       0.033129974
## Question.24.Score       0.017839346       0.033129974       1.000000000
## Question.25.Score       0.014460141       0.240990700      -0.147514491
## Question.26.Score      -0.043820788       0.041376977      -0.101540286
## Question.27.Score      -0.114960872       0.338507466      -0.196569085
## Question.28.Score       0.109298177       0.186931915       0.183742506
## Question.29.Score       0.173067427       0.290944588      -0.181175480
## Question.30.Score       0.135459664       0.137225244      -0.119852891
## Question.31.Score       0.023619123       0.415362602       0.036366929
## Question.32.Score       0.078432406       0.297561907      -0.283387284
## Question.33.Score       0.139370021       0.384919216      -0.086412836
## Question.34.Score      -0.087307723       0.107769056      -0.004763364
##                   Question.25.Score Question.26.Score Question.27.Score
## Question.1.Score        0.038991153      -0.119377340       -0.03821766
## Question.2.Score       -0.055744270       0.149803621        0.23560629
## Question.3.Score        0.274454513       0.154729269        0.43936634
## Question.4.Score        0.097758649      -0.139747238        0.15114560
## Question.5.Score        0.218578413       0.458344275        0.43805319
## Question.6.Score        0.409599026       0.332437984        0.62995888
## Question.7.Score        0.359329746       0.143037253        0.31252899
## Question.8.Score        0.323368266       0.164766379        0.30503232
## Question.9.Score        0.216073695       0.234591166        0.52504347
## Question.10.Score       0.219150274       0.169133158        0.49947732
## Question.11.Score       0.177507603       0.417146505        0.15470145
## Question.12.Score       0.303664325       0.005345905        0.23398602
## Question.13.Score       0.377313812       0.124666936        0.31475509
## Question.14.Score       0.131553787      -0.045140584        0.28563988
## Question.15.Score       0.129135785      -0.177884684        0.14354911
## Question.16.Score       0.112865040       0.311965278        0.12918056
## Question.17.Score       0.040414981      -0.172947747        0.07705644
## Question.18.Score       0.383928623       0.216677338        0.45426365
## Question.19.Score       0.482931826       0.260950285        0.33363589
## Question.20.Score       0.555743529       0.398970028        0.45852548
## Question.21.Score       0.091820622       0.039467139        0.19884522
## Question.22.Score       0.014460141      -0.043820788       -0.11496087
## Question.23.Score       0.240990700       0.041376977        0.33850747
## Question.24.Score      -0.147514491      -0.101540286       -0.19656908
## Question.25.Score       1.000000000       0.191589007        0.37937321
## Question.26.Score       0.191589007       1.000000000        0.33266379
## Question.27.Score       0.379373206       0.332663786        1.00000000
## Question.28.Score       0.005363667       0.129299265        0.32581382
## Question.29.Score       0.459665380       0.261388249        0.36029638
## Question.30.Score       0.048142763      -0.080605897        0.44794523
## Question.31.Score       0.304449714       0.250040479        0.54383330
## Question.32.Score       0.225495942       0.106264447        0.32566134
## Question.33.Score       0.371280111       0.178237401        0.35621204
## Question.34.Score       0.302265098       0.191920469        0.27492874
##                   Question.28.Score Question.29.Score Question.30.Score
## Question.1.Score        0.302616885       -0.06879692        0.31886476
## Question.2.Score        0.233286533        0.15513145        0.51735267
## Question.3.Score        0.321781351        0.25111410        0.35679282
## Question.4.Score        0.289144386        0.18075769        0.22153977
## Question.5.Score        0.181837138        0.06175462        0.07512057
## Question.6.Score        0.020522296        0.46102711        0.12139140
## Question.7.Score        0.006927903        0.12884247        0.16217234
## Question.8.Score        0.058457626        0.05497390        0.31419648
## Question.9.Score        0.155578735        0.35659057        0.26955180
## Question.10.Score       0.453702808        0.32115878        0.46905244
## Question.11.Score      -0.094797218        0.13747252        0.02666856
## Question.12.Score      -0.068866062        0.44162096        0.16227074
## Question.13.Score      -0.024300358        0.15761219       -0.18706234
## Question.14.Score       0.076110074        0.22536361        0.20612287
## Question.15.Score       0.065765910        0.12597052       -0.24977445
## Question.16.Score       0.240814641        0.27240330       -0.03387101
## Question.17.Score      -0.063767634        0.21944826        0.63440840
## Question.18.Score       0.423859359        0.06325467        0.20392562
## Question.19.Score       0.323947354        0.05969424        0.16571022
## Question.20.Score       0.077233165        0.06278774        0.02307938
## Question.21.Score       0.233892898        0.27410576        0.17187906
## Question.22.Score       0.109298177        0.17306743        0.13545966
## Question.23.Score       0.186931915        0.29094459        0.13722524
## Question.24.Score       0.183742506       -0.18117548       -0.11985289
## Question.25.Score       0.005363667        0.45966538        0.04814276
## Question.26.Score       0.129299265        0.26138825       -0.08060590
## Question.27.Score       0.325813824        0.36029638        0.44794523
## Question.28.Score       1.000000000        0.14525657        0.44411953
## Question.29.Score       0.145256568        1.00000000        0.26827301
## Question.30.Score       0.444119531        0.26827301        1.00000000
## Question.31.Score       0.261962248        0.30617356        0.30387106
## Question.32.Score       0.237072538        0.20743336        0.32155482
## Question.33.Score       0.301855688        0.29537116        0.16234286
## Question.34.Score       0.047415104        0.31886520        0.25388647
##                   Question.31.Score Question.32.Score Question.33.Score
## Question.1.Score         0.30137864        0.30858303        0.17788721
## Question.2.Score         0.31181869        0.21514138        0.28740747
## Question.3.Score         0.31792361       -0.12381963        0.07171567
## Question.4.Score         0.06879205        0.04587003        0.17267944
## Question.5.Score         0.37821807        0.08632457        0.34664171
## Question.6.Score         0.30680604        0.20472001        0.34853678
## Question.7.Score         0.23116559        0.17909534        0.04906197
## Question.8.Score         0.54101706        0.03743699        0.02844253
## Question.9.Score         0.22572127       -0.04484147        0.17858244
## Question.10.Score        0.38573053        0.25050004        0.33804378
## Question.11.Score        0.07146000       -0.06177103       -0.15543021
## Question.12.Score        0.31839022        0.23988023        0.35341631
## Question.13.Score        0.16803486        0.10167727        0.24546715
## Question.14.Score        0.09251372        0.02970097        0.05176796
## Question.15.Score        0.06608218        0.21774492        0.33144127
## Question.16.Score        0.29814697        0.26725055        0.26791034
## Question.17.Score       -0.02410716        0.19974794        0.20747139
## Question.18.Score        0.37310613        0.06764699        0.31002902
## Question.19.Score        0.24877382        0.22104342        0.40127499
## Question.20.Score        0.37532526        0.11884377        0.30305384
## Question.21.Score        0.32730631       -0.01885277        0.20051516
## Question.22.Score        0.02361912        0.07843241        0.13937002
## Question.23.Score        0.41536260        0.29756191        0.38491922
## Question.24.Score        0.03636693       -0.28338728       -0.08641284
## Question.25.Score        0.30444971        0.22549594        0.37128011
## Question.26.Score        0.25004048        0.10626445        0.17823740
## Question.27.Score        0.54383330        0.32566134        0.35621204
## Question.28.Score        0.26196225        0.23707254        0.30185569
## Question.29.Score        0.30617356        0.20743336        0.29537116
## Question.30.Score        0.30387106        0.32155482        0.16234286
## Question.31.Score        1.00000000        0.30316744        0.23892999
## Question.32.Score        0.30316744        1.00000000        0.57820420
## Question.33.Score        0.23892999        0.57820420        1.00000000
## Question.34.Score        0.41104256        0.29716344        0.32277934
##                   Question.34.Score
## Question.1.Score        0.050491063
## Question.2.Score        0.278294536
## Question.3.Score        0.201533772
## Question.4.Score        0.225107737
## Question.5.Score        0.229368892
## Question.6.Score        0.214718736
## Question.7.Score       -0.013187440
## Question.8.Score        0.219917278
## Question.9.Score        0.014559247
## Question.10.Score       0.365565606
## Question.11.Score       0.198235085
## Question.12.Score       0.441593022
## Question.13.Score       0.199809517
## Question.14.Score       0.056413080
## Question.15.Score       0.074888732
## Question.16.Score       0.047658190
## Question.17.Score       0.390440397
## Question.18.Score       0.340039384
## Question.19.Score       0.160484384
## Question.20.Score       0.276789658
## Question.21.Score       0.270744658
## Question.22.Score      -0.087307723
## Question.23.Score       0.107769056
## Question.24.Score      -0.004763364
## Question.25.Score       0.302265098
## Question.26.Score       0.191920469
## Question.27.Score       0.274928740
## Question.28.Score       0.047415104
## Question.29.Score       0.318865203
## Question.30.Score       0.253886475
## Question.31.Score       0.411042561
## Question.32.Score       0.297163437
## Question.33.Score       0.322779340
## Question.34.Score       1.000000000

Split correlations \(\phi <0 , \phi =0, \phi>0\)

Before we do so, lets get a notion of the typical variation of correlations :

cor_list <- lapply(
  seq_len(nrow(cor_matrix)),
  function(i) cor_matrix[i, -i]
)
names(cor_list) <- paste0("Q", seq_along(cor_list))

Dist of Correlations

sd <- lapply(cor_list, function(x){
  v <- c(sd(x))
  names(v) <- c("sd")
  v
})

mu <- mean(unlist(sd))
sigma <- sd(unlist(sd))


hist(unlist(sd), freq=FALSE, main = "SD of Correlations")

curve(
  dnorm(x, mean = mu, sd = sigma),
  add = TRUE, 
  col = "red"
)

bsc_sum <-
  lapply(cor_list, function(x){
  v <- c(mean(x), sd(x))
  names(v) <- c("mean", "sd")
  v
})

lapply(seq_along(cor_list), function(i) {
  x <- cor_list[[i]]
  stats <- bsc_sum[[i]]

  hist(
    x,
    main = paste("Correlation Hist:", names(cor_list)[i]),
    xlab = "Correlation"
  )

  legend(
    "topright",
    legend = c(
      paste("Mean =", round(stats["mean"], 3)),
      paste("SD =", round(stats["sd"], 3))
    ),
    bty = "n"
  )
})

What happens if I model per question Score to overall exam performance? What does that mean? Wont i get R^2 = 1?

mdl_mat <- cbind(binary_matrix, df$Total.Score)

fct_mat <- binary_matrix |> mutate(across(everything(), factor))
mdl_mat <- cbind(df$Total.Score, fct_mat)
colnames(mdl_mat)[1] <- "Total.Score" #whoops

mdl <- lm(Total.Score~., dat=mdl_mat)
paste("r.squared val: ",summary(mdl)$r.squared)

## [1] "r.squared val:  1"

plot(mdl, which = 2)

The error is exactly normally distributed

Inscribe Manual Labels :

Latent Variable : Statistical Reasoning ( Midterm 1 )

• Probability

• Regression

• Design & Experimentation