PSYC 6180: Lab 2

Background

The data for this lab is located in the file testretest.csv. Within this data contains as simple randomized control trial (RCT) structure, where a set of individuals were randomly assigned into either a control or treatment group for the purpose of evaluating the clinical efficacy of a new experimental psychotherapy for the (terrifying) target construct,“Fear of Psychometrics”.

The test itself was constructed with \(J=20\) items, each of which had 5 response options that increase in symptom severity, and each of which was assumed to be primarily related to an latent fear of psychometric methods (in particular, the fear of what comes next in the course ….).

The goal in this lab is to evaluate how well the intervention might have worked, beginning at the group level (e.g., studying marginal reliability and change behavior) and later at the individual level.

Steps to follow

Note: Add code chunks where needed to answer each question (for me this was ctrl + alt + i).

1) Read the data into R and inspect! Feel free to add any other information to the following to get a feel for the data.

dat <- readr::read_csv("testretest.csv", show_col_types = FALSE)
dat

# A tibble: 200 × 42
      ID group pre_Item_1 pre_Item_2 pre_Item_3 pre_Item_4 pre_Item_5 pre_Item_6
   <dbl> <chr>      <dbl>      <dbl>      <dbl>      <dbl>      <dbl>      <dbl>
 1     1 trea…          4          0          0          2          4          4
 2     2 trea…          4          0          1          4          3          4
 3     3 trea…          4          1          0          4          2          2
 4     4 trea…          4          3          4          4          2          4
 5     5 trea…          4          4          2          3          4          4
 6     6 trea…          3          4          0          4          4          4
 7     7 trea…          1          1          4          2          4          4
 8     8 trea…          3          0          1          0          0          0
 9     9 trea…          1          1          0          4          4          4
10    10 trea…          4          4          0          4          4          4
# ℹ 190 more rows
# ℹ 34 more variables: pre_Item_7 <dbl>, pre_Item_8 <dbl>, pre_Item_9 <dbl>,
#   pre_Item_10 <dbl>, pre_Item_11 <dbl>, pre_Item_12 <dbl>, pre_Item_13 <dbl>,
#   pre_Item_14 <dbl>, pre_Item_15 <dbl>, pre_Item_16 <dbl>, pre_Item_17 <dbl>,
#   pre_Item_18 <dbl>, pre_Item_19 <dbl>, pre_Item_20 <dbl>, post_Item_1 <dbl>,
#   post_Item_2 <dbl>, post_Item_3 <dbl>, post_Item_4 <dbl>, post_Item_5 <dbl>,
#   post_Item_6 <dbl>, post_Item_7 <dbl>, post_Item_8 <dbl>, …

dplyr::glimpse(dat)

Rows: 200
Columns: 42
$ ID           <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17…
$ group        <chr> "treatment", "treatment", "treatment", "treatment", "trea…
$ pre_Item_1   <dbl> 4, 4, 4, 4, 4, 3, 1, 3, 1, 4, 2, 0, 4, 4, 4, 3, 1, 1, 0, …
$ pre_Item_2   <dbl> 0, 0, 1, 3, 4, 4, 1, 0, 1, 4, 4, 1, 4, 2, 1, 4, 4, 0, 3, …
$ pre_Item_3   <dbl> 0, 1, 0, 4, 2, 0, 4, 1, 0, 0, 2, 0, 0, 4, 0, 2, 0, 0, 0, …
$ pre_Item_4   <dbl> 2, 4, 4, 4, 3, 4, 2, 0, 4, 4, 4, 2, 4, 0, 3, 0, 4, 4, 4, …
$ pre_Item_5   <dbl> 4, 3, 2, 2, 4, 4, 4, 0, 4, 4, 4, 0, 4, 4, 4, 1, 3, 1, 0, …
$ pre_Item_6   <dbl> 4, 4, 2, 4, 4, 4, 4, 0, 4, 4, 4, 2, 4, 4, 4, 0, 4, 0, 4, …
$ pre_Item_7   <dbl> 4, 4, 0, 4, 4, 4, 4, 1, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, …
$ pre_Item_8   <dbl> 4, 4, 0, 0, 4, 4, 1, 4, 4, 4, 4, 2, 4, 3, 4, 4, 4, 4, 4, …
$ pre_Item_9   <dbl> 4, 3, 4, 0, 0, 0, 1, 0, 1, 4, 4, 0, 4, 0, 0, 2, 4, 0, 0, …
$ pre_Item_10  <dbl> 4, 4, 4, 4, 4, 4, 3, 1, 4, 4, 4, 0, 4, 2, 4, 0, 4, 3, 4, …
$ pre_Item_11  <dbl> 4, 2, 0, 4, 4, 0, 2, 4, 1, 4, 3, 3, 4, 2, 4, 1, 4, 4, 1, …
$ pre_Item_12  <dbl> 4, 4, 1, 3, 3, 3, 0, 4, 4, 3, 4, 2, 4, 4, 4, 3, 4, 2, 4, …
$ pre_Item_13  <dbl> 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 1, 4, 4, 4, 3, 4, 4, 4, …
$ pre_Item_14  <dbl> 4, 1, 4, 4, 4, 3, 0, 0, 4, 4, 4, 0, 4, 0, 3, 0, 4, 4, 1, …
$ pre_Item_15  <dbl> 1, 4, 4, 4, 4, 4, 3, 1, 3, 4, 4, 0, 4, 0, 4, 0, 3, 3, 4, …
$ pre_Item_16  <dbl> 4, 4, 4, 4, 2, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, …
$ pre_Item_17  <dbl> 4, 3, 3, 3, 0, 2, 4, 4, 2, 4, 3, 2, 4, 4, 0, 0, 4, 4, 4, …
$ pre_Item_18  <dbl> 3, 4, 3, 4, 0, 4, 4, 4, 4, 4, 4, 3, 4, 4, 4, 0, 4, 1, 4, …
$ pre_Item_19  <dbl> 4, 4, 4, 4, 4, 3, 0, 4, 4, 4, 4, 0, 4, 4, 4, 0, 4, 4, 4, …
$ pre_Item_20  <dbl> 4, 0, 4, 4, 1, 4, 0, 2, 4, 4, 4, 0, 4, 4, 2, 4, 4, 4, 2, …
$ post_Item_1  <dbl> 2, 4, 4, 2, 4, 0, 1, 1, 3, 1, 2, 0, 3, 0, 2, 4, 3, 3, 4, …
$ post_Item_2  <dbl> 3, 4, 0, 1, 3, 4, 0, 1, 3, 4, 4, 0, 4, 1, 4, 0, 2, 0, 3, …
$ post_Item_3  <dbl> 0, 1, 0, 0, 4, 1, 1, 0, 2, 4, 0, 1, 0, 0, 1, 3, 4, 3, 1, …
$ post_Item_4  <dbl> 1, 4, 4, 3, 4, 3, 1, 0, 4, 4, 3, 3, 4, 4, 3, 3, 3, 4, 4, …
$ post_Item_5  <dbl> 0, 0, 1, 3, 0, 0, 0, 2, 2, 3, 2, 0, 4, 4, 2, 0, 4, 0, 2, …
$ post_Item_6  <dbl> 4, 4, 4, 3, 4, 1, 1, 0, 4, 0, 4, 1, 1, 4, 2, 3, 3, 4, 4, …
$ post_Item_7  <dbl> 4, 4, 1, 4, 0, 3, 4, 4, 4, 4, 4, 1, 4, 4, 4, 2, 4, 4, 1, …
$ post_Item_8  <dbl> 0, 4, 2, 0, 4, 0, 3, 0, 4, 4, 4, 0, 4, 0, 4, 2, 1, 4, 4, …
$ post_Item_9  <dbl> 4, 0, 4, 1, 0, 0, 0, 0, 3, 0, 4, 0, 4, 4, 4, 0, 0, 0, 4, …
$ post_Item_10 <dbl> 4, 4, 2, 4, 3, 2, 2, 0, 3, 0, 4, 0, 4, 3, 4, 0, 4, 4, 4, …
$ post_Item_11 <dbl> 4, 2, 4, 2, 4, 0, 0, 0, 2, 1, 2, 0, 2, 0, 3, 4, 4, 0, 4, …
$ post_Item_12 <dbl> 4, 1, 2, 2, 0, 0, 3, 0, 2, 3, 4, 0, 4, 1, 4, 1, 1, 2, 1, …
$ post_Item_13 <dbl> 4, 4, 4, 4, 4, 4, 4, 4, 0, 4, 4, 0, 4, 4, 4, 0, 4, 4, 4, …
$ post_Item_14 <dbl> 1, 4, 0, 0, 3, 1, 0, 0, 1, 0, 4, 0, 4, 4, 3, 0, 3, 0, 0, …
$ post_Item_15 <dbl> 4, 4, 4, 4, 3, 0, 0, 0, 1, 3, 0, 0, 4, 4, 1, 0, 0, 1, 3, …
$ post_Item_16 <dbl> 4, 3, 4, 3, 4, 3, 2, 0, 2, 3, 4, 0, 4, 3, 4, 2, 4, 4, 4, …
$ post_Item_17 <dbl> 4, 4, 1, 3, 3, 3, 2, 3, 4, 4, 4, 4, 3, 4, 4, 0, 1, 4, 1, …
$ post_Item_18 <dbl> 4, 4, 4, 4, 0, 4, 0, 0, 4, 4, 4, 0, 4, 3, 3, 1, 4, 4, 1, …
$ post_Item_19 <dbl> 4, 3, 4, 4, 4, 4, 2, 1, 4, 4, 4, 0, 4, 4, 4, 4, 1, 4, 4, …
$ post_Item_20 <dbl> 0, 0, 1, 4, 4, 0, 0, 0, 0, 4, 4, 0, 4, 0, 0, 4, 0, 0, 2, …

2) Find summary statistics for the sets of pre_* items and post_* items. Do this first by ignoring the group variable to obtain the marginal behavior, then again using the conditional group levels.

grepl('pre', colnames(dat))

 [1] FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[13]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE FALSE
[25] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[37] FALSE FALSE FALSE FALSE FALSE FALSE

library(dplyr)

Attaching package: 'dplyr'

The following objects are masked from 'package:stats':

    filter, lag

The following objects are masked from 'package:base':

    intersect, setdiff, setequal, union

pre_items <- select(dat, starts_with("pre_"))
post_items <- select(dat, starts_with("post_"))
psych::describe(pre_items)

            vars   n mean   sd median trimmed  mad min max range  skew kurtosis
pre_Item_1     1 200 2.54 1.58    3.0    2.68 1.48   0   4     4 -0.55    -1.34
pre_Item_2     2 200 2.10 1.67    2.5    2.12 2.22   0   4     4 -0.07    -1.71
pre_Item_3     3 200 1.14 1.50    0.0    0.92 0.00   0   4     4  0.96    -0.63
pre_Item_4     4 200 2.88 1.49    4.0    3.10 0.00   0   4     4 -0.96    -0.63
pre_Item_5     5 200 1.90 1.65    2.0    1.88 2.97   0   4     4  0.09    -1.62
pre_Item_6     6 200 2.79 1.59    4.0    2.99 0.00   0   4     4 -0.87    -0.95
pre_Item_7     7 200 3.19 1.32    4.0    3.46 0.00   0   4     4 -1.37     0.40
pre_Item_8     8 200 3.02 1.50    4.0    3.27 0.00   0   4     4 -1.13    -0.41
pre_Item_9     9 200 1.82 1.80    1.0    1.77 1.48   0   4     4  0.19    -1.80
pre_Item_10   10 200 2.81 1.50    4.0    3.01 0.00   0   4     4 -0.83    -0.83
pre_Item_11   11 200 2.16 1.66    2.0    2.21 2.97   0   4     4 -0.14    -1.62
pre_Item_12   12 200 2.10 1.64    2.0    2.12 2.97   0   4     4 -0.02    -1.62
pre_Item_13   13 200 3.59 1.09    4.0    3.92 0.00   0   4     4 -2.61     5.43
pre_Item_14   14 200 2.08 1.71    2.0    2.10 2.97   0   4     4 -0.08    -1.73
pre_Item_15   15 200 2.47 1.70    3.0    2.59 1.48   0   4     4 -0.51    -1.52
pre_Item_16   16 200 3.11 1.43    4.0    3.39 0.00   0   4     4 -1.32     0.16
pre_Item_17   17 200 2.98 1.33    4.0    3.22 0.00   0   4     4 -1.10     0.00
pre_Item_18   18 200 2.71 1.56    4.0    2.89 0.00   0   4     4 -0.72    -1.08
pre_Item_19   19 200 3.27 1.36    4.0    3.59 0.00   0   4     4 -1.59     0.96
pre_Item_20   20 200 2.44 1.70    3.0    2.56 1.48   0   4     4 -0.45    -1.52
              se
pre_Item_1  0.11
pre_Item_2  0.12
pre_Item_3  0.11
pre_Item_4  0.11
pre_Item_5  0.12
pre_Item_6  0.11
pre_Item_7  0.09
pre_Item_8  0.11
pre_Item_9  0.13
pre_Item_10 0.11
pre_Item_11 0.12
pre_Item_12 0.12
pre_Item_13 0.08
pre_Item_14 0.12
pre_Item_15 0.12
pre_Item_16 0.10
pre_Item_17 0.09
pre_Item_18 0.11
pre_Item_19 0.10
pre_Item_20 0.12

psych::describe(post_items)

             vars   n mean   sd median trimmed  mad min max range  skew
post_Item_1     1 200 2.18 1.64      2    2.23 2.97   0   4     4 -0.20
post_Item_2     2 200 1.71 1.73      1    1.63 1.48   0   4     4  0.28
post_Item_3     3 200 0.92 1.27      0    0.68 0.00   0   4     4  1.20
post_Item_4     4 200 2.60 1.57      3    2.75 1.48   0   4     4 -0.65
post_Item_5     5 200 1.42 1.56      1    1.27 1.48   0   4     4  0.60
post_Item_6     6 200 2.54 1.61      3    2.68 1.48   0   4     4 -0.51
post_Item_7     7 200 3.13 1.35      4    3.39 0.00   0   4     4 -1.22
post_Item_8     8 200 2.80 1.65      4    3.00 0.00   0   4     4 -0.81
post_Item_9     9 200 1.44 1.76      0    1.29 0.00   0   4     4  0.59
post_Item_10   10 200 2.43 1.61      3    2.54 1.48   0   4     4 -0.48
post_Item_11   11 200 1.71 1.57      2    1.63 2.97   0   4     4  0.33
post_Item_12   12 200 1.78 1.58      1    1.73 1.48   0   4     4  0.30
post_Item_13   13 200 3.24 1.44      4    3.56 0.00   0   4     4 -1.56
post_Item_14   14 200 1.52 1.64      1    1.40 1.48   0   4     4  0.44
post_Item_15   15 200 2.20 1.71      3    2.25 1.48   0   4     4 -0.20
post_Item_16   16 200 2.76 1.50      3    2.94 1.48   0   4     4 -0.82
post_Item_17   17 200 3.02 1.33      4    3.26 0.00   0   4     4 -1.12
post_Item_18   18 200 2.43 1.68      3    2.54 1.48   0   4     4 -0.46
post_Item_19   19 200 3.24 1.33      4    3.52 0.00   0   4     4 -1.43
post_Item_20   20 200 1.81 1.72      2    1.76 2.97   0   4     4  0.21
             kurtosis   se
post_Item_1     -1.59 0.12
post_Item_2     -1.69 0.12
post_Item_3      0.19 0.09
post_Item_4     -1.16 0.11
post_Item_5     -1.20 0.11
post_Item_6     -1.42 0.11
post_Item_7      0.01 0.10
post_Item_8     -1.15 0.12
post_Item_9     -1.49 0.12
post_Item_10    -1.36 0.11
post_Item_11    -1.41 0.11
post_Item_12    -1.46 0.11
post_Item_13     0.69 0.10
post_Item_14    -1.51 0.12
post_Item_15    -1.72 0.12
post_Item_16    -0.85 0.11
post_Item_17    -0.05 0.09
post_Item_18    -1.50 0.12
post_Item_19     0.44 0.09
post_Item_20    -1.67 0.12

psych::describeBy(pre_items, group=dat$group)

 Descriptive statistics by group 
group: control
            vars   n mean   sd median trimmed  mad min max range  skew kurtosis
pre_Item_1     1 100 2.66 1.56      3    2.82 1.48   0   4     4 -0.70    -1.13
pre_Item_2     2 100 2.20 1.67      3    2.25 1.48   0   4     4 -0.17    -1.70
pre_Item_3     3 100 1.22 1.51      0    1.03 0.00   0   4     4  0.82    -0.89
pre_Item_4     4 100 2.96 1.38      4    3.20 0.00   0   4     4 -1.01    -0.39
pre_Item_5     5 100 1.88 1.61      2    1.85 2.97   0   4     4  0.14    -1.57
pre_Item_6     6 100 2.87 1.52      4    3.09 0.00   0   4     4 -0.92    -0.80
pre_Item_7     7 100 3.30 1.22      4    3.56 0.00   0   4     4 -1.54     0.99
pre_Item_8     8 100 2.93 1.53      4    3.16 0.00   0   4     4 -0.98    -0.71
pre_Item_9     9 100 1.96 1.78      2    1.95 2.97   0   4     4  0.05    -1.81
pre_Item_10   10 100 3.04 1.35      4    3.30 0.00   0   4     4 -1.15     0.00
pre_Item_11   11 100 2.24 1.63      2    2.30 2.97   0   4     4 -0.22    -1.56
pre_Item_12   12 100 2.05 1.67      2    2.06 2.97   0   4     4  0.05    -1.66
pre_Item_13   13 100 3.66 1.01      4    3.96 0.00   0   4     4 -2.99     7.70
pre_Item_14   14 100 2.19 1.68      3    2.24 1.48   0   4     4 -0.21    -1.68
pre_Item_15   15 100 2.52 1.67      3    2.65 1.48   0   4     4 -0.59    -1.41
pre_Item_16   16 100 3.20 1.38      4    3.50 0.00   0   4     4 -1.46     0.58
pre_Item_17   17 100 2.98 1.37      4    3.22 0.00   0   4     4 -1.08    -0.16
pre_Item_18   18 100 2.93 1.45      4    3.16 0.00   0   4     4 -0.96    -0.59
pre_Item_19   19 100 3.51 1.09      4    3.80 0.00   0   4     4 -2.17     3.58
pre_Item_20   20 100 2.33 1.71      3    2.41 1.48   0   4     4 -0.31    -1.62
              se
pre_Item_1  0.16
pre_Item_2  0.17
pre_Item_3  0.15
pre_Item_4  0.14
pre_Item_5  0.16
pre_Item_6  0.15
pre_Item_7  0.12
pre_Item_8  0.15
pre_Item_9  0.18
pre_Item_10 0.13
pre_Item_11 0.16
pre_Item_12 0.17
pre_Item_13 0.10
pre_Item_14 0.17
pre_Item_15 0.17
pre_Item_16 0.14
pre_Item_17 0.14
pre_Item_18 0.15
pre_Item_19 0.11
pre_Item_20 0.17
------------------------------------------------------------ 
group: treatment
            vars   n mean   sd median trimmed  mad min max range  skew kurtosis
pre_Item_1     1 100 2.43 1.60    3.0    2.54 1.48   0   4     4 -0.39    -1.51
pre_Item_2     2 100 1.99 1.68    2.0    1.99 2.97   0   4     4  0.03    -1.73
pre_Item_3     3 100 1.05 1.50    0.0    0.81 0.00   0   4     4  1.11    -0.37
pre_Item_4     4 100 2.80 1.60    4.0    3.00 0.00   0   4     4 -0.87    -0.93
pre_Item_5     5 100 1.92 1.69    2.0    1.90 2.97   0   4     4  0.05    -1.69
pre_Item_6     6 100 2.71 1.67    4.0    2.89 0.00   0   4     4 -0.79    -1.15
pre_Item_7     7 100 3.09 1.41    4.0    3.36 0.00   0   4     4 -1.19    -0.14
pre_Item_8     8 100 3.11 1.48    4.0    3.39 0.00   0   4     4 -1.27    -0.10
pre_Item_9     9 100 1.68 1.81    1.0    1.60 1.48   0   4     4  0.32    -1.76
pre_Item_10   10 100 2.57 1.60    3.0    2.71 1.48   0   4     4 -0.54    -1.35
pre_Item_11   11 100 2.09 1.69    2.0    2.11 2.97   0   4     4 -0.07    -1.70
pre_Item_12   12 100 2.15 1.61    2.0    2.19 2.97   0   4     4 -0.10    -1.60
pre_Item_13   13 100 3.52 1.16    4.0    3.86 0.00   0   4     4 -2.28     3.74
pre_Item_14   14 100 1.97 1.74    2.0    1.96 2.97   0   4     4  0.05    -1.78
pre_Item_15   15 100 2.42 1.75    3.0    2.53 1.48   0   4     4 -0.42    -1.64
pre_Item_16   16 100 3.02 1.48    4.0    3.28 0.00   0   4     4 -1.17    -0.24
pre_Item_17   17 100 2.97 1.30    3.5    3.21 0.74   0   4     4 -1.10     0.10
pre_Item_18   18 100 2.50 1.64    3.0    2.62 1.48   0   4     4 -0.49    -1.43
pre_Item_19   19 100 3.03 1.55    4.0    3.29 0.00   0   4     4 -1.15    -0.40
pre_Item_20   20 100 2.56 1.70    4.0    2.70 0.00   0   4     4 -0.58    -1.41
              se
pre_Item_1  0.16
pre_Item_2  0.17
pre_Item_3  0.15
pre_Item_4  0.16
pre_Item_5  0.17
pre_Item_6  0.17
pre_Item_7  0.14
pre_Item_8  0.15
pre_Item_9  0.18
pre_Item_10 0.16
pre_Item_11 0.17
pre_Item_12 0.16
pre_Item_13 0.12
pre_Item_14 0.17
pre_Item_15 0.17
pre_Item_16 0.15
pre_Item_17 0.13
pre_Item_18 0.16
pre_Item_19 0.15
pre_Item_20 0.17

psych::describeBy(post_items, group=dat$group)

 Descriptive statistics by group 
group: control
             vars   n mean   sd median trimmed  mad min max range  skew
post_Item_1     1 100 2.59 1.59    3.0    2.74 1.48   0   4     4 -0.63
post_Item_2     2 100 1.75 1.76    1.0    1.69 1.48   0   4     4  0.24
post_Item_3     3 100 0.69 1.07    0.0    0.46 0.00   0   4     4  1.41
post_Item_4     4 100 2.76 1.48    3.0    2.95 1.48   0   4     4 -0.82
post_Item_5     5 100 1.62 1.59    1.0    1.53 1.48   0   4     4  0.39
post_Item_6     6 100 2.96 1.48    4.0    3.20 0.00   0   4     4 -1.03
post_Item_7     7 100 3.07 1.45    4.0    3.34 0.00   0   4     4 -1.16
post_Item_8     8 100 3.08 1.50    4.0    3.35 0.00   0   4     4 -1.18
post_Item_9     9 100 1.71 1.81    1.0    1.64 1.48   0   4     4  0.30
post_Item_10   10 100 2.54 1.54    3.0    2.68 1.48   0   4     4 -0.56
post_Item_11   11 100 1.87 1.58    2.0    1.84 2.97   0   4     4  0.17
post_Item_12   12 100 2.20 1.60    2.0    2.25 2.97   0   4     4 -0.06
post_Item_13   13 100 3.48 1.25    4.0    3.83 0.00   0   4     4 -2.08
post_Item_14   14 100 1.66 1.66    1.0    1.57 1.48   0   4     4  0.27
post_Item_15   15 100 2.60 1.66    3.5    2.75 0.74   0   4     4 -0.60
post_Item_16   16 100 3.09 1.31    4.0    3.34 0.00   0   4     4 -1.26
post_Item_17   17 100 3.24 1.20    4.0    3.50 0.00   0   4     4 -1.47
post_Item_18   18 100 2.63 1.59    3.0    2.79 1.48   0   4     4 -0.67
post_Item_19   19 100 3.45 1.16    4.0    3.75 0.00   0   4     4 -1.93
post_Item_20   20 100 2.37 1.68    3.0    2.46 1.48   0   4     4 -0.32
             kurtosis   se
post_Item_1     -1.23 0.16
post_Item_2     -1.75 0.18
post_Item_3      0.90 0.11
post_Item_4     -0.78 0.15
post_Item_5     -1.44 0.16
post_Item_6     -0.57 0.15
post_Item_7     -0.29 0.15
post_Item_8     -0.37 0.15
post_Item_9     -1.76 0.18
post_Item_10    -1.19 0.15
post_Item_11    -1.49 0.16
post_Item_12    -1.61 0.16
post_Item_13     2.63 0.13
post_Item_14    -1.64 0.17
post_Item_15    -1.41 0.17
post_Item_16     0.21 0.13
post_Item_17     1.04 0.12
post_Item_18    -1.18 0.16
post_Item_19     2.33 0.12
post_Item_20    -1.60 0.17
------------------------------------------------------------ 
group: treatment
             vars   n mean   sd median trimmed  mad min max range  skew
post_Item_1     1 100 1.78 1.59    2.0    1.72 2.97   0   4     4  0.20
post_Item_2     2 100 1.66 1.70    1.0    1.57 1.48   0   4     4  0.33
post_Item_3     3 100 1.14 1.41    0.5    0.93 0.74   0   4     4  0.92
post_Item_4     4 100 2.44 1.65    3.0    2.55 1.48   0   4     4 -0.47
post_Item_5     5 100 1.21 1.50    0.0    1.01 0.00   0   4     4  0.82
post_Item_6     6 100 2.13 1.63    2.0    2.16 2.97   0   4     4 -0.07
post_Item_7     7 100 3.19 1.24    4.0    3.42 0.00   0   4     4 -1.23
post_Item_8     8 100 2.52 1.74    4.0    2.65 0.00   0   4     4 -0.49
post_Item_9     9 100 1.16 1.67    0.0    0.95 0.00   0   4     4  0.91
post_Item_10   10 100 2.32 1.68    3.0    2.40 1.48   0   4     4 -0.38
post_Item_11   11 100 1.54 1.56    1.0    1.43 1.48   0   4     4  0.50
post_Item_12   12 100 1.36 1.45    1.0    1.20 1.48   0   4     4  0.67
post_Item_13   13 100 3.01 1.57    4.0    3.26 0.00   0   4     4 -1.17
post_Item_14   14 100 1.38 1.61    1.0    1.22 1.48   0   4     4  0.61
post_Item_15   15 100 1.80 1.67    1.0    1.75 1.48   0   4     4  0.17
post_Item_16   16 100 2.42 1.60    3.0    2.53 1.48   0   4     4 -0.45
post_Item_17   17 100 2.80 1.42    3.0    3.00 1.48   0   4     4 -0.82
post_Item_18   18 100 2.23 1.75    3.0    2.29 1.48   0   4     4 -0.25
post_Item_19   19 100 3.04 1.46    4.0    3.29 0.00   0   4     4 -1.04
post_Item_20   20 100 1.24 1.58    0.0    1.05 0.00   0   4     4  0.80
             kurtosis   se
post_Item_1     -1.55 0.16
post_Item_2     -1.66 0.17
post_Item_3     -0.55 0.14
post_Item_4     -1.46 0.16
post_Item_5     -0.85 0.15
post_Item_6     -1.66 0.16
post_Item_7      0.17 0.12
post_Item_8     -1.60 0.17
post_Item_9     -1.00 0.17
post_Item_10    -1.54 0.17
post_Item_11    -1.28 0.16
post_Item_12    -0.96 0.14
post_Item_13    -0.40 0.16
post_Item_14    -1.34 0.16
post_Item_15    -1.70 0.17
post_Item_16    -1.39 0.16
post_Item_17    -0.73 0.14
post_Item_18    -1.71 0.18
post_Item_19    -0.61 0.15
post_Item_20    -0.99 0.16

3) Compute the total scores for each pre and post-test data using the unweighted sum-score strategy. Once complete, plot the relationship between these sum scores, and compute the test-retest reliability within each group level (think about why you should do this!).

dat$TS.pre <-rowSums(pre_items)
dat$TS.post <- rowSums(post_items)
library(ggplot2)
ggplot(dat, aes(x=TS.pre, y=TS.post)) + geom_point()

ggplot(dat, aes(x=TS.pre, y=TS.post, colour=group)) + geom_point()

with(dat, cor(TS.pre, TS.post))

[1] 0.8304459

4) Remembering that test-retest reliability sucks for a number of reasons, obtain some version of a split half \(r_{XX^\prime}\) estimate of your choosing (coefficient \(\alpha\) of course counts here too). As before, do this for the marginal and conditional behavior w.r.t. the group information. Note that depending on how you evaluated 2), this information may already be available.

library(psych)

Attaching package: 'psych'

The following objects are masked from 'package:ggplot2':

    %+%, alpha

alpha(pre_items)

Reliability analysis   
Call: alpha(x = pre_items)

  raw_alpha std.alpha G6(smc) average_r S/N   ase mean   sd median_r
      0.87      0.87    0.88      0.24 6.4 0.013  2.6 0.82     0.27

    95% confidence boundaries 
         lower alpha upper
Feldt     0.84  0.87  0.89
Duhachek  0.84  0.87  0.89

 Reliability if an item is dropped:
            raw_alpha std.alpha G6(smc) average_r S/N alpha se  var.r med.r
pre_Item_1       0.86      0.86    0.87      0.24 6.0    0.014 0.0154  0.25
pre_Item_2       0.86      0.86    0.87      0.24 6.0    0.014 0.0150  0.25
pre_Item_3       0.88      0.88    0.88      0.27 7.0    0.013 0.0095  0.28
pre_Item_4       0.86      0.86    0.87      0.24 6.1    0.014 0.0153  0.26
pre_Item_5       0.86      0.86    0.87      0.24 5.9    0.014 0.0148  0.25
pre_Item_6       0.86      0.86    0.87      0.24 5.9    0.014 0.0147  0.25
pre_Item_7       0.86      0.86    0.87      0.25 6.2    0.014 0.0160  0.27
pre_Item_8       0.86      0.86    0.87      0.24 6.1    0.014 0.0153  0.27
pre_Item_9       0.86      0.86    0.87      0.25 6.2    0.014 0.0157  0.28
pre_Item_10      0.85      0.85    0.87      0.24 5.8    0.015 0.0146  0.25
pre_Item_11      0.87      0.87    0.88      0.25 6.4    0.013 0.0155  0.28
pre_Item_12      0.85      0.85    0.87      0.24 5.9    0.015 0.0148  0.25
pre_Item_13      0.86      0.86    0.87      0.24 6.2    0.014 0.0157  0.27
pre_Item_14      0.86      0.86    0.87      0.24 5.9    0.014 0.0148  0.25
pre_Item_15      0.86      0.86    0.87      0.24 6.1    0.014 0.0153  0.26
pre_Item_16      0.86      0.86    0.87      0.24 6.0    0.014 0.0154  0.26
pre_Item_17      0.87      0.87    0.88      0.26 6.7    0.013 0.0136  0.28
pre_Item_18      0.86      0.86    0.87      0.24 5.9    0.014 0.0150  0.26
pre_Item_19      0.86      0.86    0.87      0.24 6.1    0.014 0.0155  0.26
pre_Item_20      0.86      0.86    0.87      0.24 6.0    0.014 0.0153  0.25

 Item statistics 
              n raw.r std.r r.cor r.drop mean  sd
pre_Item_1  200  0.58  0.58 0.553  0.512  2.5 1.6
pre_Item_2  200  0.62  0.61 0.589  0.550  2.1 1.7
pre_Item_3  200  0.11  0.11 0.033  0.015  1.1 1.5
pre_Item_4  200  0.53  0.54 0.506  0.461  2.9 1.5
pre_Item_5  200  0.64  0.63 0.611  0.571  1.9 1.6
pre_Item_6  200  0.62  0.62 0.605  0.560  2.8 1.6
pre_Item_7  200  0.49  0.49 0.451  0.422  3.2 1.3
pre_Item_8  200  0.53  0.53 0.499  0.461  3.0 1.5
pre_Item_9  200  0.49  0.48 0.436  0.404  1.8 1.8
pre_Item_10 200  0.67  0.67 0.654  0.612  2.8 1.5
pre_Item_11 200  0.39  0.39 0.326  0.303  2.2 1.7
pre_Item_12 200  0.66  0.66 0.641  0.599  2.1 1.6
pre_Item_13 200  0.50  0.52 0.478  0.443  3.6 1.1
pre_Item_14 200  0.63  0.62 0.601  0.562  2.1 1.7
pre_Item_15 200  0.53  0.52 0.482  0.451  2.5 1.7
pre_Item_16 200  0.59  0.59 0.569  0.525  3.1 1.4
pre_Item_17 200  0.26  0.28 0.211  0.184  3.0 1.3
pre_Item_18 200  0.62  0.63 0.610  0.558  2.7 1.6
pre_Item_19 200  0.56  0.57 0.534  0.496  3.3 1.4
pre_Item_20 200  0.59  0.58 0.549  0.512  2.4 1.7

Non missing response frequency for each item
               0    1    2    3    4 miss
pre_Item_1  0.18 0.13 0.08 0.16 0.44    0
pre_Item_2  0.28 0.19 0.04 0.16 0.34    0
pre_Item_3  0.54 0.14 0.10 0.06 0.16    0
pre_Item_4  0.14 0.06 0.11 0.12 0.56    0
pre_Item_5  0.34 0.12 0.15 0.12 0.28    0
pre_Item_6  0.18 0.07 0.06 0.14 0.55    0
pre_Item_7  0.07 0.09 0.08 0.09 0.67    0
pre_Item_8  0.14 0.08 0.06 0.07 0.65    0
pre_Item_9  0.42 0.11 0.04 0.09 0.34    0
pre_Item_10 0.14 0.07 0.15 0.10 0.53    0
pre_Item_11 0.28 0.11 0.16 0.10 0.36    0
pre_Item_12 0.26 0.16 0.16 0.06 0.36    0
pre_Item_13 0.06 0.01 0.03 0.04 0.84    0
pre_Item_14 0.31 0.14 0.06 0.15 0.34    0
pre_Item_15 0.26 0.10 0.02 0.17 0.46    0
pre_Item_16 0.12 0.04 0.08 0.10 0.66    0
pre_Item_17 0.10 0.03 0.17 0.16 0.52    0
pre_Item_18 0.17 0.08 0.14 0.10 0.52    0
pre_Item_19 0.10 0.04 0.08 0.04 0.74    0
pre_Item_20 0.26 0.06 0.13 0.07 0.48    0

alpha(post_items)

Warning in alpha(post_items): Some items were negatively correlated with the first principal component and probably 
should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Some items ( post_Item_3 ) were negatively correlated with the first principal component and 
probably should be reversed.  
To do this, run the function again with the 'check.keys=TRUE' option

Reliability analysis   
Call: alpha(x = post_items)

  raw_alpha std.alpha G6(smc) average_r S/N   ase mean   sd median_r
      0.84      0.84    0.85       0.2 5.1 0.016  2.2 0.78     0.24

    95% confidence boundaries 
         lower alpha upper
Feldt     0.81  0.84  0.87
Duhachek  0.81  0.84  0.87

 Reliability if an item is dropped:
             raw_alpha std.alpha G6(smc) average_r S/N alpha se var.r med.r
post_Item_1       0.83      0.82    0.84      0.20 4.7    0.017 0.017  0.23
post_Item_2       0.83      0.83    0.84      0.20 4.8    0.016 0.017  0.23
post_Item_3       0.85      0.85    0.86      0.23 5.8    0.015 0.009  0.25
post_Item_4       0.84      0.83    0.85      0.20 4.9    0.016 0.017  0.24
post_Item_5       0.83      0.82    0.84      0.20 4.7    0.017 0.016  0.23
post_Item_6       0.83      0.83    0.84      0.20 4.8    0.017 0.016  0.23
post_Item_7       0.85      0.84    0.86      0.22 5.4    0.015 0.015  0.25
post_Item_8       0.84      0.83    0.85      0.21 5.0    0.016 0.017  0.25
post_Item_9       0.83      0.83    0.84      0.20 4.8    0.016 0.017  0.24
post_Item_10      0.83      0.82    0.84      0.20 4.7    0.017 0.017  0.23
post_Item_11      0.84      0.83    0.85      0.21 4.9    0.016 0.017  0.24
post_Item_12      0.83      0.83    0.84      0.20 4.7    0.017 0.016  0.23
post_Item_13      0.83      0.82    0.84      0.20 4.7    0.017 0.016  0.23
post_Item_14      0.83      0.83    0.84      0.20 4.7    0.017 0.017  0.23
post_Item_15      0.83      0.83    0.84      0.20 4.7    0.017 0.017  0.24
post_Item_16      0.83      0.82    0.84      0.20 4.7    0.017 0.016  0.23
post_Item_17      0.84      0.83    0.85      0.21 5.0    0.016 0.016  0.24
post_Item_18      0.83      0.83    0.84      0.20 4.7    0.017 0.017  0.23
post_Item_19      0.83      0.83    0.84      0.20 4.8    0.017 0.017  0.23
post_Item_20      0.83      0.82    0.84      0.20 4.6    0.017 0.016  0.23

 Item statistics 
               n  raw.r  std.r r.cor r.drop mean  sd
post_Item_1  200  0.588  0.582  0.56   0.51 2.19 1.6
post_Item_2  200  0.537  0.532  0.50   0.45 1.71 1.7
post_Item_3  200 -0.027 -0.012 -0.11  -0.11 0.92 1.3
post_Item_4  200  0.471  0.474  0.43   0.39 2.60 1.6
post_Item_5  200  0.594  0.590  0.57   0.52 1.42 1.6
post_Item_6  200  0.542  0.537  0.51   0.46 2.54 1.6
post_Item_7  200  0.190  0.203  0.12   0.11 3.13 1.3
post_Item_8  200  0.415  0.410  0.35   0.32 2.80 1.6
post_Item_9  200  0.522  0.510  0.47   0.43 1.44 1.8
post_Item_10 200  0.584  0.582  0.56   0.51 2.43 1.6
post_Item_11 200  0.466  0.462  0.42   0.38 1.71 1.6
post_Item_12 200  0.570  0.565  0.54   0.50 1.78 1.6
post_Item_13 200  0.591  0.597  0.58   0.53 3.25 1.4
post_Item_14 200  0.579  0.577  0.55   0.50 1.52 1.6
post_Item_15 200  0.561  0.557  0.53   0.48 2.20 1.7
post_Item_16 200  0.576  0.583  0.56   0.51 2.75 1.5
post_Item_17 200  0.373  0.388  0.34   0.30 3.02 1.3
post_Item_18 200  0.567  0.562  0.53   0.49 2.43 1.7
post_Item_19 200  0.535  0.549  0.52   0.47 3.25 1.3
post_Item_20 200  0.627  0.621  0.60   0.55 1.80 1.7

Non missing response frequency for each item
                0    1    2    3    4 miss
post_Item_1  0.27 0.11 0.12 0.16 0.34    0
post_Item_2  0.42 0.12 0.04 0.13 0.28    0
post_Item_3  0.56 0.17 0.12 0.07 0.07    0
post_Item_4  0.20 0.06 0.14 0.14 0.46    0
post_Item_5  0.45 0.14 0.15 0.08 0.18    0
post_Item_6  0.18 0.16 0.07 0.13 0.47    0
post_Item_7  0.08 0.09 0.12 0.06 0.66    0
post_Item_8  0.18 0.12 0.03 0.06 0.61    0
post_Item_9  0.54 0.09 0.04 0.06 0.27    0
post_Item_10 0.24 0.04 0.16 0.14 0.41    0
post_Item_11 0.34 0.15 0.20 0.06 0.24    0
post_Item_12 0.30 0.22 0.15 0.07 0.26    0
post_Item_13 0.13 0.04 0.04 0.04 0.75    0
post_Item_14 0.46 0.12 0.07 0.16 0.20    0
post_Item_15 0.28 0.16 0.03 0.16 0.38    0
post_Item_16 0.16 0.07 0.12 0.17 0.48    0
post_Item_17 0.09 0.06 0.14 0.16 0.56    0
post_Item_18 0.26 0.06 0.12 0.12 0.44    0
post_Item_19 0.07 0.10 0.06 0.04 0.72    0
post_Item_20 0.40 0.09 0.15 0.04 0.32    0

5) (Optional) Compute the True-score estimates for the pre-test and post-test data using some justifiable CTT reliability approach.

6) Finally, focusing on the true-scores for the pre and post-tests total scores, fit a linear regression model to evaluate whether any change occurred that could be predicted/attributed to the group variable and pre-test total scores. Apply any linear regression modelling tools, learned outside this course, as you see fit.

mod <- lm(TS.post ~ TS.pre + group, data=dat)
summary(mod)

Call:
lm(formula = TS.post ~ TS.pre + group, data = dat)

Residuals:
     Min       1Q   Median       3Q      Max 
-17.8100  -6.1286  -0.2608   6.0133  20.7693 

Coefficients:
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)     8.64252    2.02349   4.271 3.02e-05 ***
TS.pre          0.77366    0.03521  21.974  < 2e-16 ***
grouptreatment -6.63035    1.15467  -5.742 3.50e-08 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 8.129 on 197 degrees of freedom
Multiple R-squared:  0.7341,    Adjusted R-squared:  0.7314 
F-statistic:   272 on 2 and 197 DF,  p-value: < 2.2e-16

effectsize::eta_squared(mod)

# Effect Size for ANOVA (Type I)

Parameter | Eta2 (partial) |       95% CI
-----------------------------------------
TS.pre    |           0.72 | [0.67, 1.00]
group     |           0.14 | [0.08, 1.00]

- One-sided CIs: upper bound fixed at [1.00].

Reliable change index

7) Using mirt::RCI(), compute the reliable change index for all individuals who took the test. Once complete, filter out those who reliably changed, and by how much.

8) Plot the RCI information using the package rciplot to get a visual depiction of the overall RCI properties.