Intro

About a month ago, I delivered a 12-item questionnaire to 142 juniors. These questions are related to their perceptions of ‘being a college student’, ‘taking undergraduate courses’ and ‘assessment in education course’. As I stated in its title, this is a pseudo-instrument given that the questions lack solid justifications based on the related literature. I simply asked very simple questions using no research at all, even Google.

Intro in Turkish

Bu döküman 142 üniversite 3. sınıf öğrencisine sorduğum 12 sorunun basit bir analizini içerir. Öğrenciler üniversite, dersler ve ölçme dersi hakkında bazı sorulara cevap verdiler. Bu sorular akademik bir özen olmadan yazıldı. Örneğin birinci soru “Üniversite öğrencisi olmaktan çok mutluyum.”

Her soru için öğrenciler 0’dan 10’a kadar bir sayı belirttiler. 10 tamamen katıldıklarını, 0 işe hiç katılmadıkları gösteriyor. Datasetini aşağıdaki linkte paylaştım. Öğrencilerden sorulara verdiği cevaplarla birlikte, cinsiyet, 1. öğretim/2. öğretim ve A grubu/B grubu verileri istenildi. Böylece 3 faktörlü ve her faktörü 2 level içeren bir ANOVA için veri toplanmış oldu.

Aşağıda yazdığım kod, 12 farklı ANOVAyı yapıp, anlamlı farklılıkları bulup , bu anlamlı farklılıklar için betimleyici istatistikleri rapor ediyor. Süreci otome ettiğim için bütün bu işlemler 1 dakikadan kısa sürüyor. Bu kod kullanılarak (modifiye edilerek) modelin aynı ama bağımlı değişkenin farklı olduğu sınırsız sayıda ANOVA saniyeler içinde koşulup, anlamlı farklılıklar rapor edilebilir.

Questions

Each question is answered 0 to 10 , 10 indicating strongly aggree. You can find the data set here (see).

item1: I am very happy in college

item2: I enjoy my assessment class

item3: I study enough

item4: Not all courses are fun

item5: I wont be here if attendance is not an issue

item6: Not all my courses are useful

item7: I get bored in my assessment course

item8: I willingly attend all my courses

item9: Each and every teacher candidate should take an assesment in education course

item10: Being a college student helps my personal growth

item11: I would take an advanced assessment course

item12: I attend my courses only because its obligatory.

Data

Here is how the score distributions by items look like.

Here is the first 5 rows

sec grp gndr i1 i2 i3 i4 i5 i6 i7 i8 i9 i10 i11 i12
i a m 9 2 1 10 3 10 3 2 10 4 7 10
i a m 8 5 1 8 6 5 4 3 5 8 1 6
i a m 8 0 0 8 3 0 5 4 10 10 8 0
i a m 4 2 2 10 0 10 0 10 10 0 8 0
i a m 5 5 2 10 10 10 5 5 10 0 0 5

Automated ANOVA

Below code will run ANOVA on each 12 item, a 3-way between subjects, each factor has only 2 levels. No interactions included but can be added easily. Only the p values are saved.

pval=matrix(0,ncol=4,nrow=12)
for (i in c(4:15)){
  y=data[,i]
  model<-aov( y~data$sec+data$grp+data$gndr,na.action = na.omit)
  pval[i-3,]=c(as.numeric(summary(model)[[1]][5][[1]][1:3]),mean(y,na.rm=T))
  }
pval=data.frame(round(pval,3))
colnames(pval)=c("sec","grp","gndr","overallmean")
pval
##      sec   grp  gndr overallmean
## 1  0.033 0.181 0.001       7.014
## 2  0.081 0.279 0.017       5.514
## 3  0.415 0.303 0.005       4.725
## 4  0.987 0.466 0.137       8.801
## 5  0.951 0.053 0.171       4.542
## 6  0.508 0.479 0.831       7.113
## 7  0.137 0.881 0.085       4.617
## 8  0.932 0.316 0.009       4.099
## 9  0.390 0.196 0.293       7.979
## 10 0.946 0.188 0.293       5.319
## 11 0.017 0.487 0.359       3.493
## 12 0.553 0.615 0.010       4.218
pval[,1:3]<.05
##         sec   grp  gndr
##  [1,]  TRUE FALSE  TRUE
##  [2,] FALSE FALSE  TRUE
##  [3,] FALSE FALSE  TRUE
##  [4,] FALSE FALSE FALSE
##  [5,] FALSE FALSE FALSE
##  [6,] FALSE FALSE FALSE
##  [7,] FALSE FALSE FALSE
##  [8,] FALSE FALSE  TRUE
##  [9,] FALSE FALSE FALSE
## [10,] FALSE FALSE FALSE
## [11,]  TRUE FALSE FALSE
## [12,] FALSE FALSE  TRUE

As you can see some factors were significant. For example the p value for gender effect on question #1 is .001. I now locate significant p values (no familywise error rate correction is needed as I consider each question separately. Wont be hard to adjust though.)

which(pval<.05, arr.ind=TRUE)
##      row col
## [1,]   1   1
## [2,]  11   1
## [3,]   1   3
## [4,]   2   3
## [5,]   3   3
## [6,]   8   3
## [7,]  12   3

Lets do some automated reporting

library(psych)
sigmat=which(pval<.05, arr.ind=TRUE)
for (i in 1:nrow(sigmat))
  {
  y=data[,as.numeric(sigmat[i,1])+3]
  x=data[,as.numeric(sigmat[i,2])]
  print(paste("item",as.numeric(sigmat[i,1])))
  print(describeBy(y,x))
  print(c("##############"))
}
## [1] "item 1"
## group: 
##   vars n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 6 4.33 3.08    3.5    4.33 1.48   1  10     9  0.8    -0.84 1.26
## -------------------------------------------------------- 
## group: i
##   vars  n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 71 6.82 3.09      8    7.21 2.97   0  10    10 -0.67    -0.58 0.37
## -------------------------------------------------------- 
## group: ii
##   vars  n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 64 7.48 2.99    8.5    7.98 2.22   0  10    10 -1.11     0.22 0.37
## [1] "##############"
## [1] "item 11"
## group: 
##   vars n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 6 7.33 3.14      9    7.33 0.74   2  10     8 -0.71    -1.44 1.28
## -------------------------------------------------------- 
## group: i
##   vars  n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 71 3.34 2.98      3    3.02 2.97   0  10    10 0.64    -0.62 0.35
## -------------------------------------------------------- 
## group: ii
##   vars  n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 65 3.31 3.62      2    2.92 2.97   0  10    10 0.75    -0.83 0.45
## [1] "##############"
## [1] "item 1"
## group: 
##   vars n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 4 3.75 0.96    3.5    3.75 0.74   3   5     2 0.32    -2.08 0.48
## -------------------------------------------------------- 
## group: f
##   vars  n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 87 7.78 2.86      9    8.28 1.48   0  10    10 -1.32     0.92 0.31
## -------------------------------------------------------- 
## group: m
##   vars  n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 50 5.94 3.13    5.5    6.15 3.71   0  10    10 -0.29    -1.01 0.44
## [1] "##############"
## [1] "item 2"
## group: 
##   vars n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 4 4.25 1.71    4.5    4.25 1.48   2   6     4 -0.28    -1.96 0.85
## -------------------------------------------------------- 
## group: f
##   vars  n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 85 5.99 2.81      6     6.1 2.97   0  10    10 -0.17    -0.77 0.31
## -------------------------------------------------------- 
## group: m
##   vars  n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 51 4.82 2.67      5    4.88 2.97   0  10    10 -0.15    -0.89 0.37
## [1] "##############"
## [1] "item 3"
## group: 
##   vars n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 4    5 3.83      4       5 2.97   2  10     8 0.32    -2.08 1.91
## -------------------------------------------------------- 
## group: f
##   vars  n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 87 5.24 2.89      5    5.32 2.97   0  10    10 -0.18    -0.79 0.31
## -------------------------------------------------------- 
## group: m
##   vars  n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 51 3.82 2.36      3    3.68 2.97   0  10    10 0.47     -0.4 0.33
## [1] "##############"
## [1] "item 8"
## group: 
##   vars n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 4 4.25 3.95      3    4.25 1.48   1  10     9 0.62    -1.75 1.97
## -------------------------------------------------------- 
## group: f
##   vars  n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 87 4.69 3.38      5    4.62 4.45   0  10    10    0    -1.17 0.36
## -------------------------------------------------------- 
## group: m
##   vars  n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 50 3.06 2.64      3    2.83 2.97   0  10    10 0.47    -0.55 0.37
## [1] "##############"
## [1] "item 12"
## group: 
##   vars n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 4    2 1.63      2       2 1.48   0   4     4    0    -1.88 0.82
## -------------------------------------------------------- 
## group: f
##   vars  n mean   sd median trimmed  mad min max range skew kurtosis   se
## 1    1 87 3.66 3.64      2    3.35 2.97   0  10    10 0.67    -1.06 0.39
## -------------------------------------------------------- 
## group: m
##   vars  n mean   sd median trimmed  mad min max range  skew kurtosis   se
## 1    1 51 5.35 3.41      5    5.44 4.45   0  10    10 -0.04    -1.32 0.48
## [1] "##############"

For example scores on “item12: I attend my courses only because its obligatory.” differ significantly for gender. Male students report an overall score of 5.35 whereas Female students report 3.66, indicating female students attend courses more willingly.