1. Data two factors

+++++++++ 1 rater judges 6 subjects twice

+++++++++++ Whether the judgment is reliable (x1 ~ x2)? —>High ICCx1x2

(A) Calculate ICC by data
dat
##   x1 x2 mean var x1_mean_sq x2_mean_sq
## 1  9  7  8.0 2.0       1.00       1.00
## 2  6  5  5.5 0.5       0.25       0.25
## 3  8  6  7.0 2.0       1.00       1.00
## 4  7  5  6.0 2.0       1.00       1.00
## 5 10  7  8.5 4.5       2.25       2.25
## 6  6  5  5.5 0.5       0.25       0.25
Mx_dat=t(cbind(x1,x2))
Mx_dat                            # Matrix of x1 x2
##    [,1] [,2] [,3] [,4] [,5] [,6]
## x1    9    6    8    7   10    6
## x2    7    5    6    5    7    5
cov(Mx_dat)                       # Covariance matrix of x1,x2
##      [,1] [,2] [,3] [,4] [,5] [,6]
## [1,]    2  1.0    2    2  3.0  1.0
## [2,]    1  0.5    1    1  1.5  0.5
## [3,]    2  1.0    2    2  3.0  1.0
## [4,]    2  1.0    2    2  3.0  1.0
## [5,]    3  1.5    3    3  4.5  1.5
## [6,]    1  0.5    1    1  1.5  0.5
ICCx1x2
## [1] 0.2721519

(B) Calculate ICC using ANOVA parameters

datM_cv              #Convet datM to long form with x1,x2 = time of judgments
##    Subj Time value
## 1     1   x1     9
## 2     2   x1     6
## 3     3   x1     8
## 4     4   x1     7
## 5     5   x1    10
## 6     6   x1     6
## 7     1   x2     7
## 8     2   x2     5
## 9     3   x2     6
## 10    4   x2     5
## 11    5   x2     7
## 12    6   x2     5
model=lm(value~Subj,dat=datM_cv); anova(model)
## Analysis of Variance Table
## 
## Response: value
##           Df Sum Sq Mean Sq F value Pr(>F)
## Subj       5  16.75  3.3500  1.7478 0.2575
## Residuals  6  11.50  1.9167

n= Subj No. = Group = 6

k= observation = Time of judgement = 2

SumSq of Subj = variance between group (SSb); Degree of freedom between group (DFb)= n-1 (6-1) =5

SumSq of Residual = variance within group (SSw); Degree of freedom within group (DFw)= n x (k-1) = 6

SSb=k*sum(((x1+x2)/2-mean(c(x1,x2)))^2)

SSw=sum((x1-x2)^2/2) = sum((x1-mean_x1x2)^2 + (x2-mean_x1x2)^2)

ICCx1x2= ((SSb/DFb)-(SSw/DFw)) / ((SSb/DFb)+(k-1)*(SSw/DFw))

ICCx1x2= (3.3500-1.9167)/(3.3500+(2-1)*1.9167)
ICCx1x2                                        
## [1] 0.2721438

Remark: similar with that calculated above

(C) Calculate ANOVA parameters using variance value of x1 and x2

dat= dat%>% mutate(Sb=2*(mean-mean(mean))^2,Sw=x1_mean_sq+x2_mean_sq)
dat
##   x1 x2 mean var x1_mean_sq x2_mean_sq    Sb  Sw
## 1  9  7  8.0 2.0       1.00       1.00 3.125 2.0
## 2  6  5  5.5 0.5       0.25       0.25 3.125 0.5
## 3  8  6  7.0 2.0       1.00       1.00 0.125 2.0
## 4  7  5  6.0 2.0       1.00       1.00 1.125 2.0
## 5 10  7  8.5 4.5       2.25       2.25 6.125 4.5
## 6  6  5  5.5 0.5       0.25       0.25 3.125 0.5
SSb=sum(dat$Sb);SSb
## [1] 16.75
SSw=sum(dat$var)         # Equal with sum(dat$Sw)
SSw
## [1] 11.5
plot

2. Data four factors

+++++++++ 1 rater judges 6 subjects four times

+++++++++++ Whether the judgment is reliable (x1x2x3~x4)? —>High ICCx1x2X3X4

(A) Calculate ICC by data
dat
##      x1 x2 x3 x4
## [1,]  9  7  6  8
## [2,]  6  5  8  5
## [3,]  8  6  7  9
## [4,]  7  5  5  6
## [5,] 10  7  9  7
## [6,]  6  5  6  6
Mx_dat=t(dat)
Mx_dat
##    [,1] [,2] [,3] [,4] [,5] [,6]
## x1    9    6    8    7   10    6
## x2    7    5    6    5    7    5
## x3    6    8    7    5    9    6
## x4    8    5    9    6    7    6
cov(Mx_dat)
##            [,1]       [,2]       [,3]       [,4]      [,5]      [,6]
## [1,]  1.6666667 -1.0000000  1.0000000  1.1666667 0.5000000 0.1666667
## [2,] -1.0000000  2.0000000 -0.3333333 -0.3333333 1.3333333 0.3333333
## [3,]  1.0000000 -0.3333333  1.6666667  0.8333333 0.1666667 0.5000000
## [4,]  1.1666667 -0.3333333  0.8333333  0.9166667 0.7500000 0.2500000
## [5,]  0.5000000  1.3333333  0.1666667  0.7500000 2.2500000 0.4166667
## [6,]  0.1666667  0.3333333  0.5000000  0.2500000 0.4166667 0.2500000

```To calculate ICC we need mean of each Subject not mean of x1,x2,x3,x4