Inter-rater agreement for Dichotomous data (Relevant and
Observable)
We want to determine the agreement for single Facets for single
Tasks, irrespective of other facets or tasks. It is not possible to
calculate any forms of kappa or alpha for single items, and comparisons
across facets and tasks would not be relevant for answering the research
question. Therefore, the best approach is to report the proportion of
raters in agreement.
Presentation
Presentation relevant facets
This table presents the 15 most relevant facets on the Presentation
task as well as the number of raters who responded that the facet was
relevant (in lieu of % agreement):
Relevance of NEO Facets on Presentation Task
| Pres_Assertiveness_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Pres_SelfEfficacy_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Pres_Cooperation_Relevant |
1 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Pres_Friendliness_Relevant |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Pres_Sympathy_Relevant |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
0 |
7 |
| Pres_Anxiety_Relevant |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Pres_Liberalism_Relevant |
1 |
1 |
1 |
1 |
0 |
0 |
1 |
1 |
1 |
7 |
| Pres_Gregariousness_Relevant |
0 |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
6 |
| Pres_Cheerfulnessful_Relevant |
1 |
1 |
0 |
1 |
0 |
1 |
1 |
1 |
0 |
6 |
| Pres_Morality_Relevant |
0 |
1 |
1 |
1 |
0 |
0 |
1 |
1 |
1 |
6 |
| Pres_Vulnerability_Relevant |
1 |
1 |
0 |
1 |
0 |
1 |
1 |
1 |
0 |
6 |
| Pres_Emotionality_Relevant |
0 |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
6 |
| Pres_Intellect_Relevant |
1 |
1 |
0 |
1 |
0 |
0 |
1 |
1 |
1 |
6 |
| Pres_Altruism_Relevant |
1 |
1 |
0 |
0 |
0 |
1 |
1 |
1 |
0 |
5 |
| Pres_Modesty_Relevant |
0 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
5 |
Presentation observable facets
This table presents the 15 most observable facets on the Presentation
task:
Observability of NEO Facets on Presentation Task
| Pres_Assertiveness_Observable |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Pres_Gregariousness_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Pres_Cheerfulnessful_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Pres_SelfEfficacy_Observable |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
8 |
| Pres_Friendliness_Observable |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Pres_Cooperation_Observable |
1 |
0 |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
7 |
| Pres_Sympathy_Observable |
1 |
1 |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
7 |
| Pres_Anxiety_Observable |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Pres_Altruism_Observable |
1 |
0 |
1 |
0 |
1 |
1 |
0 |
1 |
1 |
6 |
| Pres_Modesty_Observable |
0 |
1 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
6 |
| Pres_Depression_Observable |
0 |
1 |
1 |
1 |
0 |
1 |
0 |
1 |
1 |
6 |
| Pres_SelfConsciousness_Observable |
0 |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
6 |
| Pres_Vulnerability_Observable |
1 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
6 |
| Pres_Emotionality_Observable |
0 |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
6 |
| Pres_Orderliness_Observable |
0 |
0 |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
5 |
Presentation optimal levels
This table presents the average of optimal level ratings for the
presentation task.
Optimal Level of Relevant NEO Facets on Presentation
Task
| Pres_Friendliness_OptLvl |
NA |
4 |
4 |
5 |
NA |
4 |
5 |
4 |
4 |
4.285714 |
| Pres_Assertiveness_OptLvl |
6 |
6 |
5 |
6 |
4 |
6 |
5 |
6 |
6 |
5.555556 |
| Pres_Cooperation_OptLvl |
4 |
NA |
4 |
5 |
6 |
5 |
6 |
6 |
2 |
4.750000 |
| Pres_Sympathy_OptLvl |
6 |
6 |
4 |
4 |
NA |
5 |
5 |
5 |
1 |
4.500000 |
| Pres_SelfEfficacy_OptLvl |
6 |
6 |
4 |
6 |
6 |
6 |
6 |
6 |
4 |
5.555556 |
| Pres_Anxiety_OptLvl |
NA |
5 |
3 |
1 |
NA |
1 |
2 |
1 |
1 |
2.000000 |
psych::ICC works better because it allows you to handle missing
values. The statistic we are interested in is Average_random_raters.
## Call: ICC(x = pres_optlvl, missing = FALSE, alpha = 0.05, lmer = TRUE)
##
## Intraclass correlation coefficients
## type ICC F df1 df2 p lower bound upper bound
## Single_raters_absolute ICC1 0.52 11 5 48 4.6e-07 0.24 0.88
## Single_random_raters ICC2 0.53 14 5 40 6.4e-08 0.25 0.88
## Single_fixed_raters ICC3 0.59 14 5 40 6.4e-08 0.30 0.90
## Average_raters_absolute ICC1k 0.91 11 5 48 4.6e-07 0.74 0.99
## Average_random_raters ICC2k 0.91 14 5 40 6.4e-08 0.75 0.99
## Average_fixed_raters ICC3k 0.93 14 5 40 6.4e-08 0.79 0.99
##
## Number of subjects = 6 Number of Judges = 9
## See the help file for a discussion of the other 4 McGraw and Wong estimates,
Group Discussion
Group Discussion: Relevance
Relevance of NEO Facets for Group Discussion Task
| Grp_Friendliness_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Grp_Assertiveness_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Grp_Cooperation_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Grp_Gregariousness_Relevant |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
7 |
| Grp_Modesty_Relevant |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
0 |
7 |
| Grp_AchievementStriving_Relevant |
1 |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
7 |
| Grp_Anxiety_Relevant |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
7 |
| Grp_Intellect_Relevant |
1 |
1 |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
7 |
| Grp_SelfEfficacy_Relevant |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
0 |
0 |
6 |
| Grp_SelfConsciousness_Relevant |
0 |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
6 |
| Grp_Anger_Relevant |
0 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
5 |
| Grp_Vulnerability_Relevant |
0 |
1 |
0 |
1 |
0 |
1 |
0 |
1 |
1 |
5 |
| Grp_Imagination_Relevant |
0 |
0 |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
5 |
| Grp_Cheerfulness_Relevant |
0 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
0 |
4 |
| Grp_Trust_Relevant |
1 |
1 |
0 |
1 |
0 |
0 |
1 |
0 |
0 |
4 |
Group Discussion: Observability
Observability of NEO Facets on Group Discussion Task
| Grp_Friendliness_Observable |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Grp_Gregariousness_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Grp_Assertiveness_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Grp_Cooperation_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Grp_Anxiety_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Grp_Cheerfulness_Observable |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
7 |
| Grp_Modesty_Observable |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Grp_SelfEfficacy_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
0 |
0 |
1 |
6 |
| Grp_Anger_Observable |
0 |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
6 |
| Grp_Intellect_Observable |
0 |
1 |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
6 |
| Grp_Morality_Observable |
0 |
1 |
0 |
1 |
1 |
1 |
0 |
0 |
1 |
5 |
| Grp_AchievementStriving_Observable |
0 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
5 |
| Grp_SelfConsciousness_Observable |
0 |
0 |
1 |
1 |
0 |
1 |
0 |
1 |
1 |
5 |
| Grp_Vulnerability_Observable |
0 |
1 |
0 |
1 |
0 |
1 |
0 |
1 |
1 |
5 |
| Grp_Trust_Observable |
0 |
1 |
0 |
1 |
0 |
0 |
1 |
0 |
1 |
4 |
Group Discussion: Optimal level
Optimal Level of Relevant NEO Facets on Group Discussion
Task
| Grp_Friendliness_OptLvl |
5 |
6 |
4 |
4 |
6 |
5 |
5 |
4 |
5 |
4.888889 |
| Grp_Gregariousness_OptLvl |
NA |
6 |
NA |
4 |
6 |
6 |
6 |
5 |
5 |
5.428571 |
| Grp_Assertiveness_OptLvl |
6 |
6 |
4 |
6 |
5 |
5 |
6 |
5 |
6 |
5.444444 |
| Grp_Cooperation_OptLvl |
6 |
2 |
3 |
3 |
3 |
4 |
5 |
3 |
6 |
3.888889 |
| Grp_Modesty_OptLvl |
6 |
3 |
3 |
2 |
NA |
2 |
4 |
2 |
1 |
2.875000 |
| Grp_AchievementStriving_OptLvl |
6 |
NA |
NA |
6 |
5 |
5 |
5 |
5 |
6 |
5.428571 |
## Call: ICC(x = grp_optlvl, missing = FALSE, alpha = 0.05, lmer = TRUE)
##
## Intraclass correlation coefficients
## type ICC F df1 df2 p lower bound upper bound
## Single_raters_absolute ICC1 0.47 8.8 5 48 5.3e-06 0.19 0.86
## Single_random_raters ICC2 0.47 11.1 5 40 9.7e-07 0.20 0.86
## Single_fixed_raters ICC3 0.53 11.1 5 40 9.7e-07 0.24 0.88
## Average_raters_absolute ICC1k 0.89 8.8 5 48 5.3e-06 0.68 0.98
## Average_random_raters ICC2k 0.89 11.1 5 40 9.7e-07 0.69 0.98
## Average_fixed_raters ICC3k 0.91 11.1 5 40 9.7e-07 0.74 0.99
##
## Number of subjects = 6 Number of Judges = 9
## See the help file for a discussion of the other 4 McGraw and Wong estimates,
Critique
Critique: Relevance
Relevance of NEO Facets for Critique Task
| Crit_Assertiveness_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Crit_Anger_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Crit_Friendliness_Relevant |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Crit_Altruism_Relevant |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Crit_Cooperation_Relevant |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
8 |
| Crit_Dutifulness_Relevant |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Crit_Morality_Relevant |
0 |
1 |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
7 |
| Crit_Cautiousness_Relevant |
1 |
0 |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
7 |
| Crit_Anxiety_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
0 |
1 |
0 |
7 |
| Crit_Sympathy_Relevant |
0 |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
6 |
| Crit_Cheerfulness_Relevant |
1 |
1 |
1 |
0 |
0 |
1 |
1 |
0 |
0 |
5 |
| Crit_SelfEfficacy_Relevant |
1 |
0 |
1 |
1 |
1 |
0 |
1 |
0 |
0 |
5 |
| Crit_Orderliness_Relevant |
1 |
0 |
0 |
1 |
0 |
1 |
0 |
1 |
1 |
5 |
| Crit_AchievementStriving_Relevant |
1 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
0 |
5 |
| Crit_SelfConsciousness_Relevant |
1 |
1 |
0 |
1 |
0 |
1 |
0 |
1 |
0 |
5 |
Critique: Observability
Observability of NEO Facets on Critique Task
| Crit_Anger_Observable |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Crit_Assertiveness_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Crit_Altruism_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Crit_Cooperation_Observable |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
8 |
| Crit_Anxiety_Observable |
1 |
1 |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
8 |
| Crit_Friendliness_Observable |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Crit_Sympathy_Observable |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
7 |
| Crit_Dutifulness_Observable |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
7 |
| Crit_Cheerfulness_Observable |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
0 |
1 |
6 |
| Crit_Orderliness_Observable |
0 |
0 |
1 |
1 |
1 |
1 |
0 |
1 |
1 |
6 |
| Crit_SelfConsciousness_Observable |
0 |
1 |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
6 |
| Crit_Vulnerability_Observable |
0 |
1 |
1 |
1 |
0 |
1 |
0 |
1 |
1 |
6 |
| Crit_SelfEfficacy_Observable |
0 |
0 |
1 |
1 |
1 |
0 |
1 |
0 |
1 |
5 |
| Crit_AchievementStriving_Observable |
0 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
5 |
| Crit_SelfDiscipline_Observable |
0 |
1 |
1 |
0 |
0 |
1 |
1 |
0 |
1 |
5 |
Critique: Optimal level
Optimal Level of Relevant NEO Facets on Critique Task
| Crit_Friendliness_OptLvl |
NA |
3 |
5 |
4 |
5 |
4 |
5 |
4 |
3 |
4.125 |
| Crit_Assertiveness_OptLvl |
4 |
6 |
4 |
5 |
4 |
5 |
5 |
6 |
6 |
5.000 |
| Crit_Altruism_OptLvl |
NA |
5 |
4 |
5 |
4 |
5 |
5 |
5 |
4 |
4.625 |
| Crit_Cooperation_OptLvl |
4 |
1 |
3 |
6 |
NA |
4 |
6 |
3 |
2 |
3.625 |
| Crit_Dutifulness_OptLvl |
6 |
5 |
NA |
5 |
6 |
6 |
4 |
5 |
3 |
5.000 |
| Crit_Anger_OptLvl |
2 |
4 |
2 |
1 |
2 |
1 |
1 |
1 |
4 |
2.000 |
## boundary (singular) fit: see help('isSingular')
## Call: ICC(x = crit_optlvl, missing = FALSE, alpha = 0.05, lmer = TRUE)
##
## Intraclass correlation coefficients
## type ICC F df1 df2 p lower bound upper bound
## Single_raters_absolute ICC1 0.49 9.5 5 48 2.4e-06 0.21 0.86
## Single_random_raters ICC2 0.49 9.5 5 40 5.0e-06 0.21 0.86
## Single_fixed_raters ICC3 0.49 9.5 5 40 5.0e-06 0.20 0.86
## Average_raters_absolute ICC1k 0.89 9.5 5 48 2.4e-06 0.70 0.98
## Average_random_raters ICC2k 0.89 9.5 5 40 5.0e-06 0.70 0.98
## Average_fixed_raters ICC3k 0.89 9.5 5 40 5.0e-06 0.69 0.98
##
## Number of subjects = 6 Number of Judges = 9
## See the help file for a discussion of the other 4 McGraw and Wong estimates,
Teaching task
Teaching: Relevance
Relevance of NEO Facets for Teaching Task
| Teach_Friendliness_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Teach_Assertiveness_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Teach_Cheerfulness_Relevant |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Teach_Altruism_Relevant |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Teach_Modesty_Relevant |
1 |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Teach_Intellect_Relevant |
1 |
1 |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
7 |
| Teach_Cooperation_Relevant |
0 |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
6 |
| Teach_Anger_Relevant |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
1 |
0 |
6 |
| Teach_Gregariousness_Relevant |
0 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
5 |
| Teach_Sympathy_Relevant |
0 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
5 |
| Teach_SelfEfficacy_Relevant |
1 |
0 |
0 |
1 |
1 |
0 |
1 |
1 |
0 |
5 |
| Teach_Orderliness_Relevant |
0 |
1 |
0 |
1 |
1 |
0 |
0 |
1 |
1 |
5 |
| Teach_SelfDiscipline_Relevant |
0 |
1 |
0 |
1 |
0 |
0 |
1 |
1 |
1 |
5 |
| Teach_Imagination_Relevant |
1 |
1 |
0 |
1 |
0 |
0 |
0 |
1 |
1 |
5 |
| Teach_ActivityLevel_Relevant |
1 |
0 |
0 |
1 |
0 |
1 |
1 |
0 |
0 |
4 |
Teaching: Observability
Observability of NEO Facets on Teaching Task
| Teach_Friendliness_Observable |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
9 |
| Teach_Cheerfulness_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Teach_Anger_Observable |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
8 |
| Teach_Altruism_Observable |
0 |
1 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
7 |
| Teach_Cooperation_Observable |
0 |
0 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
7 |
| Teach_Gregariousness_Observable |
0 |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
6 |
| Teach_Assertiveness_Observable |
0 |
0 |
1 |
1 |
0 |
1 |
1 |
1 |
1 |
6 |
| Teach_Modesty_Observable |
0 |
0 |
1 |
1 |
0 |
1 |
0 |
1 |
1 |
5 |
| Teach_Sympathy_Observable |
0 |
0 |
0 |
1 |
0 |
1 |
1 |
1 |
1 |
5 |
| Teach_SelfConsciousness_Observable |
0 |
0 |
1 |
1 |
0 |
0 |
1 |
1 |
1 |
5 |
| Teach_Intellect_Observable |
0 |
1 |
0 |
1 |
0 |
0 |
1 |
1 |
1 |
5 |
| Teach_Orderliness_Observable |
0 |
0 |
0 |
1 |
1 |
0 |
0 |
1 |
1 |
4 |
| Teach_Anxiety_Observable |
0 |
0 |
1 |
1 |
0 |
0 |
0 |
1 |
1 |
4 |
| Teach_Depression_Observable |
0 |
1 |
1 |
0 |
0 |
0 |
0 |
1 |
1 |
4 |
| Teach_Vulnerability_Observable |
0 |
1 |
0 |
1 |
0 |
0 |
0 |
1 |
1 |
4 |
Teaching: Optimal level
Optimal Levels of most relevant NEO Facets for Teaching
Task
| Teach_Friendliness_OptLvl |
6 |
6 |
4 |
4 |
4 |
5 |
6 |
6 |
6 |
5.222222 |
| Teach_Assertiveness_OptLvl |
4 |
5 |
2 |
5 |
4 |
4 |
5 |
4 |
5 |
4.222222 |
| Teach_Cheerfulness_OptLvl |
5 |
5 |
4 |
5 |
4 |
6 |
5 |
5 |
5 |
4.888889 |
| Teach_Altruism_OptLvl |
NA |
5 |
4 |
6 |
NA |
5 |
5 |
6 |
5 |
5.142857 |
| Teach_Modesty_OptLvl |
4 |
NA |
4 |
3 |
NA |
4 |
4 |
5 |
3 |
3.857143 |
| Teach_Intellect_OptLvl |
6 |
5 |
NA |
6 |
4 |
NA |
4 |
5 |
6 |
5.142857 |
## Call: ICC(x = teach_optlvl, missing = FALSE, alpha = 0.05, lmer = TRUE)
##
## Intraclass correlation coefficients
## type ICC F df1 df2 p lower bound upper bound
## Single_raters_absolute ICC1 0.25 4.0 5 48 0.00402 0.044 0.72
## Single_random_raters ICC2 0.27 5.4 5 40 0.00066 0.067 0.73
## Single_fixed_raters ICC3 0.33 5.4 5 40 0.00066 0.088 0.78
## Average_raters_absolute ICC1k 0.75 4.0 5 48 0.00402 0.291 0.96
## Average_random_raters ICC2k 0.77 5.4 5 40 0.00066 0.394 0.96
## Average_fixed_raters ICC3k 0.82 5.4 5 40 0.00066 0.466 0.97
##
## Number of subjects = 6 Number of Judges = 9
## See the help file for a discussion of the other 4 McGraw and Wong estimates,
Summary
This summary table displays the number of raters who agreed that each
facet was relevant in each task