The purpose of an agreement plot is to visually demonstrate the reliability of a subjective classification or rating system (i.e. ratings from two different judges.) In this example, we have ratings from husbands and wives about dancing…how often is it fun? We use data included with the vcd package.
library(vcd)
## Loading required package: MASS
## Loading required package: grid
## Loading required package: colorspace
data(SexualFun)
myData = SexualFun
myData
## Wife
## Husband Never Fun Fairly Often Very Often Always fun
## Never Fun 7 7 2 3
## Fairly Often 2 8 3 7
## Very Often 1 5 4 9
## Always fun 2 8 9 14
We use the agreementplot() function from the vcd package as follows (note the use of t to transpose the data:
ap = agreementplot(t(myData))
We have thus produced an agreement plot, also called a Bangdiwala's Observer Agreement Chart. Note that our agreement plot is a representation of a k x k confusion matrix. The observed and expected diagonal elements are represented by superposed black and white rectangles, respectively. The extent to which the rectangles are above or below the line indicates the extent of any disagreement. (above and/or below indicates direction of the disagreement). The function also computes two statistic measuring the strength of agreement (relation of respective area sums). The first statistic is accessed using the term Bandiwala. This statistic is the unweighted agreement strength statistic. The second statistic makes an adjustment for ordered ratings, and is accessed using the code Bangdiwala_Weighted. Both statistics are measured on a scale from 0 to 1, where 1 indicates perfect agreement and 0 indicates perfect disagreement.
ap$Bangdiwala # we have called our agreement plot ap
## [,1]
## [1,] 0.1465
ap$Bangdiwala_Weighted
## [,1]
## [1,] 0.4982
ap$weights # the weight vector used
## [1] 1.0000 0.8889