AHPtools was primarily developed to help me in my
research on consistency of Pairwise Comparison Matrices (PCMs) in the
Analytic Hierarchy Process (AHP) .
The functions in the AHPtools package are:
createPCMcreateRandomPCMcreateLogicalPCMimproveCRrevisedConsistencytriadConsistency (aka consEval)triadReversalIn the following sections are some illustrative use cases of
consistency research in AHP that can be achieved using
AHPtools. The Consistency Ratio (CR) based, extant
consistency measure has been shown by several researchers to have some
limitations. AHPtools provides a compendium of functions
that could help research on consistency in the AHP.
This vignette is an attempt to document some possible use cases for
consistency ratio. The vignette is not complete, and will hopefully grow
with my research. Two of my research papers which make use of
AHPtools functions are (Bose, 2023 and 2022).
# if (!requireNamespace("AHPtools", quietly = TRUE)) {
# install.packages("AHPtools")
# }
library(AHPtools)
devtools::load_all(".")
#> ℹ Loading AHPtools
suppressWarnings(suppressMessages(library(dplyr)))
suppressWarnings(suppressMessages(library(knitr)))
suppressWarnings(suppressMessages(library(kableExtra)))
library(knitr)
library(kableExtra)The list of functions with the function signatures can be obtained:
#> createRandomPCM : function (ord)
#> sensitivity : function (PCM, typePCM = TRUE, granularityLow = TRUE)
#> revisedConsistency : function (PCM, typePCM = TRUE)
#> createLogicalPCM : function (ord, prefVec = rep(NA, ord), granularityLow = TRUE)
#> createPCM : function (vec)
#> triadReversal : function (PCM, typePCM = TRUE)
#> CR : function (PCM, typePCM = TRUE)
#> triadConsistency : function (pcm, typePCM = TRUE)
#> consEval : function (pcm, typePCM = TRUE)
#> improveCR : function (PCM, typePCM = TRUE)The manual for any function, e.g. CR, can be pulled out
from CRAN using the following R command.
help("CR", package="AHPtools")
#> ℹ Rendering development documentation for "CR"The Consistency Ratio is used as a measure of consistency for a PCM in the AHP. Random PCMs used by Saaty to construct these thresholds, specifically Random Indices for various orders, is one reason why consistency evaluation is biased and often flawed.
Needless to say Random PCMs are hardly ever consistent. The
possibility of a PCM meeting the CR consistency threshold reduces even
further with increase in the order of the PCM. This Research nugget
demonstrates the testing of CR consistency thresholds on simulated
logical PCMs. Logical PCMs such as are assigned by human users are more
likely to be representative candidates for benchmarking PCM
consistency thresholds.
Consistency Ratios of simulated logical PCMs and
simulated random PCMs for various orders are compared. In
the following code snippet, 100 PCMs of each order for both the
categories are simulated and the CR values are captured.
runs <- rep(3:12, each=100)
R <- unlist(lapply(runs, function(x) CR(createRandomPCM(x))$CR))
L <- lapply(runs, function(x) CR(createLogicalPCM(x)))
Lcr <- unlist(lapply(L, function(x) x$CR))
Lcons <- unlist(lapply(L, function(x) x$CRconsistent))
exp1DF <- data.frame(Order=runs,Random.PCM=R,Logical.PCM=Lcr,Inconsistent=!Lcons)We can get the order-wise mean CRs for the two
categories of PCM, as follows.
#suppressPackageStartupMessages(library(dplyr))
library(dplyr)
summaryExp1 <- exp1DF %>% group_by(Order) %>%
summarise(Random.PCM=mean(Random.PCM), Logical.PCM=mean(Logical.PCM),
"Logical PCMs"=mean(Inconsistent))
summaryExp1$Random.PCM <- round(summaryExp1$Random.PCM,3)
summaryExp1$Logical.PCM <- round(summaryExp1$Logical.PCM,3)
summaryExp1$`Logical PCMs` <- round(summaryExp1$`Logical PCMs`,2)
kable(summaryExp1) %>%
add_header_above(c(" "=1, "Average CR of"=2, "Inconsistent"=1), line=FALSE)| Order | Random.PCM | Logical.PCM | Logical PCMs |
|---|---|---|---|
| 3 | 0.924 | 0.035 | 0.31 |
| 4 | 0.972 | 0.035 | 0.11 |
| 5 | 1.006 | 0.039 | 0.06 |
| 6 | 0.978 | 0.035 | 0.02 |
| 7 | 0.985 | 0.040 | 0.02 |
| 8 | 0.999 | 0.037 | 0.01 |
| 9 | 0.981 | 0.034 | 0.00 |
| 10 | 1.018 | 0.038 | 0.00 |
| 11 | 1.011 | 0.036 | 0.00 |
| 12 | 0.996 | 0.038 | 0.00 |
It is clear that for logically constructed PCMs the CR
threshold is far too liberal and would possibly result in large number
of false positives - i.e. inconsistent PCMs adjudged as consistent by
the CR consistency evaluation criterion.
The proportion of logical PCMs for each order that are CR inconsistent is less than 5% for orders greater than 4. For order 3 however, 31% of PCMs are CR inconsistent.
Given that the logically constructed PCMs are representative of expert assigned PCMs, this underscores the possibility of bias in consistency evaluation by using the CR method.
Given the criticality of consistency there have been several methods to enhance the consistency of a PCM. One of the earliest of these was due to Harker.
The function improveCR takes any PCM as input and using
Harker’s method repeatedly, attempts to bring the CR to an acceptable
value.
This research nugget highlights a drawback in forcing the CR to a value that is deemed consistent. The principal eigenvector of a PCM indicates the relative importance of the alternatives. Improving consistency by focusing on errant pairwise ratios in isolation often changes the overall preferences for the alternatives.
The code in this section illustrates this by simulating 10 random
PCMs of order 7, and then using improveCR to bring the CR
to an acceptable consistency level.
set.seed(93)
ind <- type <- ConsistencyRatio <- inverts <- vecRanks <- c()
for (i in 1:5) {
p1 <- createRandomPCM(7)
imp <- improveCR(p1)
ind <- c(ind, c(i,"", ""))
type <- c(type, c("Original", "Improved", ""))
ConsistencyRatio <- c(ConsistencyRatio, c(round(imp$CR.original,4),
round(imp$suggestedCR,4), ""))
inverts <- c(inverts, c(imp$inversions, " ", ""))
vecRanks <- c(vecRanks, c(imp$oriRank, imp$impRank, ""))
}
df <- data.frame(ind, type, CR=ConsistencyRatio, vecRanks, inverts)
kable(df, linesep=FALSE) %>% kable_styling(position = "center") | ind | type | CR | vecRanks | inverts |
|---|---|---|---|---|
| 1 | Original | 1.0153 | 5 7 2 1 4 3 6 | 13 |
| Improved | 0.0033 | 6 7 1 2 3 4 5 | ||
| 2 | Original | 0.7586 | 6 7 2 4 5 1 3 | 0 |
| Improved | 0.009 | 6 7 2 4 5 1 3 | ||
| 3 | Original | 1.4475 | 3 4 5 1 2 7 6 | 1 |
| Improved | 0.0099 | 3 4 5 1 2 6 7 | ||
| 4 | Original | 0.7453 | 6 1 5 2 3 4 7 | 0 |
| Improved | 0.0038 | 6 1 5 2 3 4 7 | ||
| 5 | Original | 0.8199 | 4 7 2 3 5 6 1 | 7 |
| Improved | 0.0093 | 5 7 2 3 4 6 1 | ||
It is seen that while the CR value has been drastically improved, the ranks for the alternatives have changed as indicated by the number of inversions in 3 out of the 5 simulated random PCMs.
In this function all different triads of elements of a PCM are
chosen. Triads are subsets of 3 elements chosen from the n
alternatives of an order-n PCM. A triad reversal is said to occur if any
two elements of the eigen vector of an order-3 PCM show a reversal in
preference when compared to the corresponding elements of the entire
eigen vector.
An example to illustrate this follows.
pcmVec <- c(1/3,1/2,1/8, 1,1/6, 3)
pcm <- createPCM(pcmVec)
colnames(pcm) <- rownames(pcm) <- c('a1', 'a2', 'a3', 'a4')
pcm
#> a1 a2 a3 a4
#> a1 1 0.3333333 0.5000000 0.1250000
#> a2 3 1.0000000 1.0000000 0.1666667
#> a3 2 1.0000000 1.0000000 3.0000000
#> a4 8 6.0000000 0.3333333 1.0000000The eigen vector of this PCM is ( 0.1231476, 0.2772163, 0.6444789, 0.701878 ).
The triad reversals for this PCM can be obtained as follows.
trdf <- triadReversal(pcm)
trdf
#> triadE1 triadE2 triadE3 prefRev pcmWeightE1 pcmWeightE2 pcmWeightE3
#> 1 a2 a3 a1 2.661258 0.2772163 0.6444789 0.1231476
#> 2 a3 a4 a1 1.427076 0.6444789 0.7018780 0.1231476
#> 3 a3 a4 a2 1.246666 0.6444789 0.7018780 0.2772163
#> triadWeightE1 triadWeightE2 triadWeightE3
#> 1 0.7238135 0.6323093 0.2761865
#> 2 0.7832404 0.5977243 0.1710559
#> 3 0.7238135 0.6323093 0.2761865This PCM has 3 triad reversals indicated by the 3 rows displayed above.
Let us examine the first row displayed. It shows that the triad PCM constructed by taking the row-columns corresponding to a2, a3, a1 has a preference reversal of 2.661258.
The following code segment displays this PCM.
pcm3Vec <- c(1,3, 2)
pcm3 <- createPCM(pcm3Vec)
colnames(pcm3) <- rownames(pcm3) <- c('a2', 'a3', 'a1')
pcm3
#> a2 a3 a1
#> a2 1.0000000 1.0 3
#> a3 1.0000000 1.0 2
#> a1 0.3333333 0.5 1The eigen vector of this triad PCM is ( 0.7238135, 0.6323093, 0.2761865 ).
The corresponding elements for a2, a3 from the original order-4 PCM is ( 0.2772163, 0.6444789, 0.1231476 ). For a2 and a3 the triadE1 and triadE2 elements have values 0.2772163 and 0.6444789.
The corresponding values for a2, a3 for the triad PCM are 0.7238135, 0.6323093.
There is a preference reversal in this triad because the preference ratio for a2:a3, is more than 1 for one of the comparisons and less for the other.
As per the eigen vector of the entire order-4 PCM, the preference ratio of a2:a3 is 0.2772163 / 0.6444789 = 0.4301402, indicating a higher preference for a3 compared to a2.
When we consider the triad a2, a3, a1 - row 1 of the data displayed -
we see that the preference for a2 vis-a-vis a3, is
0.7238135 / 0.6323093 = 1.1447143, indicating a higher preference for a2
compared to a3.
This reversal in preference is counter-intuitive and indicative of inconsistency. The measure of intensity of the reversal is taken to be
max(0.4301402/1.1447143, 1.1447143/0.4301402)
#> [1] 2.661259as in the first row of the triad reversals data displayed.
Similarly, considering all possible triads - there are \(\binom{4}{3}\) triads possible - we see that there are three preference reversals, of values
trdf[,4]
#> [1] 2.661258 1.427076 1.246666Triads form the smallest units prone to inconsistency - a tuple is
never inconsistent because transverse elements in a pairwise comparison
matrix are reciprocals of one another. For an order n PCM
there are \(\binom{n}{3}\) possible
triads, and for each triad there are \(\binom{3}{2}=3\) pairs which can have
preference reversals.
Thus, for an order-n PCM the maximum possible number of
preference reversals is \(3 \times
\binom{n}{3}\).
In this example we see that there are 3 preference reversals out of a
maximum possible of \(3 \times
\binom{4}{3}\) = 12, with a maximum intensity of 2.661259 between
a2 and a3 considering a1 as the
third element in the triad.
The AHPtools package also has a function
triadConsistency which summarizes the results of observed
reversals for any given PCM.
triadConsistency(pcm)
#> $logitConsistency
#> [1] 2.261657e-09
#>
#> $prop3Rev
#> [1] 0.25
#>
#> $max3Rev
#> [1] 2.661258
#>
#> $triadsData
#> triadE1 triadE2 triadE3 prefRev pcmWeightE1 pcmWeightE2 triadWeightE1
#> 1 a2 a3 a1 2.661258 0.2772 0.6445 0.7238
#> 2 a3 a4 a1 1.427076 0.6445 0.7019 0.7832
#> 3 a3 a4 a2 1.246666 0.6445 0.7019 0.7238
#> triadWeightE2
#> 1 0.6323
#> 2 0.5977
#> 3 0.6323The $triadsData is a list of all preference reversals
observed and we have already seen this as an output of
triadReversal(pcm). We have already observed that there are
3 preference reversals out of a maximum possible 12. This gives the
value of triadConsistency(pcm)$prop3Rev.
Similarly, the maximum preference reversal is given by
triadConsistency(pcm)$max3Rev and is 2.661258 as we have
already seen.
The triadConsistency(pcm)$logitConsistency value is an
indicator of the consistency of the PCM. On a scale of 0 to 1, this
value indicates that the PCM is highly likely to be inconsistent. The
evaluation is a part of my on-going research and the documentation for
this package will be suitably updated on its successful conclusion.
AHPtools functionsAmarnath Bose, (2023), “Improving consistency classification: An innovative benchmark‐based approach for the AHP”, Journal of Multi-Criteria Decision Analysis, https://doi.org/10.1002/mcda.1821, 31(1-2).
Amarnath Bose, (2022) “Consistency re-evaluation in analytic hierarchy process based on simulated consistent matrices” Journal of Multi-criteria Decision Analysis, https://doi.org/10.1002/mcda.1784, 29(5-6).