The Same/Different test is a discrimination test used to determine whether assessors can detect sensory differences between two products. In each trial, the panelist receives a pair of samples, which may be either identical or different, and must judge whether the two samples are the same or different.
This method is particularly useful when sensory memory may be unreliable, as both samples are presented simultaneously. It is commonly applied in product reformulation, ingredient substitution, or process change studies (Meilgaard et al., 2007).
The Same/Different test relies on several key assumptions:
Binary response: Each assessor must respond either “same” or “different” for each sample pair.
Chance probability: When unsure, assessors may guess, which introduces a guessing probability that must be accounted for in the statistical analysis.
Perceptual confusion: A certain proportion of errors can occur even when a difference is present, due to sensory similarity or assessor uncertainty.
In this example, a total of 200 panelists evaluated 1 sample pair. Each pair consisted either of identical samples (e.g., AA or BB, labeled as “Same”) or different samples (e.g., AB or BA, labeled as “Different”).
The distribution of responses is summarized in a 2x2 confusion matrix, where the rows represent the panelists’ responses and the columns represent the true nature of the sample pairs.
The Chi-square test evaluates whether the distribution of responses (“Same” or “Different”) is independent of the actual sample pair type. It is based on the 2×2 confusion matrix.
confusion_matrix <- table(Response = data$Response, TruePair = data$TruePair)
chisq.test(confusion_matrix, correct = FALSE)##
## Pearson's Chi-squared test
##
## data: confusion_matrix
## X-squared = 50.322, df = 1, p-value = 1.305e-12Since the p-value is less than 0.05, we reject the null hypothesis. This suggests that there is significant association between the type of sample pair received and the panelists’ responses, implying that the panelists were able to discriminate between the samples beyond chance level.
To estimate the sensory discriminability between the sample pairs, we used the samediff() function from the sensR package. This function is based on signal detection theory and allows for direct input of the response frequencies from the confusion matrix (Nyholt et al., 2013).
The function requires four counts:
the number of “same” responses on same pairs,
the number of “different” responses on same pairs,
the number of “same” responses on different pairs,
and the number of “different” responses on different pairs.
library(sensR)
model <- samediff(
nsamesame = 71,
ndiffsame = 29,
nsamediff = 21,
ndiffdiff = 79
)
summary(model)##
## Call:
## samediff(nsamesame = 71, ndiffsame = 29, nsamediff = 21, ndiffdiff = 79)
##
## Coefficients
## Estimate Std. Error Lower Upper P-value
## tau 1.4964 0.1408 1.2325 1.7840 < 2e-16 ***
## delta 2.6282 0.2502 2.1371 3.1229 1.89e-13 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Log Likelihood: -111.6108 AIC: 227.2217The signal detection model estimated a detectability index (d′) of 2.63 (95% CI: [2.14, 3.12]) and a decision criterion (c) of 1.50 (95% CI: [1.23, 1.78]).
The relatively high d′ value suggests that panelists were clearly able to discriminate between the sample pairs beyond chance level.
The criterion (tau) indicates a moderately conservative response bias, meaning that panelists tended to respond “same” unless they perceived a clear difference.
Both parameters were statistically significant (p < 0.001), confirming the reliability of the estimates.
Similar to the Same/Different test, the “A”–“Not A” test is a type of sensory discrimination method used to determine whether panelists can distinguish between two stimuli (e.g., treated vs. untreated product) when samples are presented individually. Each judge is asked to classify each sample as either “A” (the reference) or “Not A” (different from the reference).
The results are typically summarized in a 2×2 contingency table and can be analyzed using a chi-squared test. However, the sensR package provides a specific function tailored to this test, allowing for a more robust analysis based on the Thurstonian model.
library(sensR)
A_model <- AnotA(
x1 = 79,
n1 = 100,
x2 = 21,
n2 = 100
)
print(A_model)##
## Call: AnotA(x1 = 79, n1 = 100, x2 = 21, n2 = 100)
##
## Results for the A-Not A test:
##
## Estimate Std. Error Lower Upper P-value
## d-prime 1.612842 0.1998675 1.221109 2.004576 4.941986e-17Although the Same/Different and A–Not A tests can use identical response data organized into a 2×2 table, the results from the sensR functions (samediff() and AnotA()) differ due to their underlying models. The Same/Different model includes a decision uncertainty parameter (tau), accounting for an internal “gray zone” when panelists are unsure. This leads to a higher discriminability estimate (delta). In contrast, the A–Not A model assumes direct binary classification without uncertainty, estimating a lower d-prime (d′). Therefore, even with the same data, d′ and delta will not match.
When to use which test: Use the Same/Different test when samples can be presented in pairs and there’s a risk of confusion or memory effects—this model handles decision uncertainty explicitly. On the other hand, use the A–Not A test when samples must be presented individually (e.g., shelf-life studies or quality control) or when you want to mimic real-world conditions where the reference is remembered rather than directly compared.