Chapter 2: Response Bias

Response bias measures the tendency to give one response over another. Unlike d’, the measure of sensitivity, there are multiple measures of response bias. For all response bias statistics in MacMillan and Creelman:

positive bias is a tendency to say ‘no’ ( to be conservative)
negative bias is a tendency to say ‘yes’ (to be more liberal)

Measures of response bias

Suppose we have two observers, one with d’ of 1 (blue), the other with d’ of 2 (red). Both have $\lambda$ = 1.5 (distance from mean of noise distribution). Do they have similar response bias?

Relative to the blue observer, red observer says “yes” more often! So, we would want to have a response bias measure reflect that. Need to take both signal and noise distributions into account. The three measures described below do that it different ways.

Criterion Location (c) or $\lambda_{centre}$

Criterion location, $c$, is defined as the distance from the midpoint between the signal and noise distributions to the criterion.

\[ \begin{align} c = \lambda_{center} &= \lambda - \frac{1}{2}d' \\ &\text{replacing expressions for lambda and d' we can derive an expression in terms of H and FA: }\\ &=-z(FA) - \frac{1}{2}(z(H) - z(FA))\\ & = -z(FA)-\frac{1}{2}z(H) + \frac{1}{2} z(FA) = -\frac{1}{2}z(FA) - \frac{1}{2}z(H)\\ & = \frac{( z(H) + z(FA) )}{-2} \end{align} \]

Here is how c changes when d’ varies:

$c = 0$ when the two types of errors (false alarms and misses) are equally likely, $F = 1-H$, or $H = 1-F$, which is the formula for the negative diagonal.
When false alarms $<$ misses, then $H<(1-F)$ and c is positive
When false alarms $>$ misses, then $H>(1-F)$ and c is negative

Face recognition experiment example:

100 faces studied, then tested with 100 old and 100 new faces
Comparing normal and hypnotized participants.

##   Faces Normal.Yes Normal.No Hypno.Yes Hypno.No
## 1   Old         69        31        89       11
## 2   New         31        69        59       41

Calculating d’ and c:

dpNormal = qnorm((69/(69+31))) - qnorm((31/(31+69)))
dpHypno = qnorm((89/(89+11))) - qnorm((59/(59+41)))

cNormal = -0.5*(qnorm((69/(69+31))) + qnorm((31/(31+69))))
cHypno = -0.5*(qnorm((89/(89+11))) + qnorm((59/(59+41))))

(round(cbind(dpNormal, dpHypno),2))

##      dpNormal dpHypno
## [1,]     0.99       1

(round(cbind(cNormal, cHypno),2))

##      cNormal cHypno
## [1,]       0  -0.73

Hypnosis does not affect sensitivity, but it does lead to a more negative criterion c, i.e., higher false alarms compared to misses, i.e., more tendency to say ‘yes’

Now imagine that we train the observers to recognize faces and this results in an improvement in d’ from 1 to 2 (compare blue line in top and bottom figure) but no change in $c$ (length of c line is the same).But doesn’t the second case look like the criterion is less biased? In the figure above, the criterion is to the left of the mean of the noise distribution, but it moved to the right of the noise distribution in the bottom figure.

Relative criterion location (c’)

Defined as ratio of c and d’: distance from criterion to the midpoint between the two distributions in units of d’

\[\begin{align} c' &= \frac{c}{d'} = \\ &=-\frac{1}{2} (\frac{z(H)+z(F)}{z(H)-z(F)}) \end{align}\]

## [1] "pre-training where c = -0.73 and dprime = 1"

## [1] "cprime: "

## [1] -0.73

## [1] "post-training where c = -0.73 and dprime = 2"

## [1] "cprime: "

## [1] -0.365

Unlike c, c’ reflects a less extreme criterion post-training relative to pre-training.

Likelihood ratio

The likelihood ratio at the position of the decision criterion, $\lambda$, is another measure of response bias.

Likelihood ratio is the ratio of the probability density functions for signal and noise.

\[\begin{align} LR(x) = \Lambda(X) = \frac{P(X|\text{signal trial})}{P(X|\text{noise trial}}\\ \end{align}\]

Note that the likelihood ratio is equal to the slope of the ROC curve. See the illustration below for a geometrical demonstration:

When $\Delta FA > \Delta H$, the ROC slope is $<1$ and the likelihood ratio is $<1$
When $\Delta FA = \Delta H$, the ROC slope is $=1$ and the likelihood ratio is $=1$
When $\Delta FA > \Delta H$, the ROC slope is $=1$ and the likelihood ratio is $>1$

The likelihood ratio can be expressed in terms of d’ and c: $$ \[\begin{align} \text{Given that... } P(x|\text{signal}) &= \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}(x-d')^2} \text{ and } P(x|\text{noise}) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}(x)^2}\\ \text{Likelihood ratio at } \lambda = \frac{P(\lambda|\text{signal})}{P(\lambda|\text{noise})} = \beta(\lambda)&= e^{-\frac{1}{2}((\lambda-d')^2 - \lambda^2)}\\ \text{ln(}\beta(\lambda)) &= -\frac{1}{2}((\lambda-d')^2 - \lambda^2) = -\frac{1}{2}(\lambda^2-2\lambda d'+d'^2 - \lambda^2) = -\frac{1}{2}(d'^2-2\lambda d')\\ &=d'(-d'/2+\lambda) = d'(\lambda-d'/2) = d'\lambda_{center} = d'c\\ \text{ }\\ \text{When } \lambda = d'/2, \text{aka when, c=0, } \\ \text{Log of the Likelihood Ratio} = \text{ln(}\beta(\lambda)) &=d'(d'/2 - d'/2)= 0\\ \end{align}\]

$$ Log-likelihood ratio $ln(\beta)$ can be estimated using the equation with d’ and c.:

\[ ln(\beta) = d'c = -\frac{1}{2}(z(H) - z(FA)) * (z(H)+z(FA)) = -\frac{1}{2} ( z(H)^2 - z(FA)^2) \] or directly from density and distribution functions:

\[ ln(\beta) = \phi(z(H))/\phi(z(FA)) \]

Example: Hits = 0.915, False alarm: 0.265

H= 0.915
FA = 0.265

# method 1: 
-0.5*(qnorm(H)^2-qnorm(FA)^2)

## [1] -0.7442759

# method 2: 
log(dnorm(qnorm(H)) / dnorm(qnorm(FA)))

## [1] -0.7442759

Isobias curves

Isobias curves trace curves connecting H and FA rates of constant response bias for each of the different response bias measures.

Which bias measure to use?

\[c = \lambda_{centre} = \lambda-d'/2 = \text{distance from midpoint between distribution means}\] \[ ln(\beta) = cd' \]

\[ c' = \frac{c}{d'}\]

They are all equivalently ‘correct’, so their use depends on the context.
When d’ does not change, all three give the same results.
When d’ varies, all three change in different ways.
Some benefits of c: only c is statistically independent of sensitivity (d’), is defined when d’ = 0, and has the same value for two (F, HA) pairs symmetrical about the chance line.

Which ones are found empirically?

See, Warm, Dember and Howe (1997) found iso-bias curves consistent with $c$ for a vigilance task (and so did Hautus and Collins, 2003, using an auditory task), while Stretch and Wixted (1998) found curves consistent with $\beta$ in a recognition memory task.

Where should an observer place their criterion?

It depends on what the goal is.

Maximize proportion correct? :

Ideal location of criterion is equal to the reciprical of the prior odds (prior odds = p / (1-p))
$\beta(\lambda_{optimal})=\frac{1-p}{p}, \text{and } p = \text{probability of signal}$
If probability of the signal, $p$, is large, $\frac{1-p}{p}$ is small, so we are more likely to respond “Yes”. If probability of the signal, $p$, is small, $\frac{1-p}{p}$ is large, so we are less likely to respond “Yes”

Maximize expected value:

$LR(\lambda_{optimal}) = \frac{Reward(CR) - Reward(FA)}{R(H) - R(M)} * \frac{1-p}{p}$
E.g., if you get 10 cents for a hit or correct rejection, lose 1 cent for a miss, and 1$ for a false alarm, assuming equal probability, the observer should say “Yes” only when the Likelihood Ratio is greater or equal to:

    (0.1 - -1)/(0.1 - (-0.01))

## [1] 10

That is, the probability of data given the signal should be 10x greater than the data given the noise.
People rarely act in accordance with these optimal criteria in extreme scenarios
The rewards may be implicit.. e.g., X-ray reading,where false alarms are less costly than misses.

Neyman-Pearson objective: obtain as high a hit rate (H) while holding false-alarm rate at a set $\alpha$ level.

In null-hypothesis statistics, Type I error = false alarm, typically set at 0.05.. although some recommend 0.005, etc. Then likelihood ratios of data | Ha is compared to data | Ho, and rejected if it exceeds the criterion.

Chapter 3: Empirical ROCs

Ways to obtain empirical ROCs

Confidence rating experiment
Manipulate presentation probability (and inform participants)
Monetary rewards
Verbal instructions (‘be lax’ or ‘be strict’, or ‘words that you ’remember’ in the study list vs ‘know’ the word was in the study list.)

Rating experiment

A one-interval design comprising:

Two classes of stimuli
Multiple response choices ranging from full certainty in one response to indifference between response choicec, to full certainty in the second response.
- Rating scale 1 - N, where 1 = ‘fully certain it is noise’ to N = ‘fully certain it is signal’.
- Signal/Noise binary response, followed by a confidence rating (How confident are you in your answer?: e.g., 1-3). This is then converted into No-“3” No-“2” No-“1” “Yes”-1 Yes-“2” Yes-“3”

Example Old/New odor experiment based on Rabin and Cain, 1984.

## 
## 
## After 10 min delay

##     Old.3 Old.2 Old.1 New.1 New.2 New.3    
## Old   112   112    72    53    22     4 375
## New     7    38    50   117   101    62 375

## 
## 
## After 7 days delay

##     Old.3 Old.2 Old.1 New.1 New.2 New.3    
## Old    49    94    75    60    75    22 375
## New     8    37    45    60   113   112 375

Rating experiment model:

When given 6 response choices, the observer places 5 decision criteria separating each response choice.

Rating experiment analysis steps:

Convert the cell counts into probabilities. p(response | total trials for this stimulus class)
Calculate cumulative probabilities for all cells, from left to right.
Convert to z-scores
Calculate d’ for each response boundary.

## 
## 
## Step 1: 10 min delay proportions

##     Old.3 Old.2 Old.1 New.1 New.2 New.3
## Old  0.30   0.3  0.19  0.14  0.06  0.01
## New  0.02   0.1  0.13  0.31  0.27  0.17

## 
## 
## Step 2: 10 min delay cumulative sum of probabilities

##     Old.3 Old.2 Old.1 New.1 New.2 New.3
## Old  0.30  0.60  0.79  0.93  0.99     1
## New  0.02  0.12  0.25  0.56  0.83     1

## 
## 
## Step 3: 10 min delay z-transform

##      Old.3  Old.2  Old.1 New.1 New.2 New.3
## Old -0.524  0.253  0.806 1.476 2.326   Inf
## New -2.054 -1.175 -0.674 0.151 0.954   Inf

## 
## 
## Step 4: compute d-prime: z(Old) - z(New).. i.e., z(H) - z(FA)

## Old.3 Old.2 Old.1 New.1 New.2 New.3 
## 1.530 1.428 1.480 1.325 1.372   NaN

## 
## 
## Step 4: similarly can compute bias, c:-0.5*(z(Old) + z(New))

##   Old.3   Old.2   Old.1   New.1   New.2   New.3 
##  1.2890  0.4610 -0.0660 -0.8135 -1.6400    -Inf

## 
## 
## 
## 10 min delay results

##                     Old.3  Old.2  Old.1   New.1  New.2 New.3
## Old                -0.524  0.253  0.806  1.4760  2.326   Inf
## New                -2.054 -1.175 -0.674  0.1510  0.954   Inf
## tenMinDelay_dPrime  1.530  1.428  1.480  1.3250  1.372   NaN
## tenMinDelay_c       1.289  0.461 -0.066 -0.8135 -1.640  -Inf

## [1] ""

## 
## 
##  Do the same for 7 days

## 
## 
##  7 day results

##                       Old.3  Old.2  Old.1  New.1   New.2 New.3
## Old                  -1.126 -0.305  0.202  0.643  1.5550   Inf
## New                  -2.054 -1.175 -0.706 -0.253  0.5240   Inf
## sevenDayDelay_dPrime  0.928  0.870  0.908  0.896  1.0310   NaN
## sevenDayDelay_c       1.590  0.740  0.252 -0.195 -1.0395  -Inf

These data can be plotted in an ROC curve and a best-fit line can be found. Here, the slope is very close to 1, so we can fix the slope parameter to 1, fit the curve and extract d’ from the x or the y intercept.

## 10 min delay:

## (Intercept) 
##       1.427

## 7 days delay:

## (Intercept) 
##      0.9266

What if slopes are clearly not = 1??

Unequal variance example, i.e. z(ROC) slope $\neq$ 1

Data from a memory experiment with low and high frequency words. The judgement is about whether the words are Old or New, and how certain the observer is about their answer.

## Low frequency

##     Old.Sure Old.Maybe Uncertain.3 New.Maybe NewSure
## Old       61        15          15         5       4
## New        2         8          37        23      30

## High frequency

##     Old.Sure Old.Maybe Uncertain.3 New.Maybe NewSure
## Old       37        25          18        11       9
## New        4        18          28        21      29

Repeat the same steps to get estimates of H and FA.

## 
## Step 1: proportions

##     Old.Sure Old.Maybe Uncertain.3 New.Maybe NewSure
## Old     0.61      0.15        0.15      0.05    0.04
## New     0.02      0.08        0.37      0.23    0.30

## 
## Step 2: cumulative sum of probabilities

##     Old.Sure Old.Maybe Uncertain.3 New.Maybe NewSure
## Old     0.61      0.76        0.91      0.96       1
## New     0.02      0.10        0.47      0.70       1

## 
## Step 3: z-transform,d` and c calculation

##     Old.Sure Old.Maybe Uncertain.3 New.Maybe NewSure
## Old    0.279     0.706       1.341     1.751     Inf
## New   -2.054    -1.282      -0.075     0.524     Inf

##                Old.Sure Old.Maybe Uncertain.3 New.Maybe NewSure
## Old              0.2790     0.706       1.341    1.7510     Inf
## New             -2.0540    -1.282      -0.075    0.5240     Inf
## lowfreq.dPrime   2.3330     1.988       1.416    1.2270     NaN
## lowfreq.c        0.8875     0.288      -0.633   -1.1375    -Inf

Do the same for high frequency:

##                 Old.Sure Old.Maybe Uncertain.3 New.Maybe NewSure
## Old              -0.3320    0.3050       0.842     1.341     Inf
## New              -1.7510   -0.7720       0.000     0.553     Inf
## highfreq.dPrime   1.4190    1.0770       0.842     0.788     NaN
## highfreq.c        1.0415    0.2335      -0.421    -0.947    -Inf

Plotting the data reveals that the slopes are not equal, so the x- and y- intercepts are different. d’ is no longer a good measure.

Note about finding best-fit line: since we need to minimizeg both the x- and the y-distances, linear regression is incorrect.
Can use ordinal regression with the ‘ordinal’ package in R. See Knoblauch & Maloney Modeling Psychophysical Data in R, p 91.

Alternative indices to d’ when unequal variances:

$d'_1 = \text{horizontal distance from chance line to ROC when } z(H) = 0$
- $z(H) = 0$ at the mean of the signal distribution (S2)
- It is calculated in units of S1 (noise) distribution
$d'_2 = \text{vertical distance from chance line to ROC when } z(FA) = 0$
- $z(FA) = 0$ at the mean of the noise distribution (S1)
- It is calculated in units of S2 (signal) distribution
Slope of ROC, $s = d'_2/d'_1$
$\text{Equation for the unequal variance ROC curve: } z(H) = d'_2+\frac{d'_1}{d'_2}z(F)$
$d_a = \sqrt{2}*D_{YN}$
- equivalent to d’ when slope = 1
- difference between means in units of RMS: $\sqrt{\text{mean (}\sigma_n^2,\sigma_s^2)}$
- From an ROC line: $d_a = d'_2 * \sqrt{\frac{2}{1+s^2}}$, where $d'_2$ is the vertical intercept, $d'_1$ is the horizontal intercept, and $s = \frac{d'_2}{d'_1}$
- From a point on ROC: $d_a = \sqrt{\frac{2}{1+s^2}}(z(H) - sz(F))$
$d_e \text{ based on arithmetic average of } \sigma_n, \sigma_s$
- $d_e = \sqrt{\frac{2}{1+s}}(z(H) - sz(F))$
$A_z = \Phi(d_{YN}) = \Phi(d_a/\sqrt{2})$, Area under the ROC curve in proportion coordinates. Non-parametric measure of sensitivity.

Alternative indices for response bias with unequal variances

Similar adjustments have to be made to replace c with an adjusted measure using the slope of the ROC (s)

$c_1 = \frac{-1}{(1+s)} ( z(H) + z(F)$
$c_2 = \frac{-s}{(1+s)} ( z(H) + z(F)$
$c_a = \frac{-\sqrt{2}s}{\sqrt(1+s^2)(1+s)} ( z(H) + z(F)$
$c_e = \frac{-2s}{(1+s)^2} ( z(H) + z(F)$
All measures are the same up to a multiplicative constant; when s = 1, they all = c
Likelihood ratio obtained from heights of density functions for H and FA. It is no longer equal to d’c. When variances are unequal, the two curves may intersect at 2 points. Using one single decision criterion line can lead to less than chance performance. The optimal decision involves two decision lines (bottom panel, below).

Type-2 ROC

Confidence judgements about how correct or incorrect was our response.
Clarke, Birdsall, Tanner 1959 assume two-stage decision making: unbiased criterion for ‘yes/no’ response, then confidence judgements are high for extreme observations for signal or noise.
Note: this is not always observed. Sometimes observers are highly confident in their wrong responses. Also, type 1 sensitivity also influences type 2 responses. See Maniscalo & Lau, Consciousness and Cognition, 2012 and Fleming and Lau, Frontiers in Human Neuroscience, 2014 for more.

Chapter 4: Threshold Models and Choice Theory

Single High-Threshold Theory

‘Yes’ responses to noise stimuli are random guesses, not from ‘actually thinking one has seen the stimulus’.

Signal stimulus (S2) leads to state D2 (‘detect’) with probability q (sensitivity) and to state D1 (‘not detect’) with probability 1-q. From there, ‘yes’ responses occur with guess probability ‘u’.
Noise stimulus (S1) always leads to state D1 (‘not detect’)

\[ \begin{align} H &= P(\text{"yes"}|S_2 \text{ or signal}) = q + u(1-q) \\ FA &= P(\text{"yes"}|S_1 \text{ or noise}) = u \\ &\text{replacing u with FA in the first line, we can get an expression for sensitivity, q}\\ q &= H - FA+FAq\\ q-FAq &= H-FA\\ q &=(H-FA)/(1-FA) \end{align} \]

Equation for the ROC curve: straight line, with (1-q) slope and q intercept. These are ‘non-regular’, because allow H to be >0 when FA = 0.

\[ \begin{align} H &=(1-q)FA + q \\ \end{align} \]

The figure below shows the decision space assumed by High threshold theory.

Threshold is placed arbitrarily at 1. S1 is the noise distribution and it never extends beyong the threshold (below 1). S2 is the signal and extends to the left and right of the threshold.
The decision criterion is placed always below the threshold.
$u$ denotes the proportion of noise trials on which the observer says “yes”, leading to FAs.
Because the criterion is placed below the threshold, then Hit rate is equal to u + q, where q represents the space of signal distribution that the observer can truly detect (above 1, which is the threshold).
The high-threshold theory uses FA to estimate this “u” rate, and then adjusts the hit rate by this ‘guessing’ parameter to find true sensitivity, q.
note that q is a linear combination of H and F, scaled to range from 0 to 1 instead of 0 to 1-F.
$q = (H-F)/(1-F)$
u = FA rate is also the response bias statistic.

Low Threshold theory (Luce, 1963)

The main difference with the high threshold theory described above is that now, the threshold is “low” because it can be crossed by noise alone, i.e., the noise distribution extends beyond the threshold.

Signal lead to D2 (‘detect’) with probability $q_2$ and D1 (‘not detect’) with probability $1-q_2$
Noise lead to D1 (‘not detect’) with probability $q_1$ and D2 (‘detect’) with probability $1-q_1$
Sensitivity reflected by 2 measures: true hit rate and true false-alarm rate (adjusted for guessing)

Observers can use one of two different strategies:

upper-limb: some proportion u of ‘no detect’ states is given response “Yes”; ‘lax’ strategy
- $H = q_2 + u(1-q_2)$
- $FA = q_1 + u(1-q_1)$
lower-limb: only a proportion t ‘detect’ states is given response “Yes”; ‘conservative’ strategy
$H = tq_2$
$FA = tq_1$

Here is the decision space for each strategy

Here is the predicted ROC curve:

It can be very difficult to obtain enough data to distinguish this prediction from that of signal detection theory. We will discuss this theory more in Kellen et al., 2016.

Double High-Threshold Theory

This theory underlies the use of proportion correct in yes/no tasks.

\[ p_{correct} = p_{\text{signal or }S_2}H + p_{\text{noise or }S_1}(1-F) \\ = p_{S_1}+ p_{S_2}H - p_{S_1}F \\ = \text{constant + weighted difference of H and F} \]

Sensitivity $q = 2p_{correct} - 1$

The ROC curve under this theory is as follows:

Yonelinas (1997) found ROC curves consistent with double threshold theory in an associative recognition experiment:

Where these two test-pairs of words presented together in the study phase? Yes/No
data show two different sensitivities (different intercepts)
$q_1$ (recollecting that one item from pair was shown with another item)
$q_2$ (recollecting pair as old)
Argued that this suggests that unlike ‘familiarity’, which is a continuous variable, ‘recollection’ is all/nothing.

Bias measures for double-high-threshold theory:

Yes rate:

$p(\text{'yes'}| \text{signal or noise trial}) = \frac{(H + F)}{2}$ when equal probability of signal and noise
Criterion in the double high-threshold model:$k = \frac{1}{2} ( 1 - (H+F))$
Therefore, the yes rate is a linear transformation of the criterion location.

Error ratio:

The ratio of false alarms and misses is another measure of bias.
Can linearly transform k into k’: $k' = 1/ (1 + \frac{F}{1-H}) = 1-v$
- only depends on errors (F and misses (i.e., 1-H))
- similar to c’, relative criterion location

The predicted shapes of iso-bias curves for the yes rate and error ratio look similar to those of c and c’..

Choice Theory

Stems from the following ‘choice axiom’: > Odds of choosing one stimulus over another are not affected by other possible stimuli. Luce (1959)

This theory proposes another sensitivity measure, which is a nonlinear combination of H and FA:

\[ \alpha = \sqrt{\frac{H(1-F)}{(1-H)F}} \]

To express is as a difference of transformations of H and FA, we take logs on both sides, and we get:

\[ ln(\alpha) = \frac{1}{2} ln(\frac{H}{1-H})- \frac{1}{2}ln(\frac{F}{1-F}) \] note: $ln(\alpha)$ is also called $d_L$ in memory research.

Instead of the z-transformation, Choice Theory applies the log-odds transformation to H and F.

Calculating \[lg(\alpha)\] and d’ for the same pair of H and FA, we obtain:

F = 0.4
H = 0.8
(logAlpha = round(0.5*log(H/(1-H)) - 0.5*log(F/(1-F)),3))

## [1] 0.896

(dprime = qnorm(H)-qnorm(F))

## [1] 1.094968

When H vs FA are plotted in log-odds coordinates,the resulting ROC curve is a straight line.

In Linear coordinates, SDT and Choice Theory have very similar ROCs, except that Choice Theory cannot have asymmetric curves (i.e., only similar to the equal variance assumption of SDT ).

Bias measure

Criterion location: $ln(b) = -\frac{1}{2} ln(\frac{H}{1-H}) + \frac{1}{2}ln(\frac{F}{1-F})$
Relative criterion location: $b' = ln(b)/2*ln(\alpha)$
Likelihood ratio: $\beta_L = \frac{H(1-H)}{F(1-F)}$

Decision space for Choice Theory

To find a proportion given a log-odds value (logit) x, rearrange the equation for log-odds. \[ x = ln(p/(1-p))\\ e^x = p/(1-p)\\ e^x-pe^x = p\\ p = 1/(1-e^x) \] Using this knowledge, we can derive equations for H, F, and proportion correct (assuming unbiased observer and equal proportions of signal and noise) \[ H = \alpha/(\alpha + \beta)\\ F = 1/(1+\alpha\beta)\\ p_{correct} = \alpha/(\alpha+1)\\ \]

Non-parametric measures:

If only have 1 point on the ROC, various ‘nonparametric’ measures such as the area under the ROC curve (drawn with straight lines), is a combination of the above mentioned theories and is thus not better.
If obtain multiple points on the ROC, area under the ROC is a non-parametric measure of sensitivity.

Signal Detection Theory, Lecture 2

Eugenie Roudaia