Session 1: Introduction to Decision Theory

Basic Concepts

Decision theory is a formal theory of decision making under uncertainty.

A decision problem consists of:

• Possible actions: \( \{a\}_{a ∈ A} \)
• States of the world (usually uncertain): \( \{s\}_{s ∈ S} \)
• Possible consequences: \( \{c\}_{c ∈ C} \) (depends on state and action)

We work to know What is the best action?

• Answer (according to decision theory): – Measure “goodness” of consequences with a utility function u© – Measure likelihood of states with probability distribution p(s) – Best action with respect to model maximizes expected utility:

\[ a^*=arg_a max\{E[u(c) | a]\} \]

Illustrative Example

1.- Decision problem: Should patient be treated for disease?

• We suspect she may have disease but do not know
• Without treatment the disease will lead to long illness
• Treatment has unpleasant side effects

2.- Decision model:

• Actions: \( a_T \) (treat) and \( a_N \) (don’t treat)
• States of world: \( s_D \) (disease now) and \( s_W \) (well now)

• Consequences:

  • \( c_{WN} \) (well shortly, no side effects)
  • \( c_{WS} \) (well shortly, side effects)

  • \( c_{DN} \) (disease for long time, no side effects)

    	Disease	Well now
    Treat	\( c_{WS} \)	\( c_{WS} \)
    No Treat	\( c_{DN} \)	\( c_{WN} \)

• Probabilities and Utilities:

  • p(\( s_D \)) = 0.3
  • u(\( c_{WN} \)) = 100 (no treatment - no side effects -patient was healthy)
  • u(\( c_{WS} \)) = 90 (treatment - side effects - patient gets well)

  • u(\( c_{DN} \)) = 0 (no treatment - no side effects, patient had sickness, which he suffers long-time)

    	Disease	Well now
    Treat	90	90
    No Treat	0	100

3.- Expected utility:

• Treat: p(\( s_D \)) × u(\( c_{WS} \)) + (1-p(\( s_D \))) × u(\( c_{WS} \))
• Don't treat: p(\( s_D \)) x u(\( c_{DN} \)) + (1-p(\( s_D \))) x

• Best action is \( a_T \) (treat patient)

  	Disease	Well now	Expected Utility
  Treat	0.3 x 90	0.7 x 90	90
  No Treat	0.3 X 0	0.7 x 100	70

4.- Sensitivity Analysis: Optimal Decision as Function of Sickness Probability

• Expected utility of the two actions:

  • p = p(\( s_D \)) = 0.3
  • \( E[u(c) | a_T] \)= 90
  • \( E[u(c) | a_N] \)= 0p + 100(1-p) = 100(1 – p)

• Typically we are uncertain about the value of p

  • The chart shows how the optimal decision changes as we vary p

ExpU_T = 90
FUN_ExpU_T <- function(p) (ExpU_T)
FUN_ExpU_N <- function(p) 100 * (1 - p)
DIFF_FUN <- function(x) FUN_ExpU_T(x) - FUN_ExpU_N(x)
p_vector = seq(from = 0, to = 1, length.out = 100)
plot(p_vector, FUN_ExpU_N(p_vector), type = "l", col = "red", axes = F)
axis(1, at = seq(0, 1, length.out = 11), labels = T)
axis(2)
abline(h = FUN_ExpU_T(p_vector), col = "blue")
p_intersection = uniroot(DIFF_FUN, c(0, 1))$root
abline(v = p_intersection, col = "darkgreen", lty = 2)
box()

The chart shows we should treat if p > 0.1,and don’t treat if p < 0.1

Extending the Disease Example: Gathering Information

We may be able to perform a test before deciding whether to treat the patient:

• Test has two outcomes: \( t_P \) (positive) and \( t_N \) (negative)
• Quality of test is characterized by two numbers:
  • Sensitivity: Probability that test is positive if patient has disease.
  • Specificity: Probability that test is negative if patient does not have disease

We will assume:

• Sensitivity: P(\( t_P \) | \( s_D \)) = 0.95
• Specificity: P(\( t_N \) | \( s_W \)) = 0.85

How does the model change if test results are available?

• Take test, observe outcome t
• Revise prior beliefs P(\( s_D \)) to obtain posterior beliefs P(\( s_D \)|t)
• Re-compute optimal decision using P(\( s_D \)|t)

Bayes Rule: The Law of Belief Dynamics

Objective: use evidence E to update beliefs about a hypothesis H

• H: patient has (does not have) disease
• E: evidence from test Procedure: apply Bayes Rule (standard form): \begin{equation} P(H|E)=\frac{P(E \cap H)}{P(E)}=\frac{P(E|H)P(H)}{P(E)} \end{equation}

Bayes as Odds: \begin{equation} \frac{P(H_1|E)}{P(H_2|E)}=\frac{\frac{P(E|H_1)P(H_1)}{P(E)}}{\frac{P(E|H_2)P(H_2)}{P(E)}}=\frac{P(E|H_1)P(H_1)}{P(E|H_2)P(H_2)}, ~~if ~P(E)>0, P(H2)>0 \end{equation}

Disease Example with Test

• Review of Problem Ingredients:
  • P(\( s_D \)) = 0.3 (prior probability of disease)
  • P(\( t_P | s_D \)) = 0.95 (sensitivity of test)
  • P(\( t_N | s_W \)) = 0.85 (specificity of test)
  • Utilities
  • \( u(c_{WN}) = 100, u(c_{WS}) = 90; u(c_{DN}) = 0 \)
• Posterior probability of disease given test:

– If test is negative P(sD | tN) = (0.3 x 0.05)/(0.3 x 0.05 + 0.7 x 0.85) = 0.025 – If test is positive P(sD | tP) = (0.3 x 0.95)/(0.3 x 0.95 + 0.7 x 0.15) = 0.731 • Expected utility of treating patient is independent of test – EU(aT) = EU(aT | tN) = EU(aT | tP) = 90 • Expected utility of not treating patient depends on test – If test is negative EU(aN | tN) = 0.975 × 100 = 97.5 – If test is positive EU(aN | tP) = 0.269 × 100 = 26.9 – (Recall if no test EU(aN) = 0.7 × 100 = 70 ) • Optimal choice is strategy aF (FollowTest): – Treat if test is positive; don’t treat if test is negative