When dealing with jointly distributed random variables, we are sometimes interested in the conditional probability distribution function of one random variable given the that the other random variable takes on a particular value.
Let \(X\) and \(Y\) be a pair of jointly distributed random variables. The conditional probability function of the random variable \(Y\), given that the random variable \(X\) takes the value \(x\), is \[p(y|x) = \frac{p(x,y)}{p(x)} \text{ for all values of } y\] Similarly, the conditional probability function of the random variable \(X\), given that the random variable \(Y\) takes the value \(y\) is \[p(x|y) = \frac{p(x,y)}{p(y)} \text{ for all values of } x\]
Suppose \(X\) and \(Y\) have the following joint probability distribution
| \(x=-3\) | \(x=3\) | |
|---|---|---|
| \(y=1\) | \(0.2\) | \(0.2\) |
| \(y=8\) | \(0.3\) | \(0.3\) |
The conditional distribution of \(Y\) given that \(X = 3\) has two components. The first component is when \(Y=1\)
\[p(1|x=3) = \frac{p(1,3)}{p(3)}=\frac{0.2}{(0.2+0.3)}=0.4\] and the second component is when \(Y=8\) \[p(8|x=3) = \frac{p(8,3)}{p(3)}=\frac{0.3}{0.2+0.3}=0.6\] Putting it all together, we get that the conditional distribution of \(Y\) given \(X=3\) is
| \(y|x=3\) | \(1\) | \(8\) |
|---|---|---|
| \(p(y|x=3)\) | \(0.4\) | \(0.6\) |
Example 1:
Consider the joint probability distribution
| \(x=1\) | \(x=2\) | |
|---|---|---|
| \(y=0\) | \(0.30\) | \(0.20\) |
| \(y=1\) | \(0.25\) | \(0.25\) |
Find the conditional distribution of \(X\) given that \(Y=0\).
Click For Answer\[p(x=1|y=0) = \frac{P(x=1,y=0)}{P(Y=0)}=\frac{.3}{.3+.2}=0.6\] \[p(x=2|y=0) = \frac{P(x=2,y=0)}{P(Y=0)}=\frac{.2}{.3+.2}=0.4\] Which gives us
| \(x\) | \(1\) | \(2\) |
|---|---|---|
| \(p(x|y=0)\) | \(0.6\) | \(0.4\) |
\[\mu_{y|x} = E(Y|X=x) = \sum_y yp(y|x)=\sum_y y \frac{p(x,y)}{p(x)}\]
Continuing with our example from above, we can use the conditional distribution of \(Y\) given \(X = 3\) to find the conditional mean of \(Y\) given \(X=3\)
\[\mu_{y|x} = 1(0.40) + 8(0.60) = 5.2\]
Example 2:
Consider the joint probability distribution
| \(x=1\) | \(x=2\) | |
|---|---|---|
| \(y=0\) | \(0.30\) | \(0.20\) |
| \(y=1\) | \(0.25\) | \(0.25\) |
Find the conditional mean of \(X\) given that \(Y=0\).
Click For AnswerThe conditional distribution of \(X\) given that \(Y=0\) is (from Example 1)
| \(x\) | \(1\) | \(2\) |
|---|---|---|
| \(p(x|y=0)\) | \(0.6\) | \(0.4\) |
So the mean is \[E(X|Y=0) = 1(.6) + 2(.4) = 1.4\]
\[\sigma_{y|x}^2 = E[(Y-\mu_{y|x})^2|X=x] = \sum_y (y-\mu_{y|x})^2 p(y|x)=\sum_y (y-\mu_{y|x})^2 \frac{p(x,y)}{p(x)}\]
Continuing with our example from above, we can use the conditional distribution of \(Y\) given \(X = 3\) to find the conditional variance of \(Y\) given that \(X=3\)
\[\sigma_{y|x}^2 = (1-5.2)^2 \cdot 0.40 + (8-5.2)^2 \cdot 0.60 = 11.76\]
Example 3:
Consider the joint probability distribution
| \(x=1\) | \(x=2\) | |
|---|---|---|
| \(y=0\) | \(0.30\) | \(0.20\) |
| \(y=1\) | \(0.25\) | \(0.25\) |
Find the conditional variance of \(X\) given that \(Y=0\).
Click For AnswerFrom Examples 1 and 2, we have the conditional distribution and the conditional mean. Using these we can find the conditional variance:
\[\sigma_{x|y}^2 = (1-1.4)^2 \cdot 0.6 + (2-1.4)^2 \cdot 0.4 = 0.24\]
We can just take the square root of the conditional variance to get the conditional standard deviation
\[\sigma_{y|x} = \sqrt{\sigma_{y|x}^2}\]
Example 4:
Consider the joint probability distribution
| \(x=1\) | \(x=2\) | |
|---|---|---|
| \(y=0\) | \(0.30\) | \(0.20\) |
| \(y=1\) | \(0.25\) | \(0.25\) |
Find the conditional standard deviation of \(X\) given that \(Y=0\).
Click For AnswerFrom Example 3, we have that the conditional variance is \(0.28\). Taking the square root gives our result \[\sigma_{x|y} = \sqrt{0.24}\]
The same concepts hold for continuous random variables with integrals replacing the sums. More details on this case are beyond the scope of the course.