5.4 - independence vs dependence
Independence vs dependence: conceptually
- Jointly distributed random variables \(X\) and \(Y\) are independent if \(Y\) does not inform the distribution of \(X\) (or vice versa).
- Jointly distributed random variables \(X\) and \(Y\) are dependent if \(Y\) informs the distribution of \(X\) (or vice versa).
Independence corollary
If \(X\) and \(Y\) are discrete random variables with joint pmf \(p(x,y)\) and marginal pmfs \(p_X(x)\) and \(p_Y(y)\), respectively, then \(X \perp\!\!\!\perp Y\) if and only if either condition is satisfied:
- \(p_{X|Y}(x|y) = p_X(x)\)
- \(p_{Y|X}(y|x) = p_Y(y)\)
Why?
\[\scriptsize p_{X|Y}(x|y) = \frac{p(x,y)}{p_{Y}(y)} = \frac{p_X(x)p_Y(y)}{p_Y(y)} = p_X(x)\]
If \(X\) and \(Y\) are continuous random variables with joint pdf \(f(x,y)\) and marginal pdfs \(f_X(x)\) and \(f_Y(y)\), respectively, then \(X \perp\!\!\!\perp Y\) if and only if either condition is satisfied:
- \(f_{X|Y}(x|y) = f_X(x)\)
- \(f_{Y|X}(y|x) = f_Y(y)\)
Why?
\[\scriptsize f_{X|Y}(x|y) = \frac{f(x,y)}{f_{Y}(y)} = \frac{f_X(x)f_Y(y)}{f_Y(y)} = f_X(x)\]
Example: joint discrete
Suppose \(X\) and \(Y\) are jointly discrete with joint pmf:
\[p(x,y) = \left\{\begin{array}
{ll}
\frac{1}{30}(x+y) & x=0,1,2; y=0,1,2,3 \\
0 & otherwise\\
\end{array}\right.\]
Are \(X\) and \(Y\) independent?
\(p(x,y)\):
| 0 |
0/30 |
1/30 |
2/30 |
| 1 |
1/30 |
2/30 |
3/30 |
| 2 |
2/30 |
3/30 |
4/30 |
| 3 |
3/30 |
4/30 |
5/30 |
\(p_X(x)\):
| \(P(X=x)\) |
6/30 |
10/30 |
14/30 |
\(p_Y(y)\):
| 0 |
3/30 |
| 1 |
6/30 |
| 2 |
9/30 |
| 3 |
12/30 |
\[p(0,0) = 0 \ne p_X(0)p_Y(0) = 18/30^2 \implies X \not\!\perp\!\!\!\perp Y\]
Example: binomial/Poisson
- If \(X \sim POI(\lambda)\) and \(Y|X=x \sim BIN(x,p)\), are \(X\) and \(Y\) independent?
- Previously: proved \(Y\sim POI(\lambda p) \implies p_Y(y) = \frac{e^{-\lambda p}(\lambda p)^{y}}{y!}, y=0,1,2,...\)
- \(p_{Y|X}(y|x) = {x\choose y}p^y(1-p)^{x-y}, y = 0,1,2,...,x\)
- \(p_{Y|X}(y|x) \ne p_Y(y)\implies X \not\!\perp\!\!\!\perp Y\)
Necessary condition for independence
- If \(X\) and \(Y\) are jointly distributed random variables (discrete, continuous, or mix), then if \(X\) and \(Y\) are independent, the support is rectangular.
- I.e., for \(b>a\) and \(d>c\), the support is:
\[a < X < b\] \[c < Y < d\]
where the endpoints could be \(\pm \infty\).
- This yields a way to prove non-independence by contrapositive: if the support is not rectangular, then \(X\) and \(Y\) are not independent.
- Note a rectangular support is a necessary, but not sufficient condition for independence: we can have jointly distributed random variables with rectangular supports that are not independent. (This is an if, not an iff statement.)
Binomial/Poisson example revisited
If \(X \sim POI(\lambda)\) and \(Y|X=x \sim BIN(x,p)\), are \(X\) and \(Y\) independent?
Joint support, \(0 \leq Y \leq X < \infty\):
![Joint discrete support]()
Non-rectangular support \(\implies\) \(X\not \perp\!\!\!\!\!\perp Y\)
Joint discrete example revisited
Suppose \(X\) and \(Y\) are jointly discrete with joint pmf:
\[p(x,y) = \left\{\begin{array}
{ll}
\frac{1}{30}(x+y) & x=0,1,2; y=0,1,2,3 \\
0 & otherwise\\
\end{array}\right.\]
\(p(x,y)\):
| 0 |
0/30 |
1/30 |
2/30 |
| 1 |
1/30 |
2/30 |
3/30 |
| 2 |
2/30 |
3/30 |
4/30 |
| 3 |
3/30 |
4/30 |
5/30 |
Non-rectangular support \(\implies\) \(X\not \perp\!\!\!\!\!\perp Y\)
Example
- Let \(X\) and \(Y\) be bivariate discrete random variables with joint pmf given by:
| x = 0 |
1/6 |
1/6 |
1/6 |
| x = 1 |
1/6 |
1/6 |
1/6 |
- Suppose as well that marginally, \(X\sim BERN(0.5)\) and \(Y\sim UNIF\{1,2,3\}\)
- Are \(X\) and \(Y\) independent?
Rectangular support \(\implies\) \(X\) and \(Y\) could be independent!
- \(p_X(x) = \frac{1}{2}, x\in \{0,1\}\)
- \(p_Y(y) = \frac{1}{3}, y \in \{1,2,3\}\)
- \(p(x,y) = p_X(x) p_Y(y)\ \ \forall\ \ x\in \{0,1\}, y \in \{1,2,3\}\)
- \(X \perp\!\!\!\!\!\!\perp Y\)
Joint continuous example 1
Suppose \(X\) and \(Y\) are jointly continuous with joint pdf:
\[f(x,y) = \begin{cases}3x & 0 < y < x < 1 \\
0 & otherwise \end{cases}\]
Are \(X\) and \(Y\) independent?
Joint support:
Non-rectangular support \(\implies\) \(X\not \perp\!\!\!\!\!\perp Y\)
Joint continuous example 2
Suppose \(X\) and \(Y\) are jointly continuous with joint pdf:
\[f(x,y) = \begin{cases}\frac{x+y}{8} & 0 \leq x \leq 2, 0\leq y\leq 2\\
0 & otherwise \end{cases}\]
Are \(X\) and \(Y\) independent?
Joint support:
Rectangular support \(\implies\) \(X\) and \(Y\) could be independent!
- \(\scriptsize f_X(x) = \int_0^2 \frac{x+y}{8}\ dy = \left(\frac{xy}{8}+\frac{y^2}{16}\right)\big|_{y=0}^2 = \frac{x+1}{4}, 0 < x < 2\)
- \(\scriptsize f_Y(y) = \int_0^2 \frac{x+y}{8}\ dx = \left(\frac{x^2}{16}+\frac{xy}{8}\right)\big|_{x=0}^2 = \frac{1+y}{4}, 0 < y < 2\)
- \(\scriptsize f_X(x)f_Y(y) = \frac{x+1}{4}\frac{1+y}{4} \ne \frac{x+y}{8} = f(x,y)\)
- \(X\) and \(Y\) are not independent
Joint continuous example 3
Are \(X\) and \(Y\) independent if their jointly continuous CDF is:
\[\scriptsize F(x,y) = \begin{cases} 0 & x < 0\ OR\ y <0 \\
x^2 y^2 & 0 \leq x \leq 1, 0 \leq y \leq 1\\
x^2 & 0 \leq x \leq 1, y > 1\\
y^2 & 0 \leq y \leq 1, x > 1\\
1 & x > 1, y > 1 \end{cases}\]
Implied joint support:
Rectangular support \(\implies\) \(X\) and \(Y\) could be independent!
- \(\scriptsize F_X(x) = \lim_{y\rightarrow \infty}F(x,y) = x^2\)
- \(\scriptsize F_Y(y) = \lim_{x\rightarrow \infty}F(x,y) = y^2\)
- \(\scriptsize F_X(x)F_Y(y) = x^2 y^2 = F(x,y)\)
- \(X \perp\!\!\!\perp Y\)
