class: center, middle, inverse, title-slide # MATH 4140/5140 <br /> ## Lecture 3: Introduction and Review (continued) ### Ziwei Ma ### UTC ### updated: 2022-01-14 --- class: inverse, center, mid # Probability theory review --- ## Function of Random Variables - Method of Distribution Function - The Method of Transformations - The Method of Moment-generating Functions --- ## Method of Distribution Function Let `\(U\)` be a function of r.v.s `\(Y_1, \cdots, Y_n\)` ### General procedure 1. Find the region `\(U=u\)` in the `\((y_1,\cdots,y_n)\)` space 2. Find the region `\(U\leq u\)` 3. Find `\(F_u(u)\leq P(U\leq u)\)` by integrating `\(f(y_1,\cdots,y_n)\)` over the region `\(U\leq u\)` 4. Find the density function `\(f_U(u)\)` by differentiating `\(F_U(u)\)`. 5. Thus, `\(f_U(u)=F_U'(u)\)` --- ## The Method of Transformations - Univariate case Let `\(U=h(Y)\)`, where `\(h(y)\)` is either an increasing or decreasing function of `\(y\)` for all `\(y\)` such that `\(f_Y(y)>0\)` 1. Find the inverse function, `\(y = h^{−1}(u)\)` 2. Evaluate `\(\frac{d h^{-1}}{du}\)` 3. Find `\(f_U(u)\)` by $$ f_U(u)= f_Y(h^{-1}(u))|\frac{d h^{-1}}{du}| $$ --- - Multivariate case For `\(Y_1,\cdots, Y_k\)` are jointly continuous r.v.s with pdf `\(f_{Y_1,\cdots, Y_k}(y_1,\cdots, y_k)\)`. Consider `\(U_1=h_1(Y_1,\cdots, Y_k)\)`, ..., `\(U_k=h_k(Y_1,\cdots, Y_k)\)` where the transformation `$$u_1=h_1(y_1,\cdots, y_k)$$` `$$...$$` `$$u_k=h_k(y_1,\cdots, y_k)$$` is a one-to-one transformation from `\((y_1,\cdots, y_k)\)` to `\(u_1,\cdots, u_k\)` with inverse `$$y_1=h^{-1}_1(u_1,\cdots, u_k)$$` `$$...$$` `$$y_k=h^{-1}_k(u_1,\cdots, u_k)$$` and `\(h_1^{-1}, \cdots, h_k^{-1}\)` have continuous partial derivative with respective to `\(u_1,\cdots, u_k\)`. Then, the joint distribution function of `\(u_1,\cdots, u_k\)` is given by in the similar way to univariate case. --- ## Sampling Distributions Related to the Normal Distribution ### distribution of Sample mean Let `\(Y_1, Y_2, . . . , Y_n\)` be a random sample of size n from a normal distribution with mean `\(\mu\)` and variance `\(\sigma^2\)`. Then the sample mean `$$\bar{Y}=\frac{1}{n}\Sigma^{n}_{i=1}Y_i$$` is normally distributed with mean `\(\mu_{\bar{Y}}=\mu\)` and variance `\(\sigma^2_{\bar{Y}}=\sigma^2/n\)`. **Proof: ** Use moment generating function methods. --- ### Distribution of Sample variance Consider the sample variance `\(S^{2}=\frac{1}{n-1} \sum_{i=1}^{n}\left(Y_{i}-\bar{Y}\right)^{2}\)`. We want to show `\(\frac{(n-1)S^2}{\sigma^2}\sim \chi^2_{n-1}\)`. **Proof: ** There are three steps 1. Show `\(\sum_{i=1}^n\left( \frac{Y_i-\mu}{\sigma} \right)^2 \sim \chi^2_n\)` 2. Show `\(\bar{Y}\)` and `\(S^2\)` are independent 3. Use moment generating function methods to show the desired results. --- ## Student t distribution and F distribution ### t distribution There are two more useful distributions related normal population, Student t distribution and F distribution. - **t distribution** Let `\(Z \sim N(0,1)\)` and `\(W \sim \chi^2_{k}\)`. If `\(Z\)` and `\(W\)` are independent, then $$ T = \frac{Z}{\sqrt{W/k}} $$ is said to have a t distribution with degree of freedom `\(k\)`, denoted as `\(T\sim t_k\)`. **Exercise** Show statistics `\(\frac{\bar{Y}}{S/\sqrt{n}} \sim t_{(n-1)}\)`. (Hint: Let `\(Z=\frac{\bar{Y}}{\sigma/\sqrt{n}})\)` and `\(W = \frac{(n-1)S^2}{\sigma^2}\)`. --- ### t distribution ```r library(ggplot2) ggplot(data.frame(x = c(-4, 4)), aes(x = x)) + stat_function(fun = dnorm, aes(color = "Normal")) + stat_function(fun = dt, args = list(df = 3),aes(color = "Student t, df = 3")) + stat_function(fun = dt, args = list(df = 8),aes(color = "Student t, df = 8")) + scale_colour_manual("Dist", values = c("deeppink", "dodgerblue3","blue")) + theme(text = element_text(size=30)) ``` <!-- --> --- ### F distribution Let `\(W_1 \sim \chi^2_{k_1}\)` and `\(W_2 \sim \chi^2_{k_2}\)` be independently distributed. Then $$ F=\frac{W_1/k_1}{W_2/k_2} $$ is said to have F distribution with `\(k_1\)` numerator degrees of freedom and `\(k_2\)` denominator degrees of freedom. --- ```r library(ggplot2) ggplot(data.frame(x = c(0, 6)), aes(x = x)) + stat_function(fun = df, args = list(df1 = 3, df2 = 8),aes(color = "df = (3,8)")) + stat_function(fun = df, args = list(df1 = 10, df2 = 20),aes(color = "df = (10,20))")) + scale_colour_manual("Dist", values = c("deeppink", "blue")) + theme(text = element_text(size = 30)) ``` <!-- -->