1.1.1 Example 1

Two caplets are selected at random from a bottle containing three aspirins, two sedatives, and four laxative caplets.

If \(X\) and \(Y\) are, respectively, the numbers of aspirin and sedative caplets included among the two caplets drawn, find the probabilities associated with all possible pairs of values of \(X\) and \(Y\).

Solution:

The possible pairs are \((x, y)\) where \(x \in \{0, 1, 2\}\) and \(y \in \{0, 1, 2\}\), subject to \(x + y \leq 2\).

For example, the probability associated with \((1, 0)\): we want one aspirin, zero sedatives, and hence one laxative. The number of ways is \(\binom{3}{1}\binom{2}{0}\binom{4}{1}\) and the total number of ways to select two caplets from nine is \(\binom{9}{2}\).

Continuing this way, we obtain:

It is generally preferable to represent such probabilities by means of a formula — a function \(f(x, y)\) defined for any pair of values within the range of \(X\) and \(Y\).

1.1.2 Definition 1 – Joint Probability Distribution Function (Discrete)

If \(X\) and \(Y\) are discrete random variables, the function given by \(f(x, y) = P(X = x, Y = y)\) for each pair of values \((x, y)\) within the range of \(X\) and \(Y\) is called the joint probability distribution function of \(X\) and \(Y\).

1.1.3 Theorem 1

A bivariate function can serve as the joint probability distribution function of a pair of discrete random variables \(X\) and \(Y\) if and only if its values \(f(x, y)\) satisfy the conditions:

1. \(f(x, y) \geq 0\), for each pair of values \((x, y)\) within its domain.
2. \(\displaystyle\sum_x \sum_y f(x, y) = 1\), where the double summation extends over all possible pairs within the domain.

1.1.4 Example 2

Determine the value of \(k\) for which the function \(f(x, y) = k(x + y)\) for \(x = 0, 1, 2\) and \(y = 0, 1, 2\) can serve as a joint probability distribution function.

Solution:

Substituting the various values of \(x\) and \(y\) and applying Theorem 1, we get:

\[\sum_x \sum_y k(x + y) = 1\]

Solving for \(k\) gives the appropriate value.

1.2.1 Definition 2 – Joint CDF (Discrete)

If \(X\) and \(Y\) are discrete random variables, the function given by

\[F(x, y) = P(X \leq x, Y \leq y) = \sum_{s \leq x} \sum_{t \leq y} f(s, t)\]

for \(-\infty < x, y < \infty\), where \(f(s, t)\) is the value of the joint probability distribution of \(X\) and \(Y\) at \((s, t)\), is called the joint distribution function or the joint Cumulative Distribution Function (CDF) of \(X\) and \(Y\).

1.2.2 Example 3

With reference to Example 1, find \(F(1, 1)\).

Solution:

As in the univariate case, the joint distribution function is defined for all real numbers. For example, \(F(1, 1) = \sum_{x \leq 1} \sum_{y \leq 1} f(x, y)\).

1.3.1 Definition 3 – Joint PDF (Continuous)

A bivariate function with values \(f(x, y)\), defined over the \(xy\)-plane, is called a joint probability density function of a continuous random variable \(X\) and \(Y\) if and only if

\[P[(X, Y) \in A] = \iint_A f(x, y) \, dx \, dy\]

for any region \(A\) in the \(xy\)-plane.

1.3.2 Theorem 2

A bivariate function can serve as a joint probability density function of a pair of continuous random variables \(X\) and \(Y\) if and only if its values \(f(x, y)\) satisfy the conditions:

1. \(f(x, y) \geq 0\) for all \((x, y)\)
2. \(\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} f(x, y) \, dx \, dy = 1\)

1.3.3 Example 4

Given the joint probability density function \(f(x, y)\) of two random variables \(X\) and \(Y\), find \(P[(X, Y) \in A]\) where \(A\) is a given region.

Solution:

\[P[(X, Y) \in A] = \iint_A f(x, y) \, dx \, dy\]

1.4.1 Definition 4 – Joint CDF (Continuous)

If \(X\) and \(Y\) are continuous random variables, the function given by

\[F(x, y) = \int_{-\infty}^{x}\int_{-\infty}^{y} f(s, t) \, dt \, ds, \quad -\infty < x, y < \infty\]

where \(f(s, t)\) is the value of the joint probability density of \(X\) and \(Y\) at \((s, t)\), is called the joint Cumulative Density Function of \(X\) and \(Y\).

Analogous to the univariate case, partial differentiation yields:

\[\frac{\partial^2 F(x, y)}{\partial x \, \partial y} = f(x, y)\]

wherever these partial derivatives exist.

1.4.2 Example 5

If the joint probability density of \(X\) and \(Y\) is given by \(f(x, y)\), find the cumulative density function.

Solution:

\[F(x, y) = \int_{-\infty}^{x}\int_{-\infty}^{y} f(s, t) \, dt \, ds\]

1.4.3 Example 6

Find the joint probability density of the two random variables \(X\) and \(Y\) whose joint cumulative density function is given. Also use the joint probability density function to determine the required probability.

Solution:

1. Since \(f(x, y) = \dfrac{\partial^2 F(x,y)}{\partial x\, \partial y}\), partial differentiation yields the joint PDF.

2. The probability is evaluated by integrating \(f(x,y)\) over the relevant region.

For two random variables, the joint probability density is geometrically a surface, and the probability is given by the volume under this surface over the region \(A\).

1.5.1 Example 7

If the joint probability distribution of three random variables \(X\), \(Y\), and \(Z\) is given, find the specified probability.

Solution:

\[P(X = x, Y = y, Z = z) = \sum_{x}\sum_{y}\sum_{z} f(x, y, z)\]

In the continuous case, probabilities are obtained by integrating the joint probability density, and the joint CDF is given by

\[F(x_1, \ldots, x_n) = \int_{-\infty}^{x_1} \cdots \int_{-\infty}^{x_n} f(t_1, \ldots, t_n) \, dt_n \cdots dt_1\]

1.5.2 Example 8

If the trivariate probability density of \((X, Y, Z)\) is given by \(f(x, y, z)\), find the probability over a specified region \(A\).

Solution:

\[P[(X, Y, Z) \in A] = \iiint_A f(x, y, z) \, dz \, dy \, dx\]

##Exercises

1. Given the joint probability distribution table of \(X\) and \(Y\) (with rows \(y = 0, 1, 2, 3\) and columns \(x = 0, 1, 2\)), find the specified probabilities.

2. If the joint probability density of \(X\) and \(Y\) is given, find the value of \(c\), then find:

   1. \(P(X < 1, Y < 1)\)
   2. \(P(X + Y < 1)\)
   3. \(P(X > Y)\)

3. Determine \(k\) so that a given function can serve as a joint probability density function.

4. Find \(F(x, y)\) if the joint probability distribution of \(X\) and \(Y\) is given.

5. Find \(k\) if a given bivariate function can serve as a joint probability density.

6. Verify the joint distribution function of a given example.

7. If a trivariate probability density \(f(x_1, x_2, x_3)\) is given, find the specified probability.

8. A certain college gives attitude tests in the sciences and humanities. If \(X\) and \(Y\) are the proportions of correct answers in the two subjects, find the probabilities that a student will get:

   1. Less than \(\frac{1}{2}\) on both tests.
   2. More than \(\frac{1}{2}\) on the science test and less than \(\frac{1}{4}\) on the humanities test.

9. Suppose \(P\), the price of a commodity (in dollars), and \(S\), its total sales (in units), are random variables with a given joint probability density. Find the probabilities that:

   1. The price will be less than 30 cents and sales will exceed 10,000 units.
   2. The price will be between 25 cents and 30 cents and sales will be less than 6,000 units.

2.1.1 Definition 5 – Marginal Distribution (Discrete)

If \(X\) and \(Y\) are discrete random variables and \(f(x, y)\) is the value of their joint probability distribution at \((x, y)\), the function given by

\[g(x) = \sum_{y} f(x, y), \quad \text{for each } x \text{ within the range of } X\]

is called the marginal probability distribution function of \(X\).

Correspondingly, the function given by

\[h(y) = \sum_{x} f(x, y), \quad \text{for each } y \text{ within the range of } Y\]

is called the marginal probability distribution function of \(Y\).

2.1.2 Example 9

The joint probability distribution function of random variables \(X\) and \(Y\) is given in the table below (\(x, y \in \{1, 2, 3\}\)). Determine the marginal probability distribution functions of \(X\) and \(Y\).

Solution:

1. The marginal probability distribution function of \(X\): \(g(x) = \sum_y f(x, y)\)

2. The marginal probability distribution function of \(Y\): \(h(y) = \sum_x f(x, y)\)

2.1.3 Definition 6 – Marginal Density (Continuous)

If \(X\) and \(Y\) are continuous random variables and \(f(x, y)\) is the value of their joint probability density at \((x, y)\), the function given by

\[g(x) = \int_{-\infty}^{\infty} f(x, y) \, dy\]

is called the marginal probability density function of \(X\).

Correspondingly,

\[h(y) = \int_{-\infty}^{\infty} f(x, y) \, dx\]

is called the marginal probability density function of \(Y\).

2.1.4 Example 10

Given the joint probability density \(f(x, y)\), find the marginal probability density functions of \(X\) and \(Y\).

Solution:

\[g(x) = \int_{-\infty}^{\infty} f(x, y) \, dy, \qquad h(y) = \int_{-\infty}^{\infty} f(x, y) \, dx\]

2.2.1 Example 11

Considering the trivariate probability density \(f(x_1, x_2, x_3)\), find:

1. The joint marginal probability density function of \(X_1\) and \(X_2\).
2. The marginal probability density function of \(X_1\) alone.

Solution:

1. \(f_{12}(x_1, x_2) = \int_{-\infty}^{\infty} f(x_1, x_2, x_3) \, dx_3\)

2. \(f_1(x_1) = \int_{-\infty}^{\infty} f_{12}(x_1, x_2) \, dx_2\)

2.3.1 Definition 7 – Conditional Distribution (Discrete)

If \(f(x, y)\) is the value of the joint probability distribution of the discrete random variables \(X\) and \(Y\) at \((x, y)\), and \(h(y)\) is the value of the marginal distribution of \(Y\) at \(y\), the function given by

\[f(x \mid y) = \frac{f(x, y)}{h(y)}, \quad \text{for each } x \text{ within the range of } X\]

is called the conditional distribution of \(X\) given \(Y = y\).

Correspondingly, if \(g(x)\) is the value of the marginal distribution of \(X\) at \(x\):

\[f(y \mid x) = \frac{f(x, y)}{g(x)}, \quad \text{for each } y \text{ within the range of } Y\]

is called the conditional distribution of \(Y\) given \(X = x\).

2.3.2 Definition 8 – Conditional Density (Continuous)

If \(f(x, y)\) is the value of the joint density of continuous random variables \(X\) and \(Y\) at \((x, y)\), and \(h(y)\) is the value of the marginal density of \(Y\) at \(y\), the function given by

\[f(x \mid y) = \frac{f(x, y)}{h(y)}, \quad \text{for } h(y) > 0\]

is called the conditional density of \(X\) given \(Y = y\).

Correspondingly:

\[f(y \mid x) = \frac{f(x, y)}{g(x)}, \quad \text{for } g(x) > 0\]

is called the conditional density of \(Y\) given \(X = x\).

2.3.3 Example 12

Given the joint probability density \(f(x, y)\), find the marginal densities of \(X\) and \(Y\) and the conditional density of \(X\) given \(Y = y\).

Solution:

\[g(x) = \int f(x, y) \, dy, \qquad h(y) = \int f(x, y) \, dx\]

\[f(x \mid y) = \frac{f(x, y)}{h(y)}\]

2.3.4 Example 13

With reference to Example 1, find:

1. The conditional distribution of \(X\) given \(Y = 1\).
2. The conditional distribution of \(Y\) given \(X = 0\).

Solution:

The joint distribution together with marginal totals:

1. Conditional distribution of \(X\) given \(Y = 1\): \(\quad f(x \mid 1) = \dfrac{f(x, 1)}{h(1)}\)

2. Conditional distribution of \(Y\) given \(X = 0\): \(\quad f(y \mid 0) = \dfrac{f(0, y)}{g(0)}\)

2.4.1 Definition 9 – Independence (Discrete)

If \(f(x_1, x_2, \ldots, x_n)\) is the value of the joint probability distribution of \(n\) discrete random variables \(X_1, \ldots, X_n\) at \((x_1, \ldots, x_n)\), and \(f_i(x_i)\) is the value of the marginal distribution of \(X_i\), then the \(n\) random variables are independent if and only if

\[f(x_1, \ldots, x_n) = f_1(x_1) \cdot f_2(x_2) \cdots f_n(x_n)\]

for all \((x_1, \ldots, x_n)\) within their range.

2.4.2 Definition 10 – Independence (Continuous)

If \(f(x_1, x_2, \ldots, x_n)\) is the value of the joint probability density function of \(n\) continuous random variables \(X_1, \ldots, X_n\) at \((x_1, \ldots, x_n)\), and \(f_i(x_i)\) is the value of the marginal density of \(X_i\), then the \(n\) random variables are independent if and only if

\[f(x_1, \ldots, x_n) = f_1(x_1) \cdot f_2(x_2) \cdots f_n(x_n)\]

for all \((x_1, \ldots, x_n)\) within their range.

2.4.3 Example 14

If the joint probability density of \(X\) and \(Y\) is given, find:

1. The marginal density of \(X\).
2. The marginal density of \(Y\).
3. Whether the two random variables are independent.

Solution:

Compute \(g(x) = \int f(x,y)\,dy\) and \(h(y) = \int f(x,y)\,dx\). Then check whether \(f(x,y) = g(x) \cdot h(y)\).

If \(f(x,y) \neq g(x) \cdot h(y)\), the two random variables are not independent.

2.4.4 Example 15

With reference to Example 1, determine whether \(X\) and \(Y\) are independent.

Solution:

Using the marginal distributions obtained previously, check whether \(f(x,y) = g(x) \cdot h(y)\) for all pairs. If any pair fails, \(X\) and \(Y\) are not independent.

2.4.5 Example 16

Considering independent flips of a balanced coin, let \(X_i\) be the number of heads (0 or 1) obtained on the \(i\)-th flip. Find the joint probability distribution of these \(n\) random variables.

Solution:

Since each \(X_i\) has the distribution \(P(X_i = x) = \frac{1}{2}\) for \(x = 0, 1\), and the random variables are independent, their joint probability distribution is:

\[f(x_1, \ldots, x_n) = \prod_{i=1}^{n} \frac{1}{2} = \left(\frac{1}{2}\right)^n, \quad x_i \in \{0, 1\}\]

2.4.6 Example 17

Given independent random variables \(X_1\), \(X_2\), \(X_3\) with probability densities \(f_1(x_1)\), \(f_2(x_2)\), \(f_3(x_3)\), find their joint probability density and use it to evaluate a specified probability.

Solution:

The joint probability density function is:

\[f(x_1, x_2, x_3) = f_1(x_1) \cdot f_2(x_2) \cdot f_3(x_3)\]

3.1.1 Definition 1 – Bivariate Normal Distribution

A pair of random variables \(X\) and \(Y\) have a bivariate normal distribution and are referred to as jointly normally distributed random variables, if and only if their joint probability density is given by:

\[f(x, y) = \frac{1}{2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}} \exp\!\left\{-\frac{1}{2(1-\rho^2)}\left[\frac{(x-\mu_1)^2}{\sigma_1^2} - \frac{2\rho(x-\mu_1)(y-\mu_2)}{\sigma_1\sigma_2} + \frac{(y-\mu_2)^2}{\sigma_2^2}\right]\right\}\]

where \(-1 < \rho < 1\).

To study this joint distribution, we first show that the parameters \(\mu_1, \mu_2\) are the means and \(\sigma_1, \sigma_2\) are the standard deviations of \(X\) and \(Y\) respectively.

Integrating on \(y\) from \(-\infty\) to \(\infty\) gives the marginal density of \(X\). After completing the square and simplifying:

\[g(x) = \frac{1}{\sigma_1\sqrt{2\pi}} e^{-\frac{(x-\mu_1)^2}{2\sigma_1^2}}\]

which is a normal distribution with mean \(\mu_1\) and standard deviation \(\sigma_1\). By symmetry, the marginal density of \(Y\) is a normal distribution with mean \(\mu_2\) and standard deviation \(\sigma_2\).

The parameter \(\rho\) is called the correlation coefficient, and it can be shown that \(-1 < \rho < 1\). It measures how the two random variables \(X\) and \(Y\) vary together.

3.2.1 Theorem 1 – Conditional Distributions of the Bivariate Normal

If \(X\) and \(Y\) have a bivariate normal distribution, the conditional density of \(Y\) given \(X = x\) is a normal distribution with:

• Mean: \(\mu_2 + \rho\dfrac{\sigma_2}{\sigma_1}(x - \mu_1)\)
• Variance: \(\sigma_2^2(1 - \rho^2)\)

And the conditional density of \(X\) given \(Y = y\) is a normal distribution with:

• Mean: \(\mu_1 + \rho\dfrac{\sigma_1}{\sigma_2}(y - \mu_2)\)
• Variance: \(\sigma_1^2(1 - \rho^2)\)

Proof:

Writing \(f(y \mid x) = \dfrac{f(x, y)}{g(x)}\) and simplifying the exponent, one arrives at a normal density with the stated mean and variance. The corresponding result for the conditional density of \(X\) given \(Y = y\) follows by symmetry.

3.3.1 Theorem 2 – Independence in the Bivariate Normal

If two random variables have a bivariate normal distribution, they are independent if and only if \(\rho = 0\).

Proof: (exercise)

Remark: When \(\rho = 0\), the random variables are said to be uncorrelated. Note that the marginal distributions may both be normal without the joint distribution being bivariate normal — the converse does not necessarily hold.

3.4.1 Example 1

In a certain population of married couples, the height \(X\) (in meters) of the husband and the height \(Y\) of the wife have a bivariate normal distribution with given parameters.

1. Find the mean and the variance of the height for the husbands and wives.
2. Find the expected height and variance of the husband whose wife is expected to be 1.55 m tall.

Solution:

1. Comparing the given density with the general formula, we identify:

\[\mu_1 = \ldots, \quad \sigma_1 = \ldots, \quad \mu_2 = \ldots, \quad \sigma_2 = \ldots, \quad \rho = \ldots\]

2. By Theorem 1, the conditional mean of \(X\) given \(Y = 1.55\) is:

\[E(X \mid Y = 1.55) = \mu_1 + \rho\frac{\sigma_1}{\sigma_2}(1.55 - \mu_2)\]

The conditional variance of \(X\) given \(Y = y\) is:

\[\text{Var}(X \mid Y = y) = \sigma_1^2(1 - \rho^2)\]

3.4.2 Example 2

Let \(X\) and \(Y\) have a bivariate normal distribution with parameters \(\mu_1, \mu_2, \sigma_1, \sigma_2, \rho\).

Find:

1. \(P(\ldots)\) [2 marks]
2. \(P(\ldots \mid X = 74)\) [6 marks]

Solution:

1. Compute using the standard normal CDF.

2. The conditional distribution of \(Y\) given \(X = 74\) is normal with:

\[\text{Mean} = \mu_2 + \rho\frac{\sigma_2}{\sigma_1}(74 - \mu_1), \qquad \text{Variance} = \sigma_2^2(1 - \rho^2)\]

Hence, standardize and use the standard normal table to find the required probability.

4.1.1 Example 1

Problem: The probability density of \(X\) is given by:

\[f(x) = \begin{cases} 6x(1 - x), & 0 < x < 1 \\ 0, & \text{elsewhere} \end{cases}\]

Find the probability density of \(Y = X^3\).

Solution:

Letting \(G(y)\) denote the distribution function of \(Y\):

\[G(y) = \Pr(Y \leq y) = \Pr(X^3 \leq y) = \Pr\!\left(X \leq y^{1/3}\right)\]

\[= \int_0^{y^{1/3}} 6x(1-x)\, dx = 3y^{2/3} - 2y\]

Differentiating:

\[g(y) = \frac{d}{dy} G(y) = 2y^{-1/3} - 2 = \frac{2}{3}\left(y^{-1/3} - 1\right) \cdot 3 = 2\left(y^{-1/3} - 1\right)\]

More precisely:

\[g(y) = \frac{1}{3}\left(2y^{-1/3} - 2\right)\]

\[\boxed{g(y) = \begin{cases} \frac{1}{3}(2y^{-1/3} - 2), & 0 < y < 1 \\ 0, & \text{elsewhere} \end{cases}}\]

4.1.2 Example 2

Problem: If \(Y = |X|\), show that:

\[g(y) = \begin{cases} f(y) + f(-y), & y > 0 \\ 0, & \text{elsewhere} \end{cases}\]

where \(f(x)\) is the probability density of \(X\).

Solution:

For \(y > 0\):

\[G(y) = \Pr(Y \leq y) = \Pr(|X| \leq y) = \Pr(-y \leq X \leq y) = F(y) - F(-y)\]

Differentiating:

\[g(y) = \frac{d}{dy}\left[F(y) - F(-y)\right] = f(y) \cdot 1 - f(-y) \cdot (-1) = f(y) + f(-y)\]

Since \(X\) cannot be negative, \(g(y) = 0\) for \(y < 0\). Letting \(g(0) = 0\):

\[\boxed{g(y) = \begin{cases} f(y) + f(-y), & y > 0 \\ 0, & \text{elsewhere} \end{cases}}\]

4.1.3 Example 3

Problem: Given the joint density of \(X_1\) and \(X_2\):

\[f(x_1, x_2) = \begin{cases} 6e^{-x_1 - 2x_2}, & x_1 > 0,\ x_2 > 0 \\ 0, & \text{elsewhere} \end{cases}\]

Find the probability density of \(Y = X_1 + X_2\).

Solution:

Integrating the joint density over the region where \(x_1 + x_2 \leq y\):

\[F(y) = \int_0^y \int_0^{y - x_1} 6e^{-x_1 - 2x_2}\, dx_2\, dx_1\]

After integration:

\[F(y) = 1 - 2e^{-y} + 3e^{-2y} - 2e^{-3y} \quad \text{(after simplification)}\]

Differentiating gives the density:

\[\boxed{f(y) = \begin{cases} 6e^{-2y} - 6e^{-3y}, & y > 0 \\ 0, & \text{elsewhere} \end{cases}}\]

5.1.1 Discrete Case

In the discrete case, when the relationship between the values of \(X\) and \(Y = u(X)\) is one-to-one, we simply make the appropriate substitution.

5.1.2 Example 4

Problem: If \(X\) is the number of heads in four tosses of a fair coin, find the probability distribution of:

\[Y = \frac{1}{X + 1}\]

Solution:

Using the binomial distribution with \(n = 4\), \(p = \frac{1}{2}\):

Applying \(y = \frac{1}{x+1}\):

Substituting \(x = \frac{1}{y} - 1\) into the binomial formula directly:

\[g(y) = \binom{4}{\frac{1}{y}-1}\left(\frac{1}{2}\right)^4, \quad y = 1, \tfrac{1}{2}, \tfrac{1}{3}, \tfrac{1}{4}, \tfrac{1}{5}\]

Note: The probabilities remain unchanged — only the variable changes from \(X\) to \(Y\).

5.1.3 Example 5 — Non One-to-One Case

Problem: Find the probability distribution of \(Z = (X - 2)^2\) where \(X\) is as in Example 4.

Solution:

Solving \(x = 2 \pm \sqrt{z}\) and summing probabilities for each \(z\):

\[h(0) = f(2) = \frac{6}{16} = \frac{3}{8}\]

\[h(1) = f(1) + f(3) = \frac{4}{16} + \frac{4}{16} = \frac{8}{16} = \frac{1}{2}\]

\[h(4) = f(0) + f(4) = \frac{1}{16} + \frac{1}{16} = \frac{2}{16} = \frac{1}{8}\]

5.1.4 Continuous Case — Theorem 1

Theorem 1: Let \(f(x)\) be the probability density of the continuous random variable \(X\). If \(y = u(x)\) is differentiable and either strictly increasing or strictly decreasing for all values where \(f(x) > 0\), then the equation \(y = u(x)\) can be uniquely solved for \(x = w(y)\), and the probability density of \(Y = u(X)\) is:

\[g(y) = f[w(y)] \cdot \left|w'(y)\right|, \quad \text{provided } u'(x) \neq 0\]

and \(g(y) = 0\) elsewhere.

Proof sketch: For the increasing case, \(\Pr(a < Y < b) = \Pr(w(a) < X < w(b))\). Changing variables \(x = w(y)\), \(dx = w'(y)\,dy\) in the integral yields \(g(y) = f[w(y)] \cdot |w'(y)|\). The decreasing case follows similarly, with the absolute value ensuring positivity. \(\square\)

5.1.5 Example 6

Problem: If \(X\) has the exponential distribution \(f(x) = e^{-x}\) for \(x > 0\) (0 elsewhere), find the p.d.f. of \(Y = \sqrt{X}\).

Solution:

The inverse is \(x = y^2\), so \(w(y) = y^2\) and \(w'(y) = 2y\).

\[g(y) = f(y^2) \cdot |2y| = e^{-y^2} \cdot 2y = 2ye^{-y^2}\]

\[\boxed{g(y) = \begin{cases} 2ye^{-y^2}, & y > 0 \\ 0, & \text{elsewhere} \end{cases}}\]

5.1.6 Example 7 — Cauchy Distribution

Problem: A spinner gives angle \(\Theta\) with uniform density \(f(\theta) = \frac{1}{2\pi}\) for \(-\frac{\pi}{2} < \theta < \frac{\pi}{2}\) (0 elsewhere). Find the density of \(X = a\tan\Theta\), the abscissa on the \(x\)-axis.

Solution:

The inverse is \(\theta = \arctan(x/a)\), so:

\[\frac{d\theta}{dx} = \frac{a}{a^2 + x^2}\]

\[g(x) = \frac{1}{2\pi} \cdot \frac{a}{a^2 + x^2} \cdot 2\pi = \frac{a}{\pi(a^2 + x^2)} \quad \text{for } -\infty < x < \infty\]

This is the Cauchy distribution.

5.2.1 Motivation

Given the joint distribution of \(X_1\) and \(X_2\), we want the distribution of \(Y = u(X_1, X_2)\).

Strategy: Introduce an auxiliary variable \(Z\) (e.g., \(Z = X_2\)), find the joint density of \((Y, Z)\), then integrate out \(Z\) to obtain the marginal density of \(Y\).

5.2.2 Theorem 2 — Jacobian Transformation

Theorem 2: Let \(f(x_1, x_2)\) be the joint probability density of the continuous random variables \(X_1\) and \(X_2\). If:

\[y_1 = u_1(x_1, x_2) \quad \text{and} \quad y_2 = u_2(x_1, x_2)\]

are partially differentiable and define a one-to-one transformation, with inverse \(x_i = w_i(y_1, y_2)\), then the joint density of \(Y_1 = u_1(X_1, X_2)\) and \(Y_2 = u_2(X_1, X_2)\) is:

\[g(y_1, y_2) = f[w_1(y_1, y_2),\ w_2(y_1, y_2)] \cdot |J|\]

where the Jacobian is:

\[J = \begin{vmatrix} \dfrac{\partial x_1}{\partial y_1} & \dfrac{\partial x_1}{\partial y_2} \\[8pt] \dfrac{\partial x_2}{\partial y_1} & \dfrac{\partial x_2}{\partial y_2} \end{vmatrix}\]

5.2.3 Example 8 — Poisson Sum

Problem: \(X_1\) and \(X_2\) are independent Poisson random variables with parameters \(\lambda_1\) and \(\lambda_2\). Find the distribution of \(Y = X_1 + X_2\).

Solution:

Their joint distribution is:

\[f(x_1, x_2) = \frac{e^{-\lambda_1}\lambda_1^{x_1}}{x_1!} \cdot \frac{e^{-\lambda_2}\lambda_2^{x_2}}{x_2!}\]

Setting \(y = x_1 + x_2\) (so \(x_1 = y - x_2\)) and summing over \(x_2 = 0, 1, \ldots, y\):

\[h(y) = \sum_{x_2=0}^{y} \frac{e^{-(\lambda_1+\lambda_2)}\lambda_1^{y-x_2}\lambda_2^{x_2}}{(y-x_2)!\, x_2!} = \frac{e^{-(\lambda_1+\lambda_2)}(\lambda_1+\lambda_2)^y}{y!}\]

using the binomial expansion \((\lambda_1 + \lambda_2)^y = \sum_{x_2=0}^{y}\binom{y}{x_2}\lambda_1^{y-x_2}\lambda_2^{x_2}\).

Conclusion: The sum of two independent Poisson random variables with parameters \(\lambda_1\) and \(\lambda_2\) is Poisson with parameter \(\lambda_1 + \lambda_2\).

5.2.4 Example 9 — Exponential to Beta/Gamma

Problem: Given joint density \(f(x_1, x_2) = e^{-(x_1+x_2)}\) for \(x_1, x_2 > 0\) (0 elsewhere), find:

• 1. The joint density of \(Y_1 = X_1 + X_2\) and \(Y_2 = \dfrac{X_1}{X_1 + X_2}\)
• 2. The marginal density of \(Y_2\)

Solution (a):

Solving: \(x_1 = y_1 y_2\) and \(x_2 = y_1(1 - y_2)\).

\[J = \begin{vmatrix} y_2 & y_1 \\ 1-y_2 & -y_1 \end{vmatrix} = -y_1 y_2 - y_1(1-y_2) = -y_1\]

\[|J| = y_1\]

\[g(y_1, y_2) = e^{-y_1 y_2 - y_1(1-y_2)} \cdot y_1 = y_1 e^{-y_1}\]

\[\boxed{g(y_1, y_2) = \begin{cases} y_1 e^{-y_1}, & y_1 > 0,\ 0 < y_2 < 1 \\ 0, & \text{elsewhere} \end{cases}}\]

Solution (b):

\[h(y_2) = \int_0^\infty y_1 e^{-y_1}\, dy_1 = \Gamma(2) = 1! = 1\]

\[\boxed{h(y_2) = \begin{cases} 1, & 0 < y_2 < 1 \\ 0, & \text{elsewhere} \end{cases}}\]

This is the uniform distribution on \((0, 1)\).

5.2.5 Example 10 — Triangular Distribution

Problem: \(f(x_1, x_2) = 1\) for \(0 < x_1 < 1\), \(0 < x_2 < 1\) (0 elsewhere). Find the marginal density of \(Y = X_1 + X_2\).

Solution:

Let \(Y = X_1 + X_2\) and \(Z = X_2\). Then \(x_1 = y - z\), \(x_2 = z\), \(|J| = 1\).

The region maps to \(z < y < z+1\), \(0 < z < 1\), i.e., \(z < y < z+1\).

Integrating out \(z\):

\[h(y) = \begin{cases} \int_0^y dz = y, & 0 \leq y < 1 \\[4pt] \int_{y-1}^{1} dz = 2 - y, & 1 \leq y \leq 2 \\[4pt] 0, & \text{elsewhere} \end{cases}\]

This is the triangular distribution.

5.2.6 Example 11 — Three Variables

Problem: \(f(x_1, x_2, x_3) = e^{-(x_1+x_2+x_3)}\) for \(x_i > 0\) (0 elsewhere). Find the marginal density of \(Y_1 = X_1 + X_2 + X_3\).

Solution:

Let \(Y_2 = X_2\) and \(Y_3 = X_3\). Then \(x_1 = y_1 - y_2 - y_3\), \(x_2 = y_2\), \(x_3 = y_3\). The \(3\times 3\) Jacobian equals 1.

\[g(y_1, y_2, y_3) = e^{-y_1}, \quad y_1 > y_2 + y_3,\ y_2, y_3 > 0\]

Integrating out \(y_2\) and \(y_3\):

\[h(y_1) = \int_0^{y_1}\int_0^{y_1 - y_3} e^{-y_1}\, dy_2\, dy_3 = \frac{1}{2}y_1^2 e^{-y_1}\]

\[\boxed{h(y_1) = \begin{cases} \dfrac{1}{2}y_1^2 e^{-y_1}, & y_1 > 0 \\ 0, & \text{elsewhere} \end{cases}}\]

This is a Gamma distribution with \(\alpha = 3\) and \(\beta = 1\).

6.6.1 Statement

Chebyshev’s Theorem: If \(\mu\) and \(\sigma\) are the mean and standard deviation of a random variable \(X\), then for any \(k > 0\):

\[\Pr(|X - \mu| < k\sigma) \geq 1 - \frac{1}{k^2}\]

Equivalently: \(\Pr(\mu - k\sigma < X < \mu + k\sigma) \geq 1 - \dfrac{1}{k^2}\).

This provides a universal lower bound on the probability that \(X\) lies within \(k\) standard deviations of its mean, valid for any distribution.

6.6.2 Proof

By definition:

\[\sigma^2 = E[(X - \mu)^2] = \int_{-\infty}^{\infty}(x-\mu)^2 f(x)\,dx\]

Split the integral into three regions:

\[\sigma^2 \geq \int_{-\infty}^{\mu - k\sigma}(x-\mu)^2 f(x)\,dx + \int_{\mu + k\sigma}^{\infty}(x-\mu)^2 f(x)\,dx\]

Since \((x - \mu)^2 \geq k^2\sigma^2\) in the outer regions:

\[\sigma^2 \geq k^2\sigma^2 \cdot \Pr(|X - \mu| \geq k\sigma)\]

\[\Pr(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}\]

\[\Pr(|X - \mu| < k\sigma) \geq 1 - \frac{1}{k^2} \quad \square\]

6.6.3 Example 1 — Beta Distribution

Problem: \(X\) has p.d.f. \(f(x) = 630x^4(1-x)^4\) for \(0 < x < 1\). Compare the exact probability within two standard deviations of the mean to Chebyshev’s bound.

Solution:

Comparing to the Beta\((\alpha, \beta)\) density, we identify \(\alpha = 5\), \(\beta = 5\).

\[\mu = E(X) = \frac{\alpha}{\alpha + \beta} = \frac{5}{10} = 0.5\]

\[\sigma^2 = \text{Var}(X) = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)} = \frac{25}{100 \times 11} = \frac{1}{44}\]

\[\sigma = \frac{1}{\sqrt{44}} \approx 0.1508\]

The interval within 2 standard deviations: \([0.5 - 2(0.1508),\ 0.5 + 2(0.1508)] = [0.20, 0.80]\).

Exact probability:

\[P(0.20 < X < 0.80) = 630\int_{0.20}^{0.80} x^4(1-x)^4\, dx \approx \mathbf{0.96}\]

Chebyshev bound (with \(k = 2\)):

\[1 - \frac{1}{k^2} = 1 - \frac{1}{4} = 0.75\]

The exact probability \(0.96\) is much stronger than Chebyshev’s lower bound \(0.75\).

6.6.4 Example 2 — Uniform Distribution

Problem: \(X \sim \text{Uniform}(-3, 3)\). With \(k = \frac{3}{2}\), find:

• 1. The exact probability
• 2. The Chebyshev bound

Solution:

\[\mu = \frac{-3 + 3}{2} = 0, \qquad \sigma^2 = \frac{(3 - (-3))^2}{12} = \frac{36}{12} = 3, \qquad \sigma = \sqrt{3}\]

With \(k = \frac{3}{2}\): \(k\sigma = \frac{3}{2}\sqrt{3}\)

Exact probability:

\[P\!\left(|X| < \frac{3\sqrt{3}}{2}\right) = P\!\left(-\frac{3\sqrt{3}}{2} < X < \frac{3\sqrt{3}}{2}\right) = \frac{1}{6} \cdot 3\sqrt{3} \approx 0.866\]

Actually, since \(\frac{3\sqrt{3}}{2} \approx 2.598 < 3\):

\[P = \frac{2 \cdot 2.598}{6} = \frac{5.196}{6} \approx \mathbf{0.866}\]

Chebyshev bound:

\[1 - \frac{1}{(3/2)^2} = 1 - \frac{4}{9} = \frac{5}{9} \approx 0.556\]

7.1.1 Definition 1 – Random Sample

If \(X_1, X_2, \ldots, X_n\) are independent and identically distributed random variables, we say they constitute a random sample from the infinite population given by their common distribution.

If \(f(x_1, x_2, \ldots, x_n)\) is the value of the joint distribution, we can write:

\[f(x_1, x_2, \ldots, x_n) = \prod_{i=1}^{n} f(x_i)\]

where \(f(x_i)\) is the value of the population distribution at \(x_i\).

Statistical inferences are usually based on statistics — random variables that are functions of \(X_1, X_2, \ldots, X_n\).

7.1.2 Definition 2 – Sample Mean and Sample Variance

If \(X_1, X_2, \ldots, X_n\) constitute a random sample, then:

\[\bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i \quad \text{(sample mean)}\]

\[S^2 = \frac{\sum_{i=1}^{n}(X_i - \bar{X})^2}{n - 1} \quad \text{(sample variance)}\]

For observed sample data, we calculate \(\bar{x}\) and \(s^2\); these are values of the corresponding random variables \(\bar{X}\) and \(S^2\).

7.2.1 Theorem 1 – Mean and Variance of \(\bar{X}\)

If \(X_1, X_2, \ldots, X_n\) constitute a random sample from an infinite population with mean \(\mu\) and variance \(\sigma^2\), then:

\[E(\bar{X}) = \mu \qquad \text{and} \qquad \text{Var}(\bar{X}) = \frac{\sigma^2}{n}\]

Proof:

\[E(\bar{X}) = E\!\left(\frac{1}{n}\sum_{i=1}^n X_i\right) = \frac{1}{n}\sum_{i=1}^n E(X_i) = \frac{1}{n} \cdot n\mu = \mu\]

\[\text{Var}(\bar{X}) = \text{Var}\!\left(\frac{1}{n}\sum_{i=1}^n X_i\right) = \frac{1}{n^2}\sum_{i=1}^n \text{Var}(X_i) = \frac{1}{n^2} \cdot n\sigma^2 = \frac{\sigma^2}{n}\]

The standard error of the mean is \(\sigma_{\bar{X}} = \dfrac{\sigma}{\sqrt{n}}\). As \(n\) increases, \(\sigma_{\bar{X}}\) decreases — larger samples yield \(\bar{X}\) values closer to \(\mu\).

7.2.2 Theorem 2 – Chebyshev’s Bound for \(\bar{X}\)

For any positive constant \(c\), the probability that \(\bar{X}\) falls between \(\mu - c\) and \(\mu + c\) is at least:

\[1 - \frac{\sigma^2}{nc^2}\]

As \(n \to \infty\), this probability approaches 1.

Proof:

From Chebyshev’s theorem, for any random variable with mean \(\mu\) and standard deviation \(\sigma\), and any \(k > 0\):

\[P(|X - \mu| < k\sigma) \geq 1 - \frac{1}{k^2}\]

Applying this to \(\bar{X}\) (which has standard deviation \(\sigma/\sqrt{n}\)), set \(k\sigma_{\bar{X}} = c\), so \(k = \dfrac{c\sqrt{n}}{\sigma}\):

\[P(|\bar{X} - \mu| < c) \geq 1 - \frac{\sigma^2}{nc^2}\]

This result is known as the Law of Large Numbers. \(\blacksquare\)

7.3.1 Theorem 3 – Central Limit Theorem (CLT)

If \(X_1, X_2, \ldots, X_n\) constitute a random sample from an infinite population with mean \(\mu\), variance \(\sigma^2\), and moment-generating function \(M_X(t)\), then the limiting distribution of:

\[Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}}\]

as \(n \to \infty\) is the standard normal distribution.

Practical rule: The CLT approximation is used when \(n \geq 30\), regardless of the shape of the population.

Important note: The CLT does not say the distribution of \(\bar{X}\) becomes normal (since \(\text{Var}(\bar{X}) \to 0\)). It justifies approximating \(\bar{X}\) with a normal having mean \(\mu\) and variance \(\sigma^2/n\) when \(n\) is large.

7.3.2 Example 1 – Soft Drink Vending Machine

A vending machine dispenses drinks with mean \(\mu = 200\,\text{ml}\) and standard deviation \(\sigma = 15\,\text{ml}\). Find \(P(\bar{X} \geq 204)\) for \(n = 36\).

Solution:

\[\sigma_{\bar{X}} = \frac{15}{\sqrt{36}} = 2.5\]

\[P(\bar{X} \geq 204) = P\!\left(Z \geq \frac{204 - 200}{2.5}\right) = P(Z \geq 1.6)\]

\[= 1 - P(Z \leq 1.6) = 1 - 0.9452 = \boxed{0.0548}\]

7.3.3 Example 2 – Gamma Population

A random sample of size \(n = 72\) is taken from:

\[f(x) = \frac{1}{16}x\,e^{-x/4}, \quad x > 0\]

Use the CLT to find \(P(\bar{X} > 9)\).

Solution:

Identifying this as a Gamma distribution with \(\alpha = 2\), \(\beta = 4\):

\[E(X) = \alpha\beta = 8, \qquad \text{Var}(X) = \alpha\beta^2 = 32\]

\[P(\bar{X} > 9) = P\!\left(Z > \frac{9 - 8}{\sqrt{32/72}}\right) = P(Z > 1.5) = 1 - 0.9332 = \boxed{0.0668}\]

7.3.4 Theorem 4 – Exact Distribution for Normal Populations

If \(\bar{X}\) is the mean of a random sample of size \(n\) from a normal population with mean \(\mu\) and variance \(\sigma^2\), then the exact sampling distribution is:

\[\bar{X} \sim N\!\left(\mu,\; \frac{\sigma^2}{n}\right)\]

(This holds for any \(n\), without the CLT approximation.)

8.1.1 MGF of the Chi-Square Distribution

\[M_X(t) = (1 - 2t)^{-r/2}, \quad t < \frac{1}{2}\]

Proof sketch: Substituting \(y = x(1-2t)/2\) into the integral and using the Gamma function identity \(\int_0^\infty y^{r/2-1}e^{-y}\,dy = \Gamma(r/2)\) gives the result above.

8.1.2 Mean and Variance of the Chi-Square Distribution

Differentiating \(M_X(t) = (1-2t)^{-r/2}\):

\[E(X) = r \qquad \text{and} \qquad \text{Var}(X) = 2r\]

8.1.3 Theorem 5 – Square of Standard Normal

If \(Z \sim N(0,1)\), then \(Z^2 \sim \chi^2_1\).

Proof: The MGF of \(Z^2\) is derived as \((1-2t)^{-1/2}\), which is the MGF of \(\chi^2_1\). \(\blacksquare\)

8.1.4 Theorem 6 – Sum of Squared Standard Normals

If \(X_1, X_2, \ldots, X_n\) are independent \(N(0,1)\) random variables, then:

\[Y = \sum_{i=1}^{n} X_i^2 \sim \chi^2_n\]

Proof: By Theorem 5 each \(X_i^2 \sim \chi^2_1\), and since they are independent:

\[M_Y(t) = \prod_{i=1}^n (1-2t)^{-1/2} = (1-2t)^{-n/2}\]

which is the MGF of \(\chi^2_n\). \(\blacksquare\)

8.1.5 Theorem 7 – Reproductive Property

If \(X_1, \ldots, X_n\) are independent with \(X_i \sim \chi^2_{r_i}\), then:

\[Y = \sum_{i=1}^n X_i \sim \chi^2_{r_1 + r_2 + \cdots + r_n}\]

8.1.6 Theorem 8 – Difference of Chi-Square Variables

If \(X_1\) and \(X_2\) are independent, \(X_1 \sim \chi^2_{r_1}\), and \(X_1 + X_2 \sim \chi^2_{r_1+r}\), then \(X_2 \sim \chi^2_r\).

8.1.7 Theorem 9 – Sample Mean and Variance from a Normal Population

If \(\bar{X}\) and \(S^2\) are the mean and variance of a random sample of size \(n\) from \(N(\mu, \sigma^2)\), then:

(a) \(\bar{X}\) and \(S^2\) are independent.

(b) \(\dfrac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}\)

Proof of (b): Using the identity:

\[\sum_{i=1}^n \frac{(X_i - \mu)^2}{\sigma^2} = \frac{(n-1)S^2}{\sigma^2} + \left(\frac{\bar{X} - \mu}{\sigma/\sqrt{n}}\right)^2\]

The left side is \(\chi^2_n\) (by Theorem 6). The second term on the right is \(\chi^2_1\) (by Theorems 4 and 5). By Theorem 8, \(\dfrac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}\). \(\blacksquare\)

8.1.8 Example 2 – Semiconductor Thickness

A manufacturing process is “in control” if \(\sigma \leq 0.60\) thousandths of an inch. For \(n = 20\), the process is declared “out of control” if \(\dfrac{(n-1)S^2}{\sigma^2} \geq \chi^2_{0.01,\,19} = 36.191\).

With \(S = 0.84\), \(\sigma = 0.60\):

\[\frac{(n-1)S^2}{\sigma^2} = \frac{19 \times (0.84)^2}{(0.60)^2} = 37.24 > 36.191\]

Conclusion: Reject \(H_0\); the process is out of control.

8.2.1 Theorem 10 – Definition of the t-Distribution

If \(Y \sim \chi^2_r\) and \(Z \sim N(0,1)\) are independent, then:

\[T = \frac{Z}{\sqrt{Y/r}}\]

has the t-distribution with \(r\) degrees of freedom, with pdf:

\[f(t) = \frac{\Gamma\!\left(\frac{r+1}{2}\right)}{\sqrt{r\pi}\;\Gamma\!\left(\frac{r}{2}\right)} \left(1 + \frac{t^2}{r}\right)^{-(r+1)/2}, \quad -\infty < t < \infty\]

Originally introduced by W.S. Gosset under the pen-name “Student” (his employer, a brewery, did not permit employee publications). Hence also known as Student’s t-distribution.

8.2.2 Theorem 11 – t-Statistic for a Normal Sample

If \(\bar{X}\) and \(S^2\) are the mean and variance of a random sample of size \(n\) from \(N(\mu, \sigma^2)\), then:

\[T = \frac{\bar{X} - \mu}{S/\sqrt{n}} \sim t_{n-1}\]

Proof: By Theorem 9, set \(Z = \dfrac{\bar{X}-\mu}{\sigma/\sqrt{n}} \sim N(0,1)\) and \(Y = \dfrac{(n-1)S^2}{\sigma^2} \sim \chi^2_{n-1}\), which are independent. Substituting into Theorem 10:

\[T = \frac{Z}{\sqrt{Y/(n-1)}} = \frac{(\bar{X}-\mu)/(\sigma/\sqrt{n})}{\sqrt{(n-1)S^2/[\sigma^2(n-1)]}} = \frac{\bar{X}-\mu}{S/\sqrt{n}} \sim t_{n-1} \quad \blacksquare\]

8.2.3 Example 3 – Gasoline Consumption

In 16 one-hour test runs, an engine averaged \(\bar{x} = 16.4\) gallons with \(s = 2.1\) gallons. Test the claim that \(\mu = 12.0\) gallons per hour.

Solution:

\[t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}} = \frac{16.4 - 12.0}{2.1/\sqrt{16}} = \frac{4.4}{0.525} = 8.38\]

From t-tables: \(t_{0.005,\,15} = 2.947\).

Since \(8.38 > 2.947\), we reject the claim. The true average consumption exceeds 12.0 gallons per hour.

9.1.1 Theorem 12 – Definition of the F-Distribution

If \(U \sim \chi^2_{n_1}\) and \(V \sim \chi^2_{n_2}\) are independent, then:

\[F = \frac{U/n_1}{V/n_2}\]

has an F-distribution with \(n_1\) and \(n_2\) degrees of freedom, with pdf:

\[g(f) = \frac{\Gamma\!\left(\frac{n_1+n_2}{2}\right)}{\Gamma\!\left(\frac{n_1}{2}\right)\Gamma\!\left(\frac{n_2}{2}\right)} \left(\frac{n_1}{n_2}\right)^{n_1/2} \frac{f^{n_1/2 - 1}}{\left(1 + \dfrac{n_1}{n_2}f\right)^{(n_1+n_2)/2}}, \quad f > 0\]

Proof: Apply the change-of-variable \(f = \dfrac{un_2}{vn_1}\) to the joint density of \(U\) and \(V\), then integrate out \(v\) using the substitution \(w = v\!\left(1 + \dfrac{n_1}{n_2}f\right)/2\). \(\blacksquare\)

9.3.1 Theorem 13 – Ratio of Sample Variances

If \(S_1^2\) and \(S_2^2\) are the variances of independent random samples of sizes \(n_1\) and \(n_2\) from \(N(\mu_1, \sigma_1^2)\) and \(N(\mu_2, \sigma_2^2)\) respectively, then:

\[F = \frac{S_1^2 / \sigma_1^2}{S_2^2 / \sigma_2^2} \sim F(n_1-1,\; n_2-1)\]

Proof: By Theorem 9, \(\dfrac{(n_1-1)S_1^2}{\sigma_1^2} \sim \chi^2_{n_1-1}\) and \(\dfrac{(n_2-1)S_2^2}{\sigma_2^2} \sim \chi^2_{n_2-1}\), independently. Substituting into Theorem 12 gives the result. \(\blacksquare\)

The F-table gives critical values \(f_\alpha(n_1, n_2)\) such that \(P(F > f_\alpha) = \alpha\) for specified degrees of freedom.

10.1.1 Case \(n = 2\)

\[y_1 = x_1,\; y_2 = x_2 \quad \text{when } x_1 < x_2\] \[y_1 = x_2,\; y_2 = x_1 \quad \text{when } x_2 < x_1\]

10.1.2 Case \(n = 3\)

10.3.1 Smallest Value \(Y_1\)

\[g_1(y_1) = n\,f(y_1)\left[\int_{y_1}^{\infty} f(x)\,dx\right]^{n-1}\]

10.3.2 Largest Value \(Y_n\)

\[g_n(y_n) = n\,f(y_n)\left[\int_{-\infty}^{y_n} f(x)\,dx\right]^{n-1}\]

10.3.3 Sample Median (odd \(n = 2m+1\))

The sample median \(\tilde{X} = Y_{m+1}\), with sampling distribution:

\[h(\tilde{x}) = \frac{(2m+1)!}{m!\,m!} \left[\int_{-\infty}^{\tilde{x}} f(x)\,dx\right]^{m} f(\tilde{x}) \left[\int_{\tilde{x}}^{\infty} f(x)\,dx\right]^{m}\]

For samples of size \(n = 2m\), the median is defined as \(\dfrac{Y_m + Y_{m+1}}{2}\).

10.4.1 Distribution of \(Y_1\) (Minimum)

\[\int_{y_1}^{\infty} f(x)\,dx = \int_{y_1}^{\infty} \frac{1}{\theta}e^{-x/\theta}\,dx = e^{-y_1/\theta}\]

Substituting into the formula for \(g_1\):

\[g_1(y_1) = n \cdot \frac{1}{\theta}e^{-y_1/\theta} \cdot \left(e^{-y_1/\theta}\right)^{n-1} = \frac{n}{\theta}e^{-ny_1/\theta}\]

\[\boxed{g_1(y_1) = \frac{n}{\theta}\,e^{-ny_1/\theta}, \quad y_1 > 0}\]

This is an exponential distribution with parameter \(\theta/n\).

10.4.2 Distribution of \(Y_n\) (Maximum)

\[\int_{0}^{y_n} f(x)\,dx = 1 - e^{-y_n/\theta}\]

Substituting into the formula for \(g_n\):

\[g_n(y_n) = n \cdot \frac{1}{\theta}e^{-y_n/\theta} \cdot \left(1 - e^{-y_n/\theta}\right)^{n-1}\]

\[\boxed{g_n(y_n) = \frac{n}{\theta}\,e^{-y_n/\theta}\!\left(1 - e^{-y_n/\theta}\right)^{n-1}, \quad y_n > 0}\]

10.4.3 Distribution of the Median (\(n = 2m+1\))

Computing the required integrals:

\[\int_{-\infty}^{\tilde{x}} f(x)\,dx = 1 - e^{-\tilde{x}/\theta}, \qquad \int_{\tilde{x}}^{\infty} f(x)\,dx = e^{-\tilde{x}/\theta}\]

Substituting:

\[h(\tilde{x}) = \frac{(2m+1)!}{m!\,m!} \cdot \left(1 - e^{-\tilde{x}/\theta}\right)^m \cdot \frac{1}{\theta}e^{-\tilde{x}/\theta} \cdot \left(e^{-\tilde{x}/\theta}\right)^m\]

\[\boxed{h(\tilde{x}) = \frac{(2m+1)!}{m!\,m!}\,\frac{1}{\theta}\,e^{-(m+1)\tilde{x}/\theta}\!\left(1 - e^{-\tilde{x}/\theta}\right)^m, \quad \tilde{x} > 0}\]

11.1.1 Definition – Characteristic Function

If \(X\) is a random variable, its characteristic function is \(\varphi_X : \mathbb{R} \to \mathbb{C}\), defined as:

\[\varphi_X(t) = E\!\left(e^{itX}\right) = \begin{cases} \displaystyle\sum_x e^{itx} P(X = x) & \text{if } X \text{ is discrete} \\[8pt] \displaystyle\int_{-\infty}^{\infty} e^{itx} f(x)\,dx & \text{if } X \text{ is continuous} \end{cases}\]

This expectation exists for all \(t\) (since \(|e^{itx}| = 1\)).

11.2.1 1. Uniqueness

\(X\) and \(Y\) have the same characteristic function if and only if they have the same distribution function.

11.2.2 2. Moment Generation

If \(E(X^n)\) exists, then the \(n^{\text{th}}\) derivative of \(\varphi_X(t)\) exists and:

\[\varphi_X^{(n)}(t) = i^n\,E\!\left(X^n e^{itX}\right)\]

In particular, at \(t = 0\):

\[E(X^k) = \frac{\varphi_X^{(k)}(0)}{i^k}\]

Proof:

\[\frac{d}{dt}\varphi_X(t) = \frac{d}{dt}E\!\left(e^{itX}\right) = E\!\left(\frac{d}{dt}e^{itX}\right) = E\!\left(iX e^{itX}\right) = iE\!\left(Xe^{itX}\right)\]

\[\frac{d^2}{dt^2}\varphi_X(t) = i\frac{d}{dt}E\!\left(Xe^{itX}\right) = i\,E\!\left(iX^2 e^{itX}\right) = i^2 E\!\left(X^2 e^{itX}\right)\]

By induction, \(\varphi_X^{(n)}(t) = i^n E(X^n e^{itX})\). \(\blacksquare\)

11.2.3 3. Taylor Expansion

If \(E(X^n)\) exists:

\[\varphi_X(t) = \sum_{j=0}^{n} \frac{(it)^j}{j!} E(X^j) + o_n(t)\]

where \(o_n(t)\) satisfies \(\displaystyle\lim_{t \to 0} \dfrac{o_n(t)}{t^n} = 0\).

11.4.1 (a) Characteristic Function

\[\varphi_X(t) = E\!\left(e^{itX}\right) = \sum_{x=0}^{n} e^{itx}\binom{n}{x}p^x q^{n-x}\]

\[= \sum_{x=0}^{n} \binom{n}{x}(pe^{it})^x q^{n-x} = \left(q + pe^{it}\right)^n\]

by the Binomial Theorem with \(a = q\) and \(b = pe^{it}\).

\[\boxed{\varphi_X(t) = \left(q + pe^{it}\right)^n}\]

11.4.2 (b) Mean and Variance

First derivative:

\[\varphi_X'(t) = n\left(q + pe^{it}\right)^{n-1} \cdot pi e^{it}\]

At \(t = 0\), \(q + p = 1\):

\[\varphi_X'(0) = npi \implies E(X) = \frac{npi}{i} = np\]

Second derivative (using the product rule):

\[\varphi_X''(t) = n(n-1)\left(q + pe^{it}\right)^{n-2}(pie^{it})^2 + n\left(q + pe^{it}\right)^{n-1} \cdot pi^2 e^{it}\]

At \(t = 0\):

\[\varphi_X''(0) = n(n-1)p^2 i^2 + npi^2 = -n(n-1)p^2 - np\]

Variance:

\[\text{Var}(X) = \frac{\varphi_X''(0)}{i^2} - [E(X)]^2 = \frac{-n(n-1)p^2 - np}{-1} - n^2p^2\]

\[= n(n-1)p^2 + np - n^2p^2 = np - np^2 = np(1-p) = npq\]

\[\boxed{E(X) = np, \qquad \text{Var}(X) = npq}\]

11.5.1 (a) Characteristic Function

\[\varphi_X(t) = \int_0^{\infty} e^{itx} \cdot \frac{1}{\theta} e^{-x/\theta}\,dx = \frac{1}{\theta}\int_0^{\infty} e^{-x(1/\theta - it)}\,dx\]

\[= \frac{1}{\theta} \cdot \frac{1}{1/\theta - it} = \frac{1}{1 - i\theta t}\]

\[\boxed{\varphi_X(t) = \frac{1}{1 - i\theta t}}\]

11.5.2 (b) Mean and Variance

First derivative:

\[\varphi_X'(t) = \frac{i\theta}{(1-i\theta t)^2}\]

At \(t = 0\): \(\varphi_X'(0) = i\theta\), so:

\[E(X) = \frac{i\theta}{i} = \theta\]

Second derivative:

\[\varphi_X''(t) = \frac{2(i\theta)^2}{(1-i\theta t)^3} = \frac{-2\theta^2}{(1-i\theta t)^3}\]

At \(t = 0\): \(\varphi_X''(0) = -2\theta^2\), so:

\[\text{Var}(X) = \frac{-2\theta^2}{i^2} - \theta^2 = 2\theta^2 - \theta^2 = \theta^2\]

\[\boxed{E(X) = \theta, \qquad \text{Var}(X) = \theta^2}\]