1.1.1 Example 1

Two caplets are selected at random from a bottle containing three aspirins, two sedatives, and four laxative caplets.

If \(X\) and \(Y\) are, respectively, the numbers of aspirin and sedative caplets included among the two caplets drawn, find the probabilities associated with all possible pairs of values of \(X\) and \(Y\).

Solution:

The possible pairs are \((x, y)\) where \(x \in \{0, 1, 2\}\) and \(y \in \{0, 1, 2\}\), subject to \(x + y \leq 2\).

For example, the probability associated with \((1, 0)\): we want one aspirin, zero sedatives, and hence one laxative. The number of ways is \(\binom{3}{1}\binom{2}{0}\binom{4}{1}\) and the total number of ways to select two caplets from nine is \(\binom{9}{2}\).

Continuing this way, we obtain:

It is generally preferable to represent such probabilities by means of a formula — a function \(f(x, y)\) defined for any pair of values within the range of \(X\) and \(Y\).

1.1.2 Definition 1 – Joint Probability Distribution Function (Discrete)

If \(X\) and \(Y\) are discrete random variables, the function given by \(f(x, y) = P(X = x, Y = y)\) for each pair of values \((x, y)\) within the range of \(X\) and \(Y\) is called the joint probability distribution function of \(X\) and \(Y\).

1.1.3 Theorem 1

A bivariate function can serve as the joint probability distribution function of a pair of discrete random variables \(X\) and \(Y\) if and only if its values \(f(x, y)\) satisfy the conditions:

1. \(f(x, y) \geq 0\), for each pair of values \((x, y)\) within its domain.
2. \(\displaystyle\sum_x \sum_y f(x, y) = 1\), where the double summation extends over all possible pairs within the domain.

1.1.4 Example 2

Determine the value of \(k\) for which the function \(f(x, y) = k(x + y)\) for \(x = 0, 1, 2\) and \(y = 0, 1, 2\) can serve as a joint probability distribution function.

Solution:

Substituting the various values of \(x\) and \(y\) and applying Theorem 1, we get:

\[\sum_x \sum_y k(x + y) = 1\]

Solving for \(k\) gives the appropriate value.

1.2.1 Definition 2 – Joint CDF (Discrete)

If \(X\) and \(Y\) are discrete random variables, the function given by

\[F(x, y) = P(X \leq x, Y \leq y) = \sum_{s \leq x} \sum_{t \leq y} f(s, t)\]

for \(-\infty < x, y < \infty\), where \(f(s, t)\) is the value of the joint probability distribution of \(X\) and \(Y\) at \((s, t)\), is called the joint distribution function or the joint Cumulative Distribution Function (CDF) of \(X\) and \(Y\).

1.2.2 Example 3

With reference to Example 1, find \(F(1, 1)\).

Solution:

As in the univariate case, the joint distribution function is defined for all real numbers. For example, \(F(1, 1) = \sum_{x \leq 1} \sum_{y \leq 1} f(x, y)\).

1.3.1 Definition 3 – Joint PDF (Continuous)

A bivariate function with values \(f(x, y)\), defined over the \(xy\)-plane, is called a joint probability density function of a continuous random variable \(X\) and \(Y\) if and only if

\[P[(X, Y) \in A] = \iint_A f(x, y) \, dx \, dy\]

for any region \(A\) in the \(xy\)-plane.

1.3.2 Theorem 2

A bivariate function can serve as a joint probability density function of a pair of continuous random variables \(X\) and \(Y\) if and only if its values \(f(x, y)\) satisfy the conditions:

1. \(f(x, y) \geq 0\) for all \((x, y)\)
2. \(\displaystyle\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} f(x, y) \, dx \, dy = 1\)

1.3.3 Example 4

Given the joint probability density function \(f(x, y)\) of two random variables \(X\) and \(Y\), find \(P[(X, Y) \in A]\) where \(A\) is a given region.

Solution:

\[P[(X, Y) \in A] = \iint_A f(x, y) \, dx \, dy\]

1.4.1 Definition 4 – Joint CDF (Continuous)

If \(X\) and \(Y\) are continuous random variables, the function given by

\[F(x, y) = \int_{-\infty}^{x}\int_{-\infty}^{y} f(s, t) \, dt \, ds, \quad -\infty < x, y < \infty\]

where \(f(s, t)\) is the value of the joint probability density of \(X\) and \(Y\) at \((s, t)\), is called the joint Cumulative Density Function of \(X\) and \(Y\).

Analogous to the univariate case, partial differentiation yields:

\[\frac{\partial^2 F(x, y)}{\partial x \, \partial y} = f(x, y)\]

wherever these partial derivatives exist.

1.4.2 Example 5

If the joint probability density of \(X\) and \(Y\) is given by \(f(x, y)\), find the cumulative density function.

Solution:

\[F(x, y) = \int_{-\infty}^{x}\int_{-\infty}^{y} f(s, t) \, dt \, ds\]

1.4.3 Example 6

Find the joint probability density of the two random variables \(X\) and \(Y\) whose joint cumulative density function is given. Also use the joint probability density function to determine the required probability.

Solution:

1. Since \(f(x, y) = \dfrac{\partial^2 F(x,y)}{\partial x\, \partial y}\), partial differentiation yields the joint PDF.

2. The probability is evaluated by integrating \(f(x,y)\) over the relevant region.

For two random variables, the joint probability density is geometrically a surface, and the probability is given by the volume under this surface over the region \(A\).

1.5.1 Example 7

If the joint probability distribution of three random variables \(X\), \(Y\), and \(Z\) is given, find the specified probability.

Solution:

\[P(X = x, Y = y, Z = z) = \sum_{x}\sum_{y}\sum_{z} f(x, y, z)\]

In the continuous case, probabilities are obtained by integrating the joint probability density, and the joint CDF is given by

\[F(x_1, \ldots, x_n) = \int_{-\infty}^{x_1} \cdots \int_{-\infty}^{x_n} f(t_1, \ldots, t_n) \, dt_n \cdots dt_1\]

1.5.2 Example 8

If the trivariate probability density of \((X, Y, Z)\) is given by \(f(x, y, z)\), find the probability over a specified region \(A\).

Solution:

\[P[(X, Y, Z) \in A] = \iiint_A f(x, y, z) \, dz \, dy \, dx\]

##Exercises

1. Given the joint probability distribution table of \(X\) and \(Y\) (with rows \(y = 0, 1, 2, 3\) and columns \(x = 0, 1, 2\)), find the specified probabilities.

2. If the joint probability density of \(X\) and \(Y\) is given, find the value of \(c\), then find:

   1. \(P(X < 1, Y < 1)\)
   2. \(P(X + Y < 1)\)
   3. \(P(X > Y)\)

3. Determine \(k\) so that a given function can serve as a joint probability density function.

4. Find \(F(x, y)\) if the joint probability distribution of \(X\) and \(Y\) is given.

5. Find \(k\) if a given bivariate function can serve as a joint probability density.

6. Verify the joint distribution function of a given example.

7. If a trivariate probability density \(f(x_1, x_2, x_3)\) is given, find the specified probability.

8. A certain college gives attitude tests in the sciences and humanities. If \(X\) and \(Y\) are the proportions of correct answers in the two subjects, find the probabilities that a student will get:

   1. Less than \(\frac{1}{2}\) on both tests.
   2. More than \(\frac{1}{2}\) on the science test and less than \(\frac{1}{4}\) on the humanities test.

9. Suppose \(P\), the price of a commodity (in dollars), and \(S\), its total sales (in units), are random variables with a given joint probability density. Find the probabilities that:

   1. The price will be less than 30 cents and sales will exceed 10,000 units.
   2. The price will be between 25 cents and 30 cents and sales will be less than 6,000 units.

2.1.1 Definition 5 – Marginal Distribution (Discrete)

If \(X\) and \(Y\) are discrete random variables and \(f(x, y)\) is the value of their joint probability distribution at \((x, y)\), the function given by

\[g(x) = \sum_{y} f(x, y), \quad \text{for each } x \text{ within the range of } X\]

is called the marginal probability distribution function of \(X\).

Correspondingly, the function given by

\[h(y) = \sum_{x} f(x, y), \quad \text{for each } y \text{ within the range of } Y\]

is called the marginal probability distribution function of \(Y\).

2.1.2 Example 9

The joint probability distribution function of random variables \(X\) and \(Y\) is given in the table below (\(x, y \in \{1, 2, 3\}\)). Determine the marginal probability distribution functions of \(X\) and \(Y\).

Solution:

1. The marginal probability distribution function of \(X\): \(g(x) = \sum_y f(x, y)\)

2. The marginal probability distribution function of \(Y\): \(h(y) = \sum_x f(x, y)\)

2.1.3 Definition 6 – Marginal Density (Continuous)

If \(X\) and \(Y\) are continuous random variables and \(f(x, y)\) is the value of their joint probability density at \((x, y)\), the function given by

\[g(x) = \int_{-\infty}^{\infty} f(x, y) \, dy\]

is called the marginal probability density function of \(X\).

Correspondingly,

\[h(y) = \int_{-\infty}^{\infty} f(x, y) \, dx\]

is called the marginal probability density function of \(Y\).

2.1.4 Example 10

Given the joint probability density \(f(x, y)\), find the marginal probability density functions of \(X\) and \(Y\).

Solution:

\[g(x) = \int_{-\infty}^{\infty} f(x, y) \, dy, \qquad h(y) = \int_{-\infty}^{\infty} f(x, y) \, dx\]

2.2.1 Example 11

Considering the trivariate probability density \(f(x_1, x_2, x_3)\), find:

1. The joint marginal probability density function of \(X_1\) and \(X_2\).
2. The marginal probability density function of \(X_1\) alone.

Solution:

1. \(f_{12}(x_1, x_2) = \int_{-\infty}^{\infty} f(x_1, x_2, x_3) \, dx_3\)

2. \(f_1(x_1) = \int_{-\infty}^{\infty} f_{12}(x_1, x_2) \, dx_2\)

2.3.1 Definition 7 – Conditional Distribution (Discrete)

If \(f(x, y)\) is the value of the joint probability distribution of the discrete random variables \(X\) and \(Y\) at \((x, y)\), and \(h(y)\) is the value of the marginal distribution of \(Y\) at \(y\), the function given by

\[f(x \mid y) = \frac{f(x, y)}{h(y)}, \quad \text{for each } x \text{ within the range of } X\]

is called the conditional distribution of \(X\) given \(Y = y\).

Correspondingly, if \(g(x)\) is the value of the marginal distribution of \(X\) at \(x\):

\[f(y \mid x) = \frac{f(x, y)}{g(x)}, \quad \text{for each } y \text{ within the range of } Y\]

is called the conditional distribution of \(Y\) given \(X = x\).

2.3.2 Definition 8 – Conditional Density (Continuous)

If \(f(x, y)\) is the value of the joint density of continuous random variables \(X\) and \(Y\) at \((x, y)\), and \(h(y)\) is the value of the marginal density of \(Y\) at \(y\), the function given by

\[f(x \mid y) = \frac{f(x, y)}{h(y)}, \quad \text{for } h(y) > 0\]

is called the conditional density of \(X\) given \(Y = y\).

Correspondingly:

\[f(y \mid x) = \frac{f(x, y)}{g(x)}, \quad \text{for } g(x) > 0\]

is called the conditional density of \(Y\) given \(X = x\).

2.3.3 Example 12

Given the joint probability density \(f(x, y)\), find the marginal densities of \(X\) and \(Y\) and the conditional density of \(X\) given \(Y = y\).

Solution:

\[g(x) = \int f(x, y) \, dy, \qquad h(y) = \int f(x, y) \, dx\]

\[f(x \mid y) = \frac{f(x, y)}{h(y)}\]

2.3.4 Example 13

With reference to Example 1, find:

1. The conditional distribution of \(X\) given \(Y = 1\).
2. The conditional distribution of \(Y\) given \(X = 0\).

Solution:

The joint distribution together with marginal totals:

1. Conditional distribution of \(X\) given \(Y = 1\): \(\quad f(x \mid 1) = \dfrac{f(x, 1)}{h(1)}\)

2. Conditional distribution of \(Y\) given \(X = 0\): \(\quad f(y \mid 0) = \dfrac{f(0, y)}{g(0)}\)

2.4.1 Definition 9 – Independence (Discrete)

If \(f(x_1, x_2, \ldots, x_n)\) is the value of the joint probability distribution of \(n\) discrete random variables \(X_1, \ldots, X_n\) at \((x_1, \ldots, x_n)\), and \(f_i(x_i)\) is the value of the marginal distribution of \(X_i\), then the \(n\) random variables are independent if and only if

\[f(x_1, \ldots, x_n) = f_1(x_1) \cdot f_2(x_2) \cdots f_n(x_n)\]

for all \((x_1, \ldots, x_n)\) within their range.

2.4.2 Definition 10 – Independence (Continuous)

If \(f(x_1, x_2, \ldots, x_n)\) is the value of the joint probability density function of \(n\) continuous random variables \(X_1, \ldots, X_n\) at \((x_1, \ldots, x_n)\), and \(f_i(x_i)\) is the value of the marginal density of \(X_i\), then the \(n\) random variables are independent if and only if

\[f(x_1, \ldots, x_n) = f_1(x_1) \cdot f_2(x_2) \cdots f_n(x_n)\]

for all \((x_1, \ldots, x_n)\) within their range.

2.4.3 Example 14

If the joint probability density of \(X\) and \(Y\) is given, find:

1. The marginal density of \(X\).
2. The marginal density of \(Y\).
3. Whether the two random variables are independent.

Solution:

Compute \(g(x) = \int f(x,y)\,dy\) and \(h(y) = \int f(x,y)\,dx\). Then check whether \(f(x,y) = g(x) \cdot h(y)\).

If \(f(x,y) \neq g(x) \cdot h(y)\), the two random variables are not independent.

2.4.4 Example 15

With reference to Example 1, determine whether \(X\) and \(Y\) are independent.

Solution:

Using the marginal distributions obtained previously, check whether \(f(x,y) = g(x) \cdot h(y)\) for all pairs. If any pair fails, \(X\) and \(Y\) are not independent.

2.4.5 Example 16

Considering independent flips of a balanced coin, let \(X_i\) be the number of heads (0 or 1) obtained on the \(i\)-th flip. Find the joint probability distribution of these \(n\) random variables.

Solution:

Since each \(X_i\) has the distribution \(P(X_i = x) = \frac{1}{2}\) for \(x = 0, 1\), and the random variables are independent, their joint probability distribution is:

\[f(x_1, \ldots, x_n) = \prod_{i=1}^{n} \frac{1}{2} = \left(\frac{1}{2}\right)^n, \quad x_i \in \{0, 1\}\]

2.4.6 Example 17

Given independent random variables \(X_1\), \(X_2\), \(X_3\) with probability densities \(f_1(x_1)\), \(f_2(x_2)\), \(f_3(x_3)\), find their joint probability density and use it to evaluate a specified probability.

Solution:

The joint probability density function is:

\[f(x_1, x_2, x_3) = f_1(x_1) \cdot f_2(x_2) \cdot f_3(x_3)\]

3.1.1 Definition 1 – Bivariate Normal Distribution

A pair of random variables \(X\) and \(Y\) have a bivariate normal distribution and are referred to as jointly normally distributed random variables, if and only if their joint probability density is given by:

\[f(x, y) = \frac{1}{2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}} \exp\!\left\{-\frac{1}{2(1-\rho^2)}\left[\frac{(x-\mu_1)^2}{\sigma_1^2} - \frac{2\rho(x-\mu_1)(y-\mu_2)}{\sigma_1\sigma_2} + \frac{(y-\mu_2)^2}{\sigma_2^2}\right]\right\}\]

where \(-1 < \rho < 1\).

To study this joint distribution, we first show that the parameters \(\mu_1, \mu_2\) are the means and \(\sigma_1, \sigma_2\) are the standard deviations of \(X\) and \(Y\) respectively.

Integrating on \(y\) from \(-\infty\) to \(\infty\) gives the marginal density of \(X\). After completing the square and simplifying:

\[g(x) = \frac{1}{\sigma_1\sqrt{2\pi}} e^{-\frac{(x-\mu_1)^2}{2\sigma_1^2}}\]

which is a normal distribution with mean \(\mu_1\) and standard deviation \(\sigma_1\). By symmetry, the marginal density of \(Y\) is a normal distribution with mean \(\mu_2\) and standard deviation \(\sigma_2\).

The parameter \(\rho\) is called the correlation coefficient, and it can be shown that \(-1 < \rho < 1\). It measures how the two random variables \(X\) and \(Y\) vary together.

3.2.1 Theorem 1 – Conditional Distributions of the Bivariate Normal

If \(X\) and \(Y\) have a bivariate normal distribution, the conditional density of \(Y\) given \(X = x\) is a normal distribution with:

• Mean: \(\mu_2 + \rho\dfrac{\sigma_2}{\sigma_1}(x - \mu_1)\)
• Variance: \(\sigma_2^2(1 - \rho^2)\)

And the conditional density of \(X\) given \(Y = y\) is a normal distribution with:

• Mean: \(\mu_1 + \rho\dfrac{\sigma_1}{\sigma_2}(y - \mu_2)\)
• Variance: \(\sigma_1^2(1 - \rho^2)\)

Proof:

Writing \(f(y \mid x) = \dfrac{f(x, y)}{g(x)}\) and simplifying the exponent, one arrives at a normal density with the stated mean and variance. The corresponding result for the conditional density of \(X\) given \(Y = y\) follows by symmetry.

3.3.1 Theorem 2 – Independence in the Bivariate Normal

If two random variables have a bivariate normal distribution, they are independent if and only if \(\rho = 0\).

Proof: (exercise)

Remark: When \(\rho = 0\), the random variables are said to be uncorrelated. Note that the marginal distributions may both be normal without the joint distribution being bivariate normal — the converse does not necessarily hold.

3.4.1 Example 1

In a certain population of married couples, the height \(X\) (in meters) of the husband and the height \(Y\) of the wife have a bivariate normal distribution with given parameters.

1. Find the mean and the variance of the height for the husbands and wives.
2. Find the expected height and variance of the husband whose wife is expected to be 1.55 m tall.

Solution:

1. Comparing the given density with the general formula, we identify:

\[\mu_1 = \ldots, \quad \sigma_1 = \ldots, \quad \mu_2 = \ldots, \quad \sigma_2 = \ldots, \quad \rho = \ldots\]

2. By Theorem 1, the conditional mean of \(X\) given \(Y = 1.55\) is:

\[E(X \mid Y = 1.55) = \mu_1 + \rho\frac{\sigma_1}{\sigma_2}(1.55 - \mu_2)\]

The conditional variance of \(X\) given \(Y = y\) is:

\[\text{Var}(X \mid Y = y) = \sigma_1^2(1 - \rho^2)\]

3.4.2 Example 2

Let \(X\) and \(Y\) have a bivariate normal distribution with parameters \(\mu_1, \mu_2, \sigma_1, \sigma_2, \rho\).

Find:

1. \(P(\ldots)\) [2 marks]
2. \(P(\ldots \mid X = 74)\) [6 marks]

Solution:

1. Compute using the standard normal CDF.

2. The conditional distribution of \(Y\) given \(X = 74\) is normal with:

\[\text{Mean} = \mu_2 + \rho\frac{\sigma_2}{\sigma_1}(74 - \mu_1), \qquad \text{Variance} = \sigma_2^2(1 - \rho^2)\]

Hence, standardize and use the standard normal table to find the required probability.