G1
Annabelle
2024-09-22
Random Variables and Distribution Functions
CDF Definition: The cumulative distribution function (CDF),
\(F_X(x) = P(X \leq x)\)
gives the probability that the random variable \(X\) takes takes a value less than or equal to some real number \(x\).
A random variable is a real-valued function from the probability space \(X: \omega \to \mathbb{R}\)
Generally speaking, we shall use capital letters near the end of the alphabet, e.g., X, Y, Z for random variables. The range S of a random variable is sometimes called the state space
Distribution Functions
Definition 7.5. A (cumulative) distribution function of a random variable X is defined by
\(F_X(x) = P\{ \omega \in \Omega \mid X(\omega) \leq x \}\)
Recall that with quantitative observations, we called the analogous notion the empirical cumulative distribution function. Using the abbreviated notation above, we shall typically write the less explicit expression
\(F_X(x) = P\{X ≤ x\}\)
for the distribution function
Example 7.8. To give the cumulative distribution function for X, the sum of the values for two rolls of a die, we start with the table
prob <- c(1/36, 2/36, 3/36, 4/46, 5/36, 6/36, 5/36, 4/36, 3/36, 2/36, 1/36)
x <- c(2, 3, 4, 5,6,7,8,9,10, 11,12)
X.distribution <- matrix(c(x, prob), nrow = 2, byrow = TRUE)
rownames(X.distribution) <- c("x", "P{X = x}")
X.distribution## [,1] [,2] [,3] [,4] [,5] [,6]
## x 2.00000000 3.00000000 4.00000000 5.00000000 6.0000000 7.0000000
## P{X = x} 0.02777778 0.05555556 0.08333333 0.08695652 0.1388889 0.1666667
## [,7] [,8] [,9] [,10] [,11]
## x 8.0000000 9.0000000 10.00000000 11.00000000 12.00000000
## P{X = x} 0.1388889 0.1111111 0.08333333 0.05555556 0.02777778# Compute the cumulative probabilities
cum_prob <- cumsum(prob)
# Plot the CDF
plot(x, cum_prob, type = "s", xlab = "X", ylab = "F_X(x)", main = "Cumulative Distribution Function")
points(x, cum_prob, pch = 19)
If we look at the graph of this cumulative distribution function, we see that it is constant in between the possible values for X and that the jump size at \(x\) is equal to \(P\{X = x\}\).
In this example, \(P\{X = 5\} = \frac{4}{36}\), the size of the jump at x = 5.
In addition,
\(F_X(5) - F_X(2) = P\{2 < X \leq 5\} = P\{X = 3\} + P\{X = 4\} + P\{X = 5\}\)
\(= \sum_{2 < x \leq 5}{} P\{X = x\}\)
\(= \frac{2}{36} + \frac{3}{36} + \frac{4}{36}\)
We shall call a random variable discrete if it has a finite or countably infinite state space.
Thus, we have in these cases that:
\(P\{a < X \leq b\} = \sum_{a < x \leq b}{} P\{X = x\}\)
Exercise 7.9. Let X be the number of heads on three independent flips of a biased coin that turns ups heads with probability p. Give the cumulative distribution function FX for X.
To solve this problem, we need to find the cumulative distribution function (CDF) of \(X\), the number of heads in three independent flips of a biased coin that turns heads with probability \(p\).
Step 1: Possible Values of \(X\) Since we are flipping the coin three times, the number of heads \(X\) can take values from 0 to 3 i.e.\(X \in \{{0,1,2,3\}\)
Step 2: Probability Mass Function (PMF) of \(X\) We can model this scenario as a binomial distribution because we have independent trials (flips), each of which results in a head (success) with probability \(p\). The binomial probability mass function for \(X\) is: \(P(X = k) = \binom {k}{3} p^k(1 - p)^{3 - k}\) for k = 0, 1, 2, 3
Step 3: Cumulative Distribution Function (CDF) of X
- For \(x = 0\):
- \(F_X(0) = P(X \leq 0) = P(X = 0) = {(1-p)}^3\)
- For \(x = 1\):
- \(F_X(1) = P(X \leq 1) = P(X = 0) + P(X = 1) = {(1-p)}^3 + 3p{(1-p)}^2\)
- for \(x = 2\):
- \(F_X(2) = P(X \leq 2) = P(X = 0) + P(X = 1) + P(X = 2) = {(1-p)}^3 + 3p{(1-p)}^2 + 3p^2(1 - p)\)
- For \(x = 3\):
- \(F_X(3) = P(X \leq 3) = 1\)
Thus:
\(F_X(x) = \begin{cases} 0, & \text{if } x = 0 \\ (1-p)^3, & \text{if } x = 1 \\ (1-p)^3 + 3p(1-p)^2, & \text{if } x = 2 \\ (1-p)^3 + 3p(1-p)^2 + 3p^2(1-p), & \text{if } x = 3 \\ 1, & \text{if } x \geq 4 \end{cases}\)
Exercise 7.10. From the example in the section Basics of Probability, we know that
hearts <- c(0:3)
f <- choose(13, hearts) * choose(39, 3 - hearts) / choose(52, 3)
(F <- cumsum(f))## [1] 0.4135294 0.8494118 0.9870588 1.0000000plot(hearts, F, ylim = c(0, 1), type = "s")
\(F_X(x) = \begin{cases} 0, & \text{if } x < 0 \\ 0.41353, & \text{if } 0 \leq x < 1 \\ 0.84941, & \text{if } 1 \leq x < 2 \\ 0.98706, & \text{if } 2 \leq x < 3 \\ 1, & \text{if } 3 \leq x \end{cases}\)
Properties of the Distribution Function
A distribution function FX has the property that it starts at 0, ends at 1 and does not decrease with increasing values of x. This is the content of the next exercise
- \(lim_{x \to - \infty} F_x(x) = 0\)
This means that as \(x \to - \infty\), the probability \(P(X \leq x)\) tends to 0
Proof
Let \(\{x_n\}\) be a decreasing sequence such that \(x_n \to - \infty\).
This means:
\(x_1 > x_2 > ... > x_n > ...\)
Since \(\{X \leq x_1\} \supseteq \{X \leq x_1\} \supseteq ...\), the events form a nested sequence.
By the monotonical property of probabilities, this implies:
\(P(X \leq x_1) \geq (X \leq x_2) \geq \cdots\)
Note:
\(\omega\): This represents an outcome in the sample space \(\Omega\). Every random variable \(X\) assigns a real number to each \(\omega\).
\(X(\omega)\) is the value that random variable \(X\) takes for a specific outcome \(\omega\)
Sequence \(\{x_n\}\):
In the proof, we’re considering a sequence of numbers \(x_1, x_2, x_3, \cdots, x_n, \cdots\) that are getting smaller and smaller
\(X(\omega) > x_n\):
This means that for a particular outcome \(\omega\), the value of the random variable \(X\) for that outcome is eventually greater that \(x_n\) for large enough \(n\).
As the sequence \(x_n\) decreases towards negative infinity, eventually, for all outcomes \(\omega\), \(X(\omega)\), will be greater than \(x_n\), meaning that the probability of \(X\) being less than or equal to \(x_n\) (i.e., (\(P(X \leq x_n))\)) becomes smaller and smaller.
Limit as \(n \to \infty\), \(P(X \leq x_n) = 0\)
As \(n\) increases and \(x_n \to -\infty\), the event \(\{X \leq x_n\}\) becomes less and less likely because \(x_n\) is becoming very negative, and the probability that \(X\) takes a value less than \(x_n\) approaches zero. In other words, for very negative \(x_n\), there are no outcomes \(omega\) such that \(X(\omega) \leq x_n\), so \(P(X \leq x_n) \to 0\)
For each outcome \(\omega\), eventually, for some n, \((\omega) > x_n\), and hence \(\omega \notin \{X \leq x_n\}\) for large n.
and consequently no outcome \(\omega\) is in all of the events \(\{X \leq x_n\}\) and
\(\bigcap_{n=1}^{\infty} \{X \leq x_n\} = \emptyset\)
- \(lim_{x \to \infty} F_x(x) = 1\)
Let \(\{x_n\}\) be an increasing sequence such that \(x_n \to \infty\). This means:
\(x_1 < x_2 < \cdots < x_n < \cdots\)
Since \(\{X \leq x_1\} \subset \{X \leq x_2\} \subset \cdots\), the events forms an increasing nested sequence:
Thus,
\(P\{X \leq x_1\} \leq \{X \leq x_2\} \leq \cdots\)
For each outcome \(\omega\), eventually, for some n, \(X(\omega) \leq x_n\) and
\(\bigcup_{n=1}^{\infty}\{X \leq x_n\} = \Omega\)
- \(F_X\) is nondecreaing
Let \(x_1 < x_2\), then then \(\{X \leq x_1\} \subset \{X \leq x_2\}\) and by the monotonic rule for probabilities
\(P\{X \leq x_1\} \leq P\{X \leq x_2\}\)
or can be written as \(F_x(x_1) \leq F_x(x2)\)
Continuous Random Variables
Definition 7.15. A continuous random variable has a cumulative distribution function \(F_X\) that is differential.
So, distribution functions for continuous random variables increase smoothly. To show how this can occur, we will develop an example of a continuous random variable
Example 7.16. Consider a dartboard having unit radius. Assume that the dart lands randomly uniformly on the dartboard. Let X be the distance from the center.
For \(x \in [0, 1]\)
\(F_X(x) = P\{X \leq x\} = \frac{\text{area inside circle of radius}}{\text{area of circle}} = \frac{\pi x^2}{\pi} = x^2\)
Thus we have the distribution function: \(F_X(x) = \begin{cases} 0, & \text{if } x \leq 0 \\ x^2, & \text{if } 0 < x \leq \\ 1, & \text{if } x > 1 \end{cases}\)
Exercise 7.19. An exponential random variable X has cumulative distribution function
\(F_X(x) = P\{X \leq x\} = \begin{cases} 0, &\text{x } \leq 0 \\ 1-exp(-\lambda x), & \text{if } x > 0 \end{cases}\)
for some \(\lambda > 0\).
Show that \(F_X\) has the properties of a distribution function
F <- expression(1 - exp(-lambda * x))We can then evaluate \(F_X(3)\) and \(F_X(1)\) with \(\lambda = 2\) as follows.
x <- c(10, 30); lambda <- 1 / 10
(Feval <- eval(F))## [1] 0.6321206 0.9502129Feval[2] - Feval[1]## [1] 0.3180924The last expression gives the value for \(F_X(30) − F_X(10) = P\{10 < X ≤ 30\}\). This function is also stored in R and so its value at x can be computed in R using the command pexp(x,0.1) for λ = 1/10. Thus, we make the computation above by
pexp(30, 0.1) - pexp(10, 0.1)## [1] 0.3180924curve(pexp(x, 0.1), 0, 80)
Cumulative distribution function for an exponential random variable with \(\lambda = \frac{1}{10}\).