Source files: https://github.com/djlofland/DATA606_F2019/tree/master/Homework4
Area under the curve, Part I. (4.1, p. 142) What percent of a standard normal distribution \(N(\mu=0, \sigma=1)\) is found in each region? Be sure to draw a graph.
## [1] "Exercise 1: 0.088507991437402"
ggplot(NULL, aes(c(-5,5))) +
geom_area(stat = "function", fun = dnorm, fill = "#00998a", xlim = c(-5, -1.35)) +
geom_area(stat = "function", fun = dnorm, fill = "grey80", xlim = c(-1.35, 5)) +
labs(x = "z", y = "", title="Z < -1.35") +
scale_y_continuous(breaks = NULL) +
scale_x_continuous(breaks = 1)## [1] "Exercise 2 0.0694366233333318"
ggplot(NULL, aes(c(-5,5))) +
geom_area(stat = "function", fun = dnorm, fill = "grey80", xlim = c(-5, 1.48)) +
geom_area(stat = "function", fun = dnorm, fill = "#00998a", xlim = c(1.48, 5)) +
labs(x = "z", y = "", title="Z > 1.48") +
scale_y_continuous(breaks = NULL) +
scale_x_continuous(breaks = 1)## [1] "Exercise 3 0.588614540341466"
ggplot(NULL, aes(c(-5,5))) +
geom_area(stat = "function", fun = dnorm, fill = "grey80", xlim = c(-5, -0.4)) +
geom_area(stat = "function", fun = dnorm, fill = "#00998a", xlim = c(-0.4, 1.5)) +
geom_area(stat = "function", fun = dnorm, fill = "grey80", xlim = c(1.5, 5)) +
labs(x = "z", y = "", title="-0.4 < Z < 1.5") +
scale_y_continuous(breaks = NULL) +
scale_x_continuous(breaks = 1)## [1] "Exercise 4 0.0455002638963584"
ggplot(NULL, aes(c(-5,5))) +
geom_area(stat = "function", fun = dnorm, fill = "#00998a", xlim = c(-5, -2)) +
geom_area(stat = "function", fun = dnorm, fill = "#00998a", xlim = c(2, 5)) +
labs(x = "z", y = "", title="|Z| > 2") +
scale_y_continuous(breaks = NULL) +
scale_x_continuous(breaks = 1)Triathlon times, Part I (4.4, p. 142) In triathlons, it is common for racers to be placed into age and gender groups. Friends Leo and Mary both completed the Hermosa Beach Triathlon, where Leo competed in the Men, Ages 30 - 34 group while Mary competed in the Women, Ages 25 - 29 group. Leo completed the race in 1:22:28 (4948 seconds), while Mary completed the race in 1:31:53 (5513 seconds). Obviously Leo finished faster, but they are curious about how they did within their respective groups. Can you help them? Here is some information on the performance of their groups:
Remember: a better performance corresponds to a faster finish.
\[mens: N(\mu=4313, \sigma=583)\quad womens:N(\mu=5261, \sigma=807)\]
## [1] 1.089194
## [1] 0.3122677
Within his group, Leo was over 1 SD above mean which means he took longer than the mean for his group and was in the bottom ~32% of his group. Higher times, means longer run and worse performance. While Mary took longer to run, (5261 vs 4313), compared with her group, she performed closer to mean. She was still behind the pack, but not as bad as Leo compared with his group.
This depending on how we define “ranking”. If its an absolute, then we don’t know … it depend on group sizes. If there were 10 people in Leo’s group, his absolute ranking will be higher than Mark’s even though he performed worse relative to his group. If it’s a relative % ranking, then Mark will have ranked higher since she was closer to mean with a larger percentage of athletes behind her compared with Leo.
## [1] 0.1400711
Leo finished faster than 14% of his group while 86% of his group was ahead of him.
## [1] 0.3774185
Mary finished faster than 37.7% of her group while 62.3% finished ahead of her.
The z-scores are still valid within group for non-normal distribution; however, as the distributions skew, so will the z-score distribution. As such, the portions of athletes in front and behind won’t be the same as normal since outliers drag the mean their direction, ie fewer athletes in the tail group than expecte. That said, z-score is tolerant to an extent within group. Comparision across groups will become problematic as the z-score distributions deviate from each other. Since Leo and Mary’s performance was not very close, I suspect it would take a fair bit of skew before we’d question performance against each other. If their z-scores were close, then the distribution skew might matter.
Heights of female college students Below are heights of 25 female college students.
\[ \stackrel{1}{54}, \stackrel{2}{55}, \stackrel{3}{56}, \stackrel{4}{56}, \stackrel{5}{57}, \stackrel{6}{58}, \stackrel{7}{58}, \stackrel{8}{59}, \stackrel{9}{60}, \stackrel{10}{60}, \stackrel{11}{60}, \stackrel{12}{61}, \stackrel{13}{61}, \stackrel{14}{62}, \stackrel{15}{62}, \stackrel{16}{63}, \stackrel{17}{63}, \stackrel{18}{63}, \stackrel{19}{64}, \stackrel{20}{65}, \stackrel{21}{65}, \stackrel{22}{67}, \stackrel{23}{67}, \stackrel{24}{69}, \stackrel{25}{73} \]
heights <- c(54, 55, 56,56,57,58,58,59,60,60,60,61,61,62,62,63,63,63,64,65,65,67,67,69,73)
(mu <- mean(heights))## [1] 61.52
## [1] 4.583667
# Percent Between (1)SD and (-1)SD ... compare with ~68%
pnorm(mu+sigma, mean=mu, sd=sigma) - pnorm(mu-sigma, mean=mu, sd=sigma)## [1] 0.6826895
# Percent Between (2)SD and (-2)SD ... compare with ~65%
pnorm(mu+2*sigma, mean=mu, sd=sigma) - pnorm(mu-2*sigma, mean=mu, sd=sigma)## [1] 0.9544997
# Percent Between (3)SD and (-3)SD ... compare with ~99.7%
pnorm(mu+3*sigma, mean=mu, sd=sigma) - pnorm(mu-3*sigma, mean=mu, sd=sigma)## [1] 0.9973002
The 68%, 95% and 97.5% Rule holds up quite well for this group of students.
A Q-Q plot plots the quantiles of a sample distribution against a reference distribution (Clay Ford 2015). If the plot is a straight line then we can assume they came from the same distribution. Looking at the heights, it appear like a normal distribution fits the sample data.
Defective rate. (4.14, p. 148) A machine that produces a special type of transistor (a component of computers) has a 2% defective rate. The production is considered a random process where each transistor is independent of the others.
# Geometric Distribution - How many trials to observe a "success" (ie desired outcome)
# Rule are met: each trial is independent and rate is equally distributed (ie consistent)
p <- 0.02 # defect rate
n <- 10 # trials
(prob <- ((1 - p)^(n-1) * p))## [1] 0.01667496
## [1] 0.1326196
# Geometric Distribution - How many trials to observe a "success" (ie desired outcome)
p <- 0.02 # defect rate
n <- 100 # trials
(mu <- 1/p) # mean represents the average # trials until the 2% defect rate occurs (on average)## [1] 50
## [1] 49.49747
# Geometric Distribution - How many trials to observe a "success" (ie desired outcome)
p <- 0.05 # defect rate
n <- 100 # trials
(mu <- 1/p) # mean represents the average # trials until the 5% defect rate occurs (on average)## [1] 20
## [1] 19.49359
Note with Geometric distributions, as the probability increase the time to wait (on average) decreases.
As probability increases, the time to wait (mean) decreases - this makes sense. If there is a higher error rate, it should happen more often.
We see the standard deviation also decrease. With a low error rate, there is a greater spread between events with equal probability of an error occuring (p is independent). This leads to a very high SD indicating a very broad distribution. As the error rate increases, the separation between events decreases and the conceptually, we have a narrower distribution with a much higer peak. With this shape, we would expect the SD, a measure of spread, to decrease.
Male children. While it is often assumed that the probabilities of having a boy or a girl are the same, the actual probability of having a boy is slightly higher at 0.51. Suppose a couple plans to have 3 kids.
p_b <- 0.51
p_g <- 1 - p_b
n <- 3
k <- 2
(p_2boys <- factorial(3) * p_b^k * p_g^1 / (factorial(k)*factorial(n-k)))## [1] 0.382347
| Seq | children | Probability |
|---|---|---|
| 1 | B B G | 0.51 * 0.51 * 0.49 |
| 2 | B G B | 0.51 * 0.49 * 0.51 |
| 3 | G B B | 0.49 * 0.51 * 0.51 |
\[P(\text{1}^{st}\text{ Seq or 2}^{nd}\text{ Seq or 3}^{rd}\text{ Seq})\\ =P(\text{1}^{st}\text{ Seq})+P(\text{2}^{nd}\text{ Seq})+P(\text{3}^{rd}\text{ Seq})\\ =3 * [0.51 * 0.51 * 0.49]\\ = 0.3823\text{ or }38.23\%\]
Part (a) [Exercise 18] Has a simple formula that captures the combinatorial math making the calcuations far easier. In Part (b) [Exercise 19] we’d have to write out all combinations, then multiply out the probabilities without introducing bugs and finally multiply the final combination component with the single event probability … all of that is way too much work and hassle. It’s good to understand how the formula was derived in case we have a complex problem requiring we deviate from the formula.
Serving in volleyball. (4.30, p. 162) A not-so-skilled volleyball player has a 15% chance of making the serve, which involves hitting the ball so it passes over the net on a trajectory such that it will land in the opposing team’s court. Suppose that her serves are independent of each other.
# Negative Binomial - Probability of observing kth success on nth trial.
# Rules: 1) trials must be in dependent, 2) all outcomes are success or fail
# 3) P remains consistent between trials 4) last trial MUST be a success
p <- 0.15
n <- 10
k <- 3
(prob <- factorial(n-1) * p^k * (1-p)^(n-k) / (factorial(k-1)*factorial(n-k)))## [1] 0.03895012
Since we assume independence between trials, the probability of success on any single given serve is 15%. So, on her 10th serve she has a 15% chance of success. It doesn’t matter what has come before.
In Part (b), since we assume independence between trials, the probability of success on any single given serve is 15%. So, on her 10th serve she has a 15% chance of success. It doesn’t matter what has come before.
In Part (a), we were asking what is the probability of a sequence of events happening in a specific way/order. Now we are having to account for 2 previous successful serves and the 15% probability of the 10th serve being successful.
Clay Ford, Statistical Research Consultant. 2015. Understanding Q-Q Plots. University of Virginia Library. https://data.library.virginia.edu/understanding-q-q-plots/.