On Tuesday, June 18, we simulated random selection of airline pilots for promotion. We wanted to see whether there was evidence of gender bias in pilot selection.
Data simulated in R using 1,000 replications (trials) and 10,000 replications:
Example: Compare and contrast the two simulated histograms with our in-class simulation. Does the pattern change much? Did we ever observe 7 females chosen for promotion?
I used the tally function to estimate some probabilities based on the 10,000 simulated data points.
##
## 0 1 2 3 4 5 6 7 Total
## 0.0057 0.0605 0.2100 0.3295 0.2630 0.1084 0.0213 0.0016 1.0000
Example: Use the estimated probability distribution to find the probability of choosing at least 5 females for management.
The probability of choosing at least 5 females for management is what we would refer to in statistics as a p-value. A p-value is the probability of observing a sample result as or more extreme than the one we observed.
Example: Is this p-value, 0.1304, large enough for us to believe that the pilots weren't selected at random? Why or why not?
Simulations are an incredibly powerful tool for us in statistics! They let us take advantage of computing power to estimate sampling distributions.
Sampling distribution: the probability distribution of a sample statistic. The sampling distribution describes how likely a certain sample statistic is for a given sample!
That's what we've done with our airline pilot example. So far, we have:
Simulations are a natural and intutitive way to make statistical decisions. With a simulation study we aren't just relying on our imaginations or knowledge of probability theory – we can see exactly what happens with repeated random samples.
Example: Can People Distinguish Dog Food from Duck Pate?
On Homework 4, you read a study which investigated whether dog food was distinguishable from a variety of other products. The authors of the study used traditional statistical methods to analyze their data. Before we look at how to do traditional statistical tests, let's try using a simulation study.
The research objective we're interested in is:
In the study we read, 3 out of 18 participants correctly identified the dog food. To do this simulation, we need a few more pieces of information:
We can think of this experiment as a weighted coin. 20% of the time, a subject will correctly guess which is the dog food – this is like a coin with 20% probability of heads. Since we have a “weighted coin scenario”, we can use R to quickly simulate the data. With the rflip function, we can simulate flipping 18 coins (n=18) that each have a 20% chance of coming up heads (prob=0.2).
library(mosaic)
rflip(n = 18, prob = 0.2)
##
## Flipping 18 coins [ Prob(Heads) = 0.2 ] ...
##
## T H T H T T T H H T T T T T T H T T
##
## Result: 5 heads.
rflip(n = 18, prob = 0.2)
##
## Flipping 18 coins [ Prob(Heads) = 0.2 ] ...
##
## T H T T T T T T T T H H T T H T T H
##
## Result: 5 heads.
We need a way to speed this up! Using the do function, we can tell R to repeat this calculation many times.
tally function to estimate the probabilities from your sampling distribution. Compare your histogram and estimated probabilities to a neighbors'. Are they exactly the same? Are they close?sim.data <- do(50) * rflip(n = 18, prob = 0.2)
xhistogram(~heads, data = sim.data, width = 1)
tally(~heads, data = sim.data, format = "proportion")
##
## 0 1 2 3 4 5 6 7 Total
## 0.04 0.10 0.20 0.22 0.22 0.10 0.06 0.06 1.00
Let's see what happens over more trials.
sim.data2 <- do(5000) * rflip(n = 18, prob = 0.2)
xhistogram(~heads, data = sim.data2, width = 1)
tally(~heads, data = sim.data2, format = "proportion")
##
## 0 1 2 3 4 5 6 7 8 9
## 0.0156 0.0868 0.1700 0.2326 0.2124 0.1458 0.0802 0.0394 0.0130 0.0032
## 10 Total
## 0.0010 1.0000
This simulated sampling distribution represents the probabilities for each number of subjects correctly identifying the dog food (out of 18 total subjects), if they are all truly guessing.