You need these data to do this homework:
load(url("http://www.soc.duke.edu/~dee4/soc333data/hw7.data"))
So, you were probably thoroughly confused about type II error. Like I said, I don't care if you can't calculate type II error. I do care that you understand when Type II error occurs.
And you're in luck. R has a command that will calculate the power of a test for you. The power of the test is just 1-Type II error. Remember our example from class. If we want to calculate the probability of making a Type II error when the difference between UK and MSU is 0.5, we just need to give R one command.
power.t.test(n=,delta=,sd=,power=NULL,type=,alternative=) where n = number of observations PER group delta = the difference you are trying to test. sd = pooled standard deviation sqrt(s12 + s22) type=“two.sample” or type=“paried” for independent or paired t-test alternative=“two.sided” or alternative=“one.sided”
# Pooled sd
s = sqrt(1.152^2 + 1.094^2)
power.t.test(65, delta = 0.5, sd = s, power = NULL, type = "two.sample", alternative = "one.sided")
##
## Two-sample t test power calculation
##
## n = 65
## delta = 0.5
## sd = 1.589
## sig.level = 0.05
## power = 0.5556
## alternative = one.sided
##
## NOTE: n is number in *each* group
# Power is 0.56, Type II error rate = 1-Power = 0.44 If we want to calculate
# a 1 point difference, we are in much better luck:
power.t.test(65, delta = 1, sd = s, power = NULL, type = "two.sample", alternative = "one.sided")
##
## Two-sample t test power calculation
##
## n = 65
## delta = 1
## sd = 1.589
## sig.level = 0.05
## power = 0.9729
## alternative = one.sided
##
## NOTE: n is number in *each* group
# Type II error=1-.97=0.03 Easier to measure things farther apart. If we
# double our poll size for a 1/2 point difference:
power.t.test(130, delta = 0.5, sd = s, power = NULL, type = "two.sample", alternative = "one.sided")
##
## Two-sample t test power calculation
##
## n = 130
## delta = 0.5
## sd = 1.589
## sig.level = 0.05
## power = 0.8122
## alternative = one.sided
##
## NOTE: n is number in *each* group
# Now about a 19% Type II error rate
We also learned how to calculate whether a die is fair, and whether there is a difference in propotions. Again, there's convoluted math involved, but we'll just use a built-in R command, chisq.test.
It's pretty much the same as t.test. Say we observed 10 heads and 20 tails. How likely is this by chance?
# First, we the specify the number of heads and tails and the predicted
# number in the population and use chisq.test() should always specify
# correct=F
coins = c(10, 20)
chisq.test(coins, p = c(0.5, 0.5), correct = F)
##
## Chi-squared test for given probabilities
##
## data: coins
## X-squared = 3.333, df = 1, p-value = 0.06789
** Exercise 1: **
We roll a six-sided die and observed the following number of 1s, 2s, 3s, 4s, 5s, 6s: freq = c(22,21,22,27,22,36) Is the die fair? (i.e. probs = c(1,1,1,1,1,1)/6)
We know that in English, the five most commonly occuring letters are [E T N R O], that occur in the following proportions: [.29 .21 .17 .17 .16]. In other languages, the distribution of these letter is much different. If count up the number of letters on a section of text and come up with 100 E's, 110 T's, 80 N's, 55 R's, and 14 O's, is this likely to be written in English?
** Exercise 2: **
UCBAdmissions contains a random sample of a group of admittees to UC Berkeley based on gender. We want to see if the proportion of men and women admitted and rejected are the same across departments. Run the following commands to get the data.data(UCBAdmissions)
x = ftable(UCBAdmissions) #flatten the data
# We want to test rows one and two and see if they are different. We get
# those rows with x[1:2,] The actual proportions are given by:
prop.table(x, 1)
## Dept A B C D E F
## Admit Gender
## Admitted Male 0.42738 0.29466 0.10017 0.11519 0.04424 0.01836
## Female 0.15978 0.03052 0.36266 0.23519 0.16876 0.04309
## Rejected Male 0.20965 0.13865 0.13731 0.18687 0.09243 0.23510
## Female 0.01487 0.00626 0.30595 0.19092 0.23396 0.24804
Are the proportion of men and women admitted different by department?
Are the proportion of men and women rejected different by department?
Is the total number of men admitted vs men rejected different than the total number of women admitted and rejected? HINT: To get the total number of men and women use:
# Total men admitted and rejected:
m = c(sum(x[1, ]), sum(x[3, ]))
# Total women admitted and rejected:
w = c(sum(x[2, ]), sum(x[4, ]))
# Need to turn this into a table
x2 = rbind(m, w)
** Exercise 3 **
The dataset HW7 contains a variable for religious tradition, reltrad. Create a table of reltrad by sex and then compare whether the percentage of men and women differ by religious traditions.
How about for just the unaffiliated? Does their appear to be a difference? (HINT (see below): You need to set up a new table with the rows by sex and the columns by “unaffiliated”,“not-unaffiliated”. Easiest thing is to create a new variable that indicates whether someone is unaffiliated. This then allows you to make the table easily).
How about with other faiths? Are the proportion of men and women different? Here, you will need to use the syntax below to create an “otherfaith” variable.
# ifelse creates a new variable based on a condition.
unaff = ifelse(HW7$reltrad == "nonaffiliated", "yes", "no")
table(HW7$sex, unaff)
## unaff
## no yes
## male 3866 936
## female 5218 740
** Exericise 4: **
We survey 100 people (40 men and 60 women) and obtained their average scores on a test for depression. The mean for men was 6 with a standard deviation of 1.2. The mean for women was 6.5 with a standard deviation of 1.5. If we set our Type I error rate at 0.025 (two-sided), we are saying that we are willing to live with a situation where we erroneously reject the Null when we observe what values?
If the true difference is -.5, what is the Type II error rate? Use the formula: t_Beta = t_crit - (value to test/SE) and then covert t_Beta to a p-value.
If the true difference is .5, what is the Type II error rate?
If the true difference is 1, what is the Type II error rate?
Bob has developed a new teaching strategy for mathematics for 3rd graders. He wants to prove that his method works by comparing reading scores administered before and after the teaching strategy is introduced. He wants to figure out how many children he will need to test to detect a 5 point difference on a reading test scored out of 100. He doesn't know how much standard deviation to expect in the difference, so to be conservative, he assumes the standard deviation of the difference to be 5 (that is, 95% of the differences will be within 10 points of the average change). He wishes to use a 95% confidence level and a two-sided test. Because this test will have a big impact on how well his test sells, he wants to be able to detect a real difference 90% of the time (i.e. Type II error < 10%). What n will he need to use to obtain a Type II error rate less than 5%? Hint: Keep fiddling with n in power.t.test() until the power comes out to 90% or more.