Homework 6

EPsy 8261

Part I

Using the function rnorm()' to draw two random samples \( (n_1=10~\mathrm{and}~n_2=10) \) from a normally distributed population having a mean of 5 and standard deviation of 1.

sample.1 <- rnorm(n = 10, mean = 5, sd = 1)
sample.2 <- rnorm(n = 10, mean = 6, sd = 1)

Carry out an independent samples t-test to test whether the population means are equal. 

t.test(sample.1, sample.2, var.equal = TRUE)

Using the results from the t-test, fill in Table 1. Please repeat the process of drawing two random samples and testing whether the population means are equal an additional 9 times, each time adding the results to Table 1.

Table 1
Replication Mean of Sample 1 Mean of Sample 2 t-value p-value
1
2
3
4
5
6
7
8
9
10
  1. Define a type II error?
  2. Did you make any type II errors in the 10 tests that you carried out? Explain.

Part II

We will now draw the two random samples and carry out the t-test 1000 times. To automate this process, we will wrap the syntax in a for loop. The syntax to help you do this is below.

# Set up an empty vector to store the 1000 p-values
my.p <- c()

for(i in 1:1000){
     sample.1 <- rnorm(n = 10, mean = 5, sd = 1)
     sample.2 <- rnorm(n = 10, mean = 6, sd = 1)

     # Store the p-values in the ith element of my.p
     my.p[i] <- t.test(sample.1, sample.2, var.equal = TRUE)$p.value
     }

  1. Plot the 1000 p-values stored in the vector my.p. to compute the following:

  2. Compute the probability of making a type II error based on values stored in my.p. (Show your computational work.)

  3. Change the sample size from 10 to 50 in each of the random samples inside the for loop. Re-run the syntax above (create an empty vector, and run the for loop.) Re-compute the probability of making a type II error based on values stored in my.p'. (Show your computational work.)

  4. What is the relationship between sample size and probability of a type II error?

Part III

Consider that you are an educational researcher interested in carrying out a study to exaimne the effects of teacher professional development on student achievement.

The Regional Education Laboratory's report Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement summarized the results from more than 1,300 studies identified as potentially addressing the effect of teacher professional development on student achievement in three key content areas. Of those studies, only nine meet the standards of evidence for inclusion in the What Works Clearinghouse.

All nine studies employed workshops or summer institutes. In all but one follow-up sessions supported the main professional development event. Afterward, the average student achievement was computed for each classroom in the study. These classroom averages were then used as the data to compare those teachers that participated in the professional development (treatment) and those that did not (control) using an independent samples t-test. The average effect size across the nine studies was 0.54, ranging from –0.53 to 2.39. (treatment – control).

  1. Using the average size of effect found as a guideline (0.54), carry out an a priori power analysis to determine how many treatment and control teachers you would need to participate in a replication study. Write-up the results of this analysis in a couple of sentences. Be sure to include all of the necessary information used so that another researcher could replicate your power analysis.

  2. Use ggplot to create a publishable plot of the power curves (power vs. sample size) for a range of effect sizes (at least three, but no more than five). (Note: Feel free to use G*Power3 to obtain the data. It may need to be re-formatted for ggplot.)