Learning objectives

  1. Recognize the difference between an observational study and an experiment
  2. Identify the critical principles that make up a well designed experiment as well as their successful application
  3. Recognize sources of bias in experiments as well as confounding variables
  4. Understand that an experiment is the only way to definitively infer a causal relationship between two variables

Key vocabulary

  1. An randomized experiement is a procedure in which a sample is forcibly divided via random selection into a treatment and control group in order determine the causal relationship between two or more variables
  2. A control group is a group that is given a placebo in order to compare to the treatment group
  3. The treatment group is the group that is given some sort of treatment. This may be a medication or some other intervention.
  4. A placebo is a fake treatment that is known to be ineffective

Case studies

Case study 1: Gender and political preferences

It has been suggested that women and men differ in their political preferences. Women may be more likely than men to prefer Democratic candidates. A political scientist selects a large sample of registered voters, both men and women, and asks them whether they voted for the Democratic or Republican candidate in the last Congressional election. Is this study an experiment? Why or why not?

Case study 2: Better essays and English class

You wish to learn if students in an English course write better essays when they are required to use computer word-processing than when they write and revise their essays by hand. There are 120 students in an English course available as subjects.

  1. Outline the design of an experiment to determine if word-processing results in better essays.
  2. What precautions would you take in doing this experiment that don’t appear in your outline of the design?

Case study 3: 2nd grade experiments

Two second grade teachers, Miss Earls (who has been teaching for 10 years) and Mrs. Morrow (who has been teaching for two years), were really excited by a new curriculum that used animations to teach science. They decided to use their classrooms for an experiment. Since Miss Earls had access to computers in her class, she used the animation lessons. Mrs. Morrow covered similar material with her students using handouts followed by discussions. After students had completed the materials, they were given a test designed by Miss Earls. There were 21 students in Miss Earls’ class and 29 students in Mrs. Morrow’s class. Miss Earls’ class scored, on average, 15 points higher on the test. The two teachers decided that the animation science series was an excellent series. Based on this study, Miss Earls talked her school into purchasing this curriculum.

  1. Critique the study conducted by Miss Earls and Mrs. Morrow. Do you think Miss Earls’ school made a good decision in purchasing the animation science curriculum based on this study?

Case study 4: Diabetes and smoking

A study reported in the Annals of Internal Medicine (January 2010) followed 10,892 middle-aged adults over a nine year period. At the start of the study none of the subjects had diabetes. Roughly 45% of the subjects were smokers. The study found that compared to those who never smoked, subjects who quit smoking had an increased risk of diabetes

  1. Is the study described above an observational study or an experiment? Explain.
  2. Based on this study, should you conclude that quitting smoking causes diabetes? Justify your answer

Case study 5: Hearing impaired customers and store treatments

The research question for an undergraduate research project was whether hearing-impaired consumers were treated differently by store clerks than non-hearing-impaired consumers. There were 20 consumers, 10 of whom were hearing impaired. The consumers were sent in pairs into stores. The hearing-impaired pairs used sign language to communicate with each other and the non-hearing impaired pairs entered stores speaking English to each other. The subjects consisted of 77 sales clerks in 27 stores (from the 175 stores) in a large shopping mall. The response variable was the time from when the pair entered the store and made eye contact with the sales clerk until the clerk approached and offered assistance.

  1. Describe how you would design the rest of the experiment.