Observational studies and causality

Objectives

Identify anecdotal evidence and the problem with drawing conclusions based on anecdotal evidence
Identify explanatory and response variables and their role in research
Understand association rarely implies causation, at least not without further evidence
Differentiate between observational studies and experiments
Understand that causation rarely can be established using an observational study
Identify lurking variables and how they might confound our search for causality

Key vocabulary

Anecdotal evidence is evidence based on few observations or cases that may not be representative of the entire population
An explanatory variable is a variable which explains a change in another variable
A response variable is a variable that we think might might be affected by a explanatory variable
An observational study is a study where researchers collect data without interfering with how the data arises
A lurking variable also known as a confounding variable is a variable that is associated with both the explanatory and response variable. Confounding variables often confound our ability to draw conclusions about causality from observational studies.

Part I Case studies: anecdotal evidence

Case study 1: Average time to graduation at SMART

You are interested in determining how many years on average a student at SMART takes to graduate. Your heard about a group of four students who took 5 years to graduate and and decide it must take longer to graduate from SMART than from other high schools.

Explain why this is a case of using anecdotal evidence to draw conclusions about a population.
Why might the conclusion you drew about graduation times not be valid based on this data?

Case study 2: A new drug and heart attacks

Statement: My friends dad had a heart attack and died after they gave him a new heart disease drug, so the drug must not work.

Explain why this is a case of using anecdotal evidence to draw conclusions about a population.
Why might the conclusion about how effective this drug is not be valid based on this one observation?

Case study 3: Swordfish and mercury

Statement: A man on the news got mercury poisoning from eating swordfish, so the average amount of mercury in a swordfish must be dangerously high

Explain why this is a case of using anecdotal evidence to draw conclusions about a population.
Why might the conclusion in the statement above not be valid for the average swordfish?

Part II Case studies: explanatory and response variables; causality

Case study 4: Car price and weight

Take a look at the scatter plot below which shows the weight and price of 54 randomly selected cars.

Take a look at the scatterplot above. Does there seem to be an association between the two variables? If so, is it positive or negative?
Which variable is the explanatory variable in this scatterplot? Which is the response variable?
Do you think the relationship between weight and price is causal? In other words does a car weighing a lot cause it to be more expensive?
Think of a few confounding variables which may be associated with both a car’s price and weight and may be confounding the causal relationship between them.

Case study 5: Mammal life spans and gestation length

The scatterplot below plots the average gestation length of a mammal against average lifespan of 62 mammals. Note: gestation length is the length of pregnancy for a mammmal

Take a look at the scatterplot above. Does it seem that there is an association between gestation length and life span? If so is it positive or negative?
In the scatterplot, which variable is the explanatory variable? Which is the response variable?
Do you think the relationship between gestation length and life span is causal? In other words does having a long gestation period cause a mammal to have a long lifespan?
Think of a few confounding variables which may be associated with both a mammal’s gestation and life span and may be confounding the causal relationship between them.

Case study 6: Hours spent watch TV and grade in stats class

The scatterplot below shows the data for 25 students in a statistics course. The scatter plot compares hours spent watch TV per week to grade in the class.

Does there seem to be an association between how much TV a student watches per week and their final grade in the class? If so, what sort of association is it?
In the scatterplot, which variable is the explanatory variable and which is the response variable?
Do you think that the relationship between TV watched per week and grades in causal? In other words does watching a lot of TV cause your grades to be better or worse? Explain.
Think of a few confounding variables which may be associated with both the number of hours a student watches tv and their grade in statistics class and may be confounding the causal relationship between them.