Pseudoreplication and Sampling

• Reading Assignment for Wednesday (NO QUIZ)
  • Whitlock & Schluter, Chapter 13: Handling violations of assumptions

Goal: Understand variation

We want to distinguish between variation of interest and variation from other sources (again, increase signal-to-noise ratio).

“Whenever we carry out an experiment or observational study, we are either interested in measuring random variation, or (more often) trying to find ways to remove or reduce the effects of random variation, so that the effects that we care about can be seen more clearly.”

Definition: Replication involves making the same manipulations to and taking the same measurements on a number of different, independent experimental subjects.

We are essentially talking about sample size here, but there is more to it than that due to the independence issue.

http://www.zoology.ubc.ca/~whitlock/kingfisher/SamplingNormal.htm

Example: Does gender have an effect on human height?

Measure height in 10 married couples of the opposite sex. Are we safe in restricting our sample to married couples?

Definition: Pseudoreplicates are dependent measures.

Definition: Replicate measures must be independent of each other, i.e. a measurement made on one individual should not provide any useful information about that factor on another individual.

Definition: Pseudoreplication occurs if we analyze pseudoreplicates as if they were replicates.

When we pseudoreplicate, we are making a false claim about the amount of replication.

Both accuracy and precision are affected by pseudoreplication!

Accuracy: Pseudoreplication changes our question from general and interesting to more specific and less interesting.

Precision: Pseudoreplication underestimates the precision due to dependence of measures of interest.

Question: Do blue tit nestlings raised in nest boxes suffer more from external parasites than those raised in natural cavities?

Experimental Design: Investigate the four nestlings in a particular nest box and count the number of parasites on each.

Discuss: Why are these pseudoreplicates? Explain how they affect precision and accuracy.

Answer #1: This design gives you information on parasite load only for birds in this particular nest box.

Answer #2: Nestlings will be similar in many ways due to sharing nest box, and thus variation of parasite load between nestlings in this box will be smaller than between all nestlings.

Pseudoreplication is a biological and experimental design issue, not a statistical issue. Data doesn't look pseudoreplicated.

Common sources of pseudoreplication:

• A shared enclosure
  • Environment affects all individuals similarly
  • Individuals can affect other individuals
• The common environment
• Relatedness
• Pseudoreplicated stimulus
• Measurements over time
• Species comparisons

Make sure you are…

1. accounting for all possible variation.
2. controlling for possible confounding variables.

Experimental study: Make sure individuals differ systematically only in the explanatory variable(s) of interest.

Observational studies: Be aware of confounding variables.

Random sampling, or randomization, can solve many of these problems.

Blocking is another technique to address pseudoreplication in experimental studies.

Matching is analogous to blocking for observational studies.

Definition: Proper randomization means that any individual experimental subject has the same chance as any other individual of finding itself in each experimental group, as well as prepared, setup, or measured in the same way.

• Improper randomization can lead to the introduction of confounding variables in the experimental protocol itself (experimental artifacts).
• Randomization breaks the association between possible confounding variables and the explanatory variable.
• Randomization allows the causal relationship between the explanatory and response variables to be assessed.

Definition: Proper randomization means that any individual experimental subject has the same chance as any other individual of finding itself in each experimental group, as well as prepared, setup, or measured in the same way.

• Randomization does not eliminate variation by confounding variables, only their correlation with treatment.
• Randomization ensures that variation by confounding variables is similar between treatment groups and occurs by chance alone.

Question: Does a specific genetic modification to a tomato plant affect its growth rate?

Experimental Design: Place 50 genetically modified plants, and 50 unmodified plants, into individual pots with compost, and then put them all into a growth chamber.

Discuss: Where can improper randomization appear in this example?

Answer: For example:
-      Difference in compost quality.
-      Difference in temperature across chamber.

Let's look at temperature as a possible confounding variable:

The above randomization would not remove temperature difference across chamber, but simply remove correlation with treatment.

What if we would like to reduce the variation from temperature? We can try blocking.

Our attempt to control for temperature:

Discuss: What’s right and wrong with this particular design?

This particular blocking design is properly replicated and randomized.

The variation due to temperature in each chamber has been reduced, so that the difference between treatments becomes more apparent.

There was a systematic difference of temperature across the original chamber. We have now adjusted the design to systematically account for this difference.

What if you can't do experiments? Randomization does not apply here.

Two strategies are used to limit effects of confounding variables on a difference between treatments in a controlled observational study.

Definition: With matching, every individual in the treatment group is paired with a control individual having the same of closely similar values for the suspected confounding variable.

Definition: With adjustment, use a statistical method, such as analysis of covariance, to correct for differences between treatment and control groups in suspected confounding variables.

Assigning treatments to subjects (one possibility):

1. List all \( n \) subjects, one per row, in a spreadsheet.
2. Use the computer to give each subject a random number.
3. Assign treatment A or B to those subjects receiving the lowest or highest numbers, respectively.

Remember, randomization is important in all processes of the experiment, including preparation, setup, and measurement.

Randomize measurement of replicates in time:

• Watching 50 hours of great tit courtship behaviour on video increases your ability to observe
• After 10 hours of counting through a microscope, tiredeness kicks in
• Aging equipment

This shows time of measurement could be a confounding factor.