Intro to Statistics, Data & Experimental Design

• Can we know reality with certainty?
• Is anything certain or known?
• What is evidence?
• What is data?
• What is statistics?

Discuss: What’s missing?

Answer: Experimental Design and Data Collection!

1. Data Planning (Experimental Design)
   • Pilot Studies (Micro. Ver. of #2-4 below)
2. Data Collection (Experiment/Field Study)
3. Data Cleaning/Curation (e.g. remove missing values, units)
4. Data Exploration & Analysis
   • Data Validation (sanity checks, e.g. values make biological sense?)
   • Data Munging/Wrangling (raw -> processed)
   • Data Analysis (Statistics)
   • Data Visualization
5. Data Dissemination (Data Communication)

Q1. What feature of an estimate—precision or accuracy—is most strongly affected when individuals differing in the variable of interest do not have an equal chance of being selected?

Answer: Accuracy

Q2. Is the following study observational or experimental?

Psychologists tested whether the frequency of illegal drug use differs between people suffering from schizophrenia and those not having the disease. They measured drug use in a group of schizophrenia patients and compared it with that in a similar sized group of randomly chosen people.

Sub-questions:

• What is the explanatory and response variable?
• Are they categorical or numerical?

Answer: Observational

Q2. Is the following study observational or experimental?

Psychologists tested whether the frequency of illegal drug use differs between people suffering from schizophrenia and those not having the disease. They measured drug use in a group of schizophrenia patients and compared it with that in a similar sized group of randomly chosen people.

Explanatory variable has values “schizophrenic” and “not schizophrenic”, which is a categorical variable.

Observational because treatment groups (or values of the explanatory variable) not assigned randomly by scientist!!

Definition: A study is experimental if the researcher assigns treatments randomly to individuals, whereas a study is observational if the assignment of treatments is not made by the researcher.

Definition: A parameter is a quantity describing a population, whereas an estimate or statistic is a related quantity calculated from a sample.

Parameter examples: Averages, proportions, measures of variation, and measures of relationship

Statistics is a technology that describes and measures aspects of nature from samples.

Statistics lets us quantify the uncertainty of these measures.

Statistics makes it possible to determine the likely magnitude of measurements departure from the “truth”.

Statistics is about estimation, the process of inferring an unknown quantity of a target population using sample data.

The two sides of the statistical coin:

• Parameter estimation
• Hypothesis testing

Definition: A statistical hypothesis is a specific claim regarding a population parameter.

Definition: Hypothesis testing uses data to evaluate evidence for or against statistical hypotheses.

The two sides of the statistical coin:

• Parameter estimation
• Hypothesis testing

Example: A trapping study measures the rate of fruit fall in forest clear-cuts.

The two sides of the statistical coin:

• Parameter estimation
• Hypothesis testing

Example: A clinical trial is carried out to determine whether taking large doses of vitamin C benefits health of advanced cancer patients.

…well, most of the time.

• Many statistical techniques require assumptions about where your data is coming from (i.e. properties of the population)
• In other words, an assumed probability model describes the population
• Statistical techniques that are based on probability models are called parametric techniques, while those that are not are called non-parametric techniques.

Quote: “Huh?”
- Student

“Modern statisticians are familiar with the notion that any finite body of data contains only a limited amount of information on any point under examination; that this limit is set by the nature of the data themselves, and cannot be increased by any amount of ingenuity expended in their statistical examination: that the statistician's task, in fact, is limited to the extraction of the whole of the available information on any particular issue.”

- R. A. Fisher (biologist!)

There is desired and undesired information in data.

Goals:

• Get accurate information by reducing bias (do we have the right signal?)

• Get precise information by reducing sampling error due to random variation (increase signal-to-noise ratio)

  Definition: Bias is a systematic discrepancy between the estimates we would obtain, if we could sample a population again and again, and the true population characteristic.

There is desired and undesired information in data.

Goals:

• Get accurate information by reducing bias (do we have the right signal?)

• Get precise information by reducing sampling error due to random variation (increase signal-to-noise ratio)

  Definition: Sampling error is the difference between an estimate and the population parameter being estimated caused by chance.

https://www.zoology.ubc.ca/~whitlock/Kingfisher/SamplingNormal.htm

There is desired and undesired information in data.

Goals:

• Get accurate information by reducing bias (do we have the right signal?)

• Get precise information by reducing sampling error due to random variation (increase signal-to-noise ratio)

  “An approximate answer to the right problem is worth a good deal more than an exact answer to an approximate problem.”
  
  - John Tukey

For your question, there is desired (signal) and undesired (noise) information in your data.

Goals:

• Isolate desired information by reducing or controlling for confounding factors (i.e. undesired information)

“The aim … is to provide a clear and rigorous basis for determining when a causal ordering can be said to hold between two variables or groups of variables in a model…”

- H. Simon

“Designing experiments is as much about learning to think scientifically as it is about the mechanics of the statistics that we use to analyse the data once we have it. It is about having confidence in your data, and knowing that you are measuring what you think you are measuring. It is about knowing what can be concluded from a particular type of experiment and what cannot.

- Ruxton & Colegrave

Design your experiment so that:

• Measurements lead to useful data.
• Useful data has information addressing your hypothesis.
• Statistics are tailored to your data and powerful enough to separate out signal from noise.
• Results of statistics can be properly interpreted as evidence for or against your original hypothesis.

“It might be said that the two major goals of designing experiments are to minimize random variation and account for confounding factors.

- Ruxton & Colegrave

Definition: Random variation is the differences between measured values of the same variable taken from different experimental subjects.

Good experiments minimize or control for "unwanted” random variation, so that any variation due to the factors of interest can be detected more easily.

“It might be said that the two major goals of designing experiments are to minimize random variation and account for confounding factors.

- Ruxton & Colegrave

Definition: If we want to study the effect of variable A on variable B, but variable C also affects B, then C is a confounding factor.

“Designing effective experiments needs thinking about biology more than it does mathematical calculations.”

“Experimental design is about the biology of the system, and that is why the best people to devise biological experiments are biologists themselves.”

- Ruxton & Colegrave

• Reading Quiz for Wednesday
  • Whitlock & Schluter, Chapter 2: Displaying data
• Lab #0 due Wednesday at 11:59 pm