STM1001 Topic 5B Lecture

class: middle
background-image: url(data:image/png;base64,#LTU_logo.jpg)
background-position: top left
background-size: 30%

# STM1001 [Topic 5B](https://bookdown.org/a_shaker/STM1001_Topic_5B_Sci/) Lecture
## Designing a Study Part III
### La Trobe University
This presentation complements the [Topic 5B readings](https://bookdown.org/a_shaker/STM1001_Topic_5B_Sci/)

---

# Topic 5B: Designing a Study Part III

### In this topic, we will introduce the concept of internal validity. Following this introduction, we will consider internal validity in the context of experimental studies.

---
# Internal validity
.content-box-blue[
**Definition (Internal validity)** Internally valid refers to the strength of the association between the outcome and the comparison/connection.]

**Example** In a study of treating depression in adults (Danielsson et al. 2014), three treatments were compared: exercise, basic body awareness therapy, or advice.

If any differences between the treatments were found, the researchers need to be confident that the differences were due to the treatment.

For this reason, the three groups were compared to ensure the groups were similar in terms of average ages, percentage of women, taking of anti-depressants, and many other aspects.

---

# The explanatory variable

The explanatory variable may be associated with changes in the values of the response variable. However, it may not; after all, determining this is the purpose of the study.

If nothing else influenced the values of the response variable, life would be easy: Any change of a given size in the value of the explanatory variable would always result in a change of the same size in the value of the response variable.

.content-box-blue[
**Example (Explanatory variable)** In the typing-speed study (Example 2.4), the explanatory variable is the sex of the person. If nothing else influenced typing speed, all females would record the same typing speed every time, and all males would record the same typing speed every time. This is clearly unreasonable.]

---

# Extraneous variables 
**Definition (Extranaeous variable)** An extraneous variable is any variable that is (potentially) associated with the response variable, but is not the explanatory variable.

**Definition (Confounding variable)** A confounding variable (or a confounder) is an extraneous variable associated with the response and explanatory variables.

**Definition (Confounding)** Confounding is when a third variable influences the relationship between the response and explanatory variable.

**Definition (Lurking variable)** A lurking variable is an extraneous variable associated with the response and explanatory variables (that is, is a confounding variable), but whose values are not measured, assessed, described or recorded in the study.

***We will consider an example in a moment.***

---

name: menti
class: middle
background-image: url(data:image/png;base64,#menti.jpg)
background-size: 115%

# Kahoot!

## Go to [kahoot.it](https://kahoot.it/) and use

## the code provided

---

#  Internal validity and experimental studies

The conclusions drawn from a study are only as good as the data that the conclusions are based on, and the data are only as good as the study design that the data emerge from.

A good study requires high *internal validity*: When studying the relationship between the response and explanatory variable, other possible issues that might influence the value of the response variable should be eliminated.

Specific design strategies that we consider for maximising internal validity are:

* Managing **confounding**;
* Managing the **carry-over effect** using washout periods;
* Managing the **Hawthorne effect** by blinding individuals;
* Managing the **observer effect** by blinding the researchers; and
* Managing the **placebo effect** using controls.

---

# Managing confounding

Confounding has the potential to compromise the internal validity of the study. Suppose, for example, that the researchers created two groups:

Group A: Women recruited at a female-only gym.

Group B: Men recruited at a local nursing home.

The researchers then gave Himalaya 292 to Group A, and the refined cereal to Group B. Suppose metabolic markers were tested 2 hours after eating (e.g. blood sugar levels). If a difference was found between the two groups, the difference may because: *diet*, *sex*, *age*, *health and fitness levels*.

The **groups** being compared **should be as similar as possible**.

Confounding can be managed by:

* **Restricting** the study to a certain group (e.g., only people under 30).
* **Blocking.** Analyse the data separately for different groups (e.g., analyse the data separately for people under 30, and 30 and over).
* **Analysing** using special methods (after measuring ages).
* **Randomly allocating** people to groups: Older and younger people would be spread approximately evenly between groups.

---

# Random allocation vs random sampling

*Random sampling* and *random allocation* are two different concepts, that serve two different purposes, but are often confused:

**Random sampling** allows results to be generalised to a larger population, and impacts *external* validity. It concerns *how the sample is found to study*.

**Random allocation** tries to eliminate confounding issues, by evening-out possible confounders across treatment groups. *Random allocation* of treatments helps establish cause-and-effect, and impacts internal validity. It concerns *how the members of the chosen sample get the treatments*.

---

# Carry-over effect and washout periods
In the *Himalaya* study, what if patients spent two weeks on the *Himalaya* 292 diet, then the next two weeks on the refined cereal diet?

.content-box-blue[
**Definition (Carryover effect)** The carry-over effect is when the influence of past experience(s) of the individuals carry over to influence future experience(s) of the individuals.]

---

# Hawthorne effect and blinding individuals
People often behave differently (either positively or negatively) if they know (or think) they are in a study or are being watched. This is called the **Hawthorne effect**.

.content-box-blue[
**Definition (Hawthorne effect)** The Hawthorne effect is the tendency of people (or animals, or...) to behave differently if they know (or think) they are being observed.]

The impact of the Hawthorne effect can be minimized by *blinding* the individuals in the experiment so that they do not know:

* that they are in a study;
* the aims of the study, and/or
* which treatment they are receiving.

---

# Observer effect and blinding researchers

What if the *researchers* assessing the outcomes *knew the diet* allocated to each patient? Perhaps surprisingly, this can have an (unconscious) impact on the values of the response variable. This is called the *observer effect*. This could also compromise the internal validity of the study.

.content-box-blue[
**Definition (Observer effect)** The observer effect is when the researchers unintentionally influence the behaviour of subjects.]

The impact of the observer effect can be minimized by blinding the researchers so that they do not know: which treatments the individuals are receiving.

---
# Placebo effect and using controls
What if people *thought* they were on the wholegrain diet, but they weren't? Perhaps surprisingly, individuals in a study may report effects of a treatment (either positive or negative), even if they have not received an active treatment. This could also compromise the internal validity of the study.

.content-box-blue[
**Definition (Placebo effect)** The placebo effect is when individuals report perceived or actual effects without having received the treatment.]

Managing the placebo effect is difficult! However, impact of the placebo effect can be minimized using a *control group*: units of analysis without the treatment applied, but as similar as possible in every other way to those units of analysis receiving the treatment.

A **control** is a unit of analysis without the treatment applied. A **placebo** is a treatment with no intended effect or active ingredient.

---

background-image: url(data:image/png;base64,#computerlab.jpg)
background-position: bottom
background-size: 75%
class: center

# See you in the computer labs!

---
class: middle

These notes have been prepared by Illia Donhauzer and Amanda Shaker. They are based on material written by Peter K. Dunn. Unless otherwise stated, material within this work is licensed under a Creative Commons Attribution-Non Commercial-Share Alike License <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">CC BY-NC-SA </a>