BIO2POS Lecture Topic 4B

class: middle
background-image: url(data:image/png;base64,#LTU_logo_clear.jpg)
background-position: top left
background-size: 25%

# BIO2POS 
# Calculating Power, and choosing the appropriate test
## Data Analysis Topic 4B
### La Trobe University

---

# Welcome!

### In this lecture we will discuss how to calculate statistical power, and have a quick recap of the statistical tests we have covered so far this semester.

Over the following slides, we will cover:

* .orangered_style[Calculating Power using software]

* Software Details - G*Power and jpower
--

* Choosing the appropriate test
  
--

* Recap of the tests covered so far

---

# Intended Learning Objectives

### By the end of this lecture you will:

* be able to carry out simple power calculations in .navy_style[G*Power] and jamovi

* have revised which statistical test (out of the ones we have covered so far) is most appropriate to use, given a specific scenario
  
--

A solid knowledge of statistical power will be beneficial for any experiment design you conduct, and revising the tests we have introduced so far will help consolidate your understanding of the DA content.

---

# Statistical Power

We introduced the concept of .orangered_style[Statistical Power] in the [DA Topic 4A lecture](https://rpubs.com/LTU_BIO2POS/DA4A). A short recap is provided below:

**Formal Definition:** .orangered_style[Statistical Power] `$(1 - \beta)$` is the probability of correctly rejecting the null hypothesis of a statistical test.

**Informal Definition:** Power is the probability of observing a statistically significant result, when it is truly there to be found.

We have covered the various ways in which statistical power can be increased or reduced, but we have not yet covered how to actually calculate it - we will discuss this now, in the first part of this lecture.

---

# Calculating Statistical Power

To calculate .orangered_style[Statistical Power], we won't use by-hand calculations.

Instead, we will use jamovi, and another great, free software program called .navy_style[[G*Power](https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower)] (see Faul et al. 2007; Faul et al. 2009).

* .navy_style[G*Power] can look daunting at first, but is reasonably easy to use once you get the hang of it
  
--

We can use .navy_style[G*Power] to conduct .orangered_style[Statistical Power] analyses, and to compute .orangered_style[effect sizes] for:

* `$t$`-tests,
  
  * `$F$`-tests, and
  
  * other tests such as `$\chi^2$`-tests (which we will cover later in the semester)

---

# G*Power

Instructions for downloading .navy_style[G*Power] are available on the LMS

* See Section 1.2 of the *Additional Reading: Statistical Power* document, or simply click [here](https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower) and navigate to the `Download` section
  
--

Worksheets for using .navy_style[G*Power] for various statistical tests are provided on the LMS

* You can use these as reference for your ED assessment task
  
--

We will walk through the process for a .navy_style[G*Power] .orangered_style[Statistical Power] calculation over the next few slides.

---

# Calculating Power - One Sample `$t$`-test Example

Suppose we would like to assess whether the average age (in years) of .seagreen_style[BIO2POS] students is different to 20.

To test this, we could conduct a one sample `$t$`-test. Our hypotheses would be:

--
  
`$$H_0: \mu_{age} = 20 \text{ vs } H_1: \mu_{age} \neq 20$$`

where `$\mu_{age}$` denotes the true population mean age of .seagreen_style[BIO2POS] students.

Suppose that we require our test to have a .orangered_style[Statistical Power] of **at least** `$0.8$`.

* I.e., we want at least an 80% chance of observing a statistically significant result, when it is truly there to be found
  
--

As a guess, what do you think our minimum sample size would need to be, to achieve this power level?

---

# Calculating Power - One Sample `$t$`-test Example

.pull-left[
When we first open up .navy_style[G*Power], this is what we will see:
]

.pull-right[
<img src="data:image/png;base64,#gpower_main_menu.jpg" width="375px" style="display: block; margin: auto;" />
]

---

# Calculating Power - One Sample `$t$`-test Example

.pull-left[
To conduct our power calculation, we need to:

1: Choose **Test family:** `$t$`-tests
]

.pull-right[
<img src="data:image/png;base64,#gpower-ttest-1.jpg" width="375px" style="display: block; margin: auto;" />
]

---

# Calculating Power - One Sample `$t$`-test Example

.pull-left[
2: Choose **Statistical test:** Means: difference from constant (one sample case)
]

.pull-right[
<img src="data:image/png;base64,#gpower-ttest-2.jpg" width="375px" style="display: block; margin: auto;" />
]

---

# Calculating Power - One Sample `$t$`-test Example

.pull-left[
3: Choose **Type of power analysis:** *A priori*: Compute required sample size - given `$\alpha$`, power, and effect size
]

.pull-right[
<img src="data:image/png;base64,#gpower-ttest-3.jpg" width="375px" style="display: block; margin: auto;" />
]

---

# Calculating Power - One Sample `$t$`-test Example

.pull-left[
4: Input Parameters: Here we will use:

* Number of Tails (normally Two)
  
  * Effect size (we will calculate this last)
  
  * `$\alpha$` (normally 0.05)
  
  * Power (selected as 0.8 here)
]

.pull-right[
<img src="data:image/png;base64,#gpower-ttest-4.jpg" width="375px" style="display: block; margin: auto;" />
]

---

# Calculating Power - One Sample `$t$`-test Example

.left-column[
.longequation_style[
5: To calculate the .orangered_style[effect size], click **Determine =>** and then:

* Specify a **Mean `$H0$`** value (20 here)
 
 * Specify a **Mean `$H1$`** value (we will choose 22)
 
 * Specify a **SD `$\sigma$`** value (we will choose 5 - normally we would use .seagreen_style[pilot data] to help estimate this)
]
]

.right-column[
<img src="data:image/png;base64,#gpower-ttest-5.jpg" width="550px" style="display: block; margin: auto;" />
]

---

# Calculating Power - One Sample `$t$`-test Example

.left-column[
6: Click **Calculate and transfer to main window**
]

.right-column[
<img src="data:image/png;base64,#gpower-ttest-6.jpg" width="550px" style="display: block; margin: auto;" />
]

---

# Calculating Power - One Sample `$t$`-test Example

.left-column[
7: Finally, click **Calculate** in the main window
]

.right-column[
<img src="data:image/png;base64,#gpower-ttest-7.jpg" width="550px" style="display: block; margin: auto;" />
]

---

# Calculating Power - One Sample `$t$`-test Example

So for this example, it turns out that we require a sample size of at least 52 students, to achieve a .orangered_style[Statistical Power] of at least 0.8 for this test.

The processes for other power calculations are fairly similar.

* In your own time, try adjusting some of the different inputs, and check the impacts on the resultant .orangered_style[Statistical Power] and .orangered_style[effect size].

---

# Calculating Power in jamovi

We can also conduct power calculations in jamovi, using the .navy_style[jpower] module.

This is not as comprehensive as .navy_style[G*Power], but is arguably easier to use.

* We can only use .navy_style[jpower] for `$t$`-test power calculations
  
--

* If we are trying to determine an appropriate sample size, we will already need to know the minimum desired effect size

* .navy_style[jpower] includes options to produce several helpful visual diagrams and a summary explanation of the power calculation process
  
---

# jpower - One Sample `$t$`-test Example

We can replicate our earlier .seagreen_style[one sample t-test example] in .navy_style[jpower] as follows:

---

# Choosing the Right Test

We have covered a variety of important and widely-used statistical tests so far this semester.

Using this knowledge, when you collect or receive data as part of a scientific study, you should now feel more confident in your ability to conduct the following steps:

1. Explore and describe the data through .seagreen_style[descriptive statistics] and .seagreen_style[data visualisations]

2. Formulate appropriate statistical hypotheses `$(H_0 \text{ and } H_1)$`

3. Choose an appropriate statistical test and determine the associated power

4. Conduct the chosen test, and check the relevant test assumptions

5. Summarise your results clearly

Let us go over some of these steps in more detail.

---

# Descriptive Statistics and Visualisations

You should feel comfortable exploring the data using descriptive statistics and data visulations:

* .orangered_style[Numeric Variables]
  
    * Create Histograms and Box Plots
    
    * Tabulate descriptive statistics - mean, median, IQR etc
    
--

* .orangered_style[Categorical Variables]
  
    * Bar Charts
    
    * Frequency Tables
    
---

# Hypothesis Test Steps

Recall that there are 7 main steps we typically follow when conducting a .seagreen_style[hypothesis test]:

1. Establish the null hypothesis `$H_0$` and the alternate hypothesis `$H_1$`

2. Determine the appropriate statistical test

3. Set the .orangered_style[level of significance] `$\alpha$` (before collecting data if possible)

4. Gather your sample data (if possible)

5. Analyse your data

6. Reach a statistical conclusion (reject `$H_0$`/fail to reject `$H_0$`)

7. Summarise results in a written conclusion

---

# Review - One Sample `$t$`-test

.left-column[
**Process:** Compare the mean from one sample (numeric data) to a fixed reference frame `$(H_0)$`

**Assumptions:**

* Normality of Data
  
  * Independence of Observations
]

.right-column[

]

---

# Review - Paired `$t$`-test

**Process:** Test for mean difference with paired/repeated measures data `$(H_0: \text{ mean difference } = 0)$`

**Assumptions:**

* Normality of Difference Variable
 
 * Independence of Observations
 
<img src="data:image/png;base64,#paired_wiggle.jpg" width="800px" style="display: block; margin: auto;" />

---

# Review - Paired `$t$`-test Assumptions

.left-column[
**If Assumptions Violated:**

* Use .orangered_style[Wilcoxon Signed-Rank Test]
  
]

.right-column[
<img src="data:image/png;base64,#wilcoxon_wiggle.jpg" width="550px" style="display: block; margin: auto;" />
]

---

# Review - Two Sample `$t$`-test

**Process:** Compare means of two independent groups `$(H_0: \text{ means are equal})$`

**Assumptions:**

* Normality of Variables
  
  * Independence of Observations
  
  * Variances of both groups equal

---

# Review - Two Sample `$t$`-test Assumptions

.left-column[
**If Assumptions Violated:**

* If equal variance assumption violated, use .orangered_style[Welch's *t*-test]

* If Normality assumption violated, use .orangered_style[Mann-Whitney U Test]
  
]

.right-column[
<img src="data:image/png;base64,#mann-whitney_petrels.jpg" width="550px" style="display: block; margin: auto;" />
]

---

# Review - One-way ANOVA

**Process:** Compare means of two+ independent groups `$(H_0: \text{all means are equal})$`

**Assumptions:**

* Normality of .orangered_style[Residuals]
  
  * Independence of Observations
  
  * Variances of all groups equal

---

# Review - One-way ANOVA post-hoc Tests

.left-column[
**Process:** Conduct multiple pairwise comparisons

**Assumptions:**

* Use either .orangered_style[Tukey HSD] or .orangered_style[Games-Howell] test
  
]

.right-column[
<img src="data:image/png;base64,#anova_posthoc_gh_petrel.jpg" width="550px" style="display: block; margin: auto;" />
]

---

# Review - One-way ANOVA Assumptions

.left-column[
**If Assumptions Violated:**

* If equal variance assumption violated, use .orangered_style[Welch's *F*-test]

* If Normality assumption violated, use .orangered_style[Kruskal-Wallis Test]
]

.right-column[
<img src="data:image/png;base64,#kw_petrel.jpg" width="500px" style="display: block; margin: auto;" />
]

---

# Review - One-way Repeated Measures ANOVA

**Process:** Compare means of two+ dependent groups `$(H_0: \text{ means are equal})$`

**Assumptions:**

* Normality of .orangered_style[Residuals]
 
 * Independence of Observations
 
 * Variances of differences for all within-subject condition pairs are equal (.orangered_style[Sphericity])
 
<img src="data:image/png;base64,#sauna_rmanova_q23_all.png" width="650px" style="display: block; margin: auto;" />

---

# Review - One-way RM ANOVA post-hoc Tests

.left-column[
**Process:** Conduct multiple pairwise comparisons

**Assumptions:**

* Use either .orangered_style[Tukey HSD] test or .orangered_style[Bonferroni Correction] process
  
]

.right-column[
<img src="data:image/png;base64,#sauna_posthoc.png" width="550px" style="display: block; margin: auto;" />
]

---

# Review - One-way RM ANOVA Assumptions

.left-column[
**If Assumptions Violated:**

* If .orangered_style[Sphericity] assumption violated, use either .orangered_style[Greenhouse-Geisser] or .orangered_style[Huynh-Feldt]  `$F$`-test

* If Normality assumption violated, use .orangered_style[Friedman Test]
]

.right-column[
<img src="data:image/png;base64,#sauna_friedman.png" width="500px" style="display: block; margin: auto;" />
]

---

# Summary

We can calculate .orangered_style[Statistical Power] using both the .navy_style[jpower] jamovi module and the .navy_style[G*Power] software.

We have covered the following statistical tests so far, which you should feel comfortable using, given a data set:

These tests should hold you in good stead for a wide variety of scientific studies.

---

# End

That concludes our lecture on calculating statistical power, and our review of the statistical tests covered so far in .seagreen_style[BIO2POS].

### What to do next:

* .seagreen_style[Quick Kahoot revision quiz]: Please go to [kahoot.it](kahoot.it) and type in the code shown

* If you have any questions, check the LMS, email us or ask in the computer labs

### Further Reading

* PDF booklet on statistical power on the LMS. **Please note Section 1.3 of this booklet is assessable.**
 
---

# References

* Cohen, J. (1988). *Statistical Power Analysis for the Behavioral Sciences*. 2nd edition. New York: Academic Press.

* Faul, F., Erdfelder, E., Buchner, A., and Lang, A.-G. (2009). Statistical power analyses using G\*Power 3.1: Tests for correlation and regression analyses. *Behavior Research Methods*, 41, 1149-1160.

* Faul, F., Erdfelder, E., Lang, A.-G., and Buchner, A. (2007). G\*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. *Behavior Research Methods*, 39, 175-191.

* Kokoska, S. (2020). Introductory statistics: a problem-solving approach (Third edition..). W H FREEMAN.

* The jamovi project. (2022). *Jamovi [Computer Software]*. [https://www.jamovi.org](https://www.jamovi.org).

---
class: middle

These notes have been prepared by Rupert Kuveke, Amanda Shaker, and other members of the Department of Mathematical and Physical Sciences. The copyright for the material in these notes resides with the authors named above, with the Department of Mathematical and Physical Sciences and with the Department of Environment and Genetics and with La Trobe University. Copyright in this work is vested in La Trobe University including all La Trobe University branding and naming. Unless otherwise stated, material within this work is licensed under a Creative Commons Attribution-Non Commercial-Non Derivatives License 
<a href = "https://creativecommons.org/licenses/by-nc-nd/4.0/" target="_blank"> BY-NC-ND. </a>