BIO2POS: DA Computer Lab 4

Topic 4: Power Calculations and Revision

Welcome to Computer Lab 4 for the Data Analysis (DA) component of BIO2POS!

Please note that we historically do not have a scheduled on-campus lab for DA Topic 4 content - these questions are additional content provided for you to complete at your own pace off-campus.

In DA Topic 4, we introduced the concept of statistical power, and discussed how to guarantee a high power for your chosen statistical test. We covered the different factors that influence power, and how to calculate power using the free software G*Power (see Faul et al. 2007; Faul et al. 2009).

In this computer lab, you will practice conducting statistical power calculations in G*Power, and check your understanding and interpretation of output for some of the statistical tests we have covered so far.

Learning Outcomes

These labs are designed to provide you with plenty of opportunities to practice different aspects of the statistical content covered in the lectures.

Each lab consists of core questions (with the 🌱 symbol) and extension questions (with the 🌳 symbol).

• We recommend that you aim to complete at least the core component question(s) to solidify your understanding of the factors that influence power
• If you have time, you can work through the extension component question(s)
• We recommend that you aim to complete all questions before the next DA lecture

Having completed this lab, you will be able to:

• Conduct statistical power calculations in G*Power
• Understand and discuss the impact on power of changing specific study design factors

You will also have refreshed your understanding of how to interpret results of certain statistical tests.

Before you begin, please check the following:

1. Have you attended this week’s lectures/watched the lecture recordings?
2. Have you completed this week’s DA online learning activity (if applicable)?
3. Have you completed this week’s DA Quiz (if applicable)?

Please complete at least step 1. first, as doing so will help you to better understand the concepts you will need for this computer lab.

You may like to take screenshots of your results and save them e.g. in a Word document to your OneDrive, so you can access them easily at a later date.

1 G*Power Sample Size Calculation - One Sample t-test 🌱

In the DA Topic 4B lecture we covered how to conduct a statistical power calculation in G*Power.

Remember that you can download G*Power free of charge here (navigate to the Download section).

Consider the following scenario:

Suppose you would like to assess whether the average height (in cms) of BIO2POS students is different from 165 cm.

Let \(\mu_{height}\) denote the true population mean height of BIO2POS students.

You decide to conduct a one sample \(t\)-test, with hypotheses:

\[H_0: \mu_{height} = 165 \text{ versus } H_1: \mu_{height} \neq 165\]

Following the process outlined in the DA Topic 4B lecture, use G*Power to answer the following questions.

1.1

Suppose that based on an initial pilot study, you expect the standard deviation of observed values to be 5cm, and that you consider a difference of 4cm to be meaningful.

For your main study, you require an \(\alpha\) value of \(0.05\), and a minimum power value of \(0.8\).

Calculate the sample size required, to achieve this level of power.

1.2

Repeat part 1.1, but now consider a difference of 6cm, rather than 4cm, to be meaningful.

1.3

Compare your results from parts 1.1 and 1.2. What has been the impact on the actual power value and total sample size of increasing the difference considered meaningful?

1.4

Repeat part 1.1, but now suppose that the expected standard deviation of observed values is 3cm.

Compare your result to your result from part 1.1. What has been the impact on the actual power value and total sample size of reducing the standard deviation value?

1.5

Repeat part 1.1, but now suppose that the level of significance is \(\alpha = 0.1\).

Compare your result to your result from part 1.1. What has been the impact on the actual power value and total sample size of increasing the \(\alpha\) value?

2 G*Power Power Calculation - One Sample t-test 🌱

Navigate to the Type of power analysis section in G*Power, and select the Post hoc option from the drop-down menu there.

With this setting, we can calculate the power that a test will have, when we already have our sample data. It would be preferable to conduct the G*Power analysis a priori, i.e. before collecting our data, but in some situations this is not always possible.

For this question, consider the scenario from question 1, and use G*Power to answer the following questions.

2.1

Suppose that based on an initial pilot study, you expect the standard deviation of observed values to be 5cm, and you consider a difference of 4cm to be meaningful.

For your main study, you sample \(n=10\) students, and set \(\alpha = 0.05\).

What is the power value for your one sample \(t\)-test, given these conditions?

2.2

Repeat part 2.1, but now suppose that the level of significance is \(\alpha = 0.1\).

Compare your result to your result from part 2.1. What has been the impact on the power value of increasing the \(\alpha\) value?

2.3

Repeat part 2.1, but now suppose that you have sampled \(n=20\) students.

Compare your result to your result from part 2.1. What has been the impact on the power value of increasing the sample size?

2.4

Given your result for part 2.3, do you think it would be worthwhile to double the sample size again, to \(n=40\). Explain your reasoning clearly.

3 jpower Power Calculations 🌱

Now that you have had some practice using G*Power, let’s repeat questions 1 and 2, using the jpower module in jamovi. You should obtain the same answers.

For these questions, you can use the relevant effect sizes you previously calculated, as jpower doesn’t offer a method for calculating effect sizes based on pilot study data.

When checking the output, you may like to click on the Power contour plot and Power curve by effect size options in the Plots section, to help visualise the impact of adjustments.

Remember that you can install new modules in jamovi by clicking on the + Modules sign in the top right of the jamovi UI. Then click jamovi library, and search for jpower.

4 G*Power Power Adjustments 🌱

Figure 4.1: Note. From File:Melopsittacus_undulatus_(24082227505).jpg, by Sean Kelleher, 2014, Wikimedia Commons (https://commons.wikimedia.org/). CC BY-SA 2.0 DEED

Suppose that you are planning the design of a study to determine if there is a difference in the mean external parasite loads of Budgerigar nestlings in nest boxes compared to those in natural cavities. You believe that budgerigar nestlings in nest boxes are likely to have a lower mean external parasite load, compared to those in natural cavities.

• Let \(\mu_{nest}\) denote the true population mean external parasite load of Budgerigar nestlings in nest boxes.
• Let \(\mu_{cave}\) denote the true population mean external parasite load of Budgerigar nestlings in natural cavities

You decide to conduct a two sample \(t\)-test, with the hypotheses:

\[H_0: \mu_{nest} = \mu_{cave} \text{ versus } H_1: \mu_{nest} \neq \mu_{cave}\] Suppose you choose to use the initial following settings and assumptions, based on a pilot study:

• \(\alpha = 0.05\)
• Power \((1 - \beta) = 0.8\)
• Allocation Ratio = 1
• Mean for nest box group = 0.2
• Mean for natural cavities group = 0.15
• SD for nest box group = 0.1
• SD for natural cavities group = 0.11
• Effect size \(d \approx 0.4757\)

To achieve the desired power, your initial G*Power analysis suggests you need a total sample size of 142, with 71 birds sampled from each group (screenshot provided below for reference)

4.1

Based on the information provided above in 4, suggest at least two viable options for reducing the sample size required, while maintaining the power level of 0.8. Verify that your suggestions will work using G*Power.

4.2

Based on the information provided above in 4, suggest at least two viable options for increasing the power level above 0.8. Verify that your suggestions will work using G*Power.

5 ANOVA Analysis Interpretation Check 🌱

Suppose that your friend has conducted an analysis of some of the Wolf River data we assessed in Computer Lab 3.

They are a little unsure about their results, and have asked you to check their interpretation.

They provide you with the following conclusion and jamovi output:

Conclusion: The ANOVA was statistically significant as the \(p\)-value was small. Therefore we reject the null hypothesis and conclude that HCB levels increase as the Depth level increases. This result is supported by the medium effect size of 0.183, which supports the statistical significance of the result. The Levene’s test and Shapiro-Wilk test results clearly show that there are no issues with the test assumptions.

jamovi output:

Critique your friend’s conclusion, and discuss any errors or omissions they have made. For the purposes of this question you can assume that the jamovi output shown is correct.

Write out a corrected version of their summary.

6 AI jamovi Output Interpretation 🌳

The following prompt and image were provided to the Microsoft AI tool Copilot:

Prompt: Summarise the results of the statistical test output shown in this image and comment on the statistical significance of the results at the 5% level of significance please

Image:

The AI output is included in the chunk below (minimised by default). Click on the symbol to expand the section.

Please note that each time you submit queries to an AI tool, the output you receive will be different. The output shown here was one of several versions obtained - some were more accurate, while some were less accurate.

AI answer

The statistical test output in the image pertains to a One-Way ANOVA analysis. Here are the key takeaways:

One-Way ANOVA Results: The p-value for HCB is 0.044, indicating evidence to reject the null hypothesis at a 5% significance level. The p-value for Fisher’s test is 0.065, suggesting no significant difference at the same level.

Assumption Checks: The Homogeneity of Variances Test (Levene’s) has a p-value of 0.153, implying no significant violation of homogeneity of variances.

Post Hoc Tests: Mean differences and p-values comparing Surface, Middldepth, and Bottom levels are not statistically significant at the 5% level.

In summary, there are complex relationships within the data, with significant differences in means observed in some tests but not others. The results highlight the need for further exploration and context-specific interpretation.

Do you agree with the AI output? Has it missed anything important?

7 G*Power Calculation - Pea Plant Data 🌳

Recall that in previous computer labs we have assessed pea plant seedling data collected as part of a 2022 experiment in an LTU BIO1AP lab class.

Using G*Power, determine the statistical power of the following tests originally conducted in Computer Lab 2:

1. The one sample \(t\)-test conducted in part 5.2
2. The two sample \(t\)-test conducted in part 5.3
3. The two sample \(t\)-test conducted in part 5.4

Comment on your results. Do you think that the tests have sufficient power to yield meaningful results?

References

Faul, F., Erdfelder, E., Buchner, A., and Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160.

Faul, F., Erdfelder, E., Lang, A.-G., and Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.

These notes have been prepared by Rupert Kuveke. The copyright for the material in these notes resides with the author named above, with the Department of Mathematical and Physical Sciences and with the Department of Environment and Genetics and with La Trobe University. Copyright in this work is vested in La Trobe University including all La Trobe University branding and naming. Unless otherwise stated, material within this work is licensed under a Creative Commons Attribution-Non Commercial-Non Derivatives License BY-NC-ND.