BIO2POS Lecture Topic 1B

class: middle
background-image: url(data:image/png;base64,#LTU_logo_clear.jpg)
background-position: top left
background-size: 25%

# BIO2POS 
# Data-Based Decision Making in Science
## Data Analysis Topic 1B
### La Trobe University

---

# Welcome!

### In this lecture we will cover an Introduction to Data-Based Decision Making in Science, focusing on techniques for constructing, testing and assessing the results of scientific hypotheses.

Over the following slides, we will cover:

* .orangered_style[Hypothesis Testing Concepts]

* `$p$`-values
    
--

* Statistical Significance
    
--

* .orangered_style[Confidence Intervals]

If you have completed previous statistics subjects (e.g. .seagreen_style[STM1001]), then this content should be familiar already, and the lecture will act as a refresher.

---

# Intended Learning Objectives

### By the end of this lecture you will:

* be able to discuss the relationship between `$p$`-values and statistical significance

* be able to describe and interpret confidence intervals

The foundational content you learn in Topic 1A and 1B will help you tackle the statistical material in all the future DA Topics.

We will practice content from this topic in this week's DA computer lab, and the computer lab has some additional extension material if you would like to extend your knowledge.

*There is also the .seagreen_style[Online Learning Activity] in this week's LMS tile - it is important to go through this to consolidate your understanding of this topic's content.*

---

# Sample Data vs Population Data

When we conduct statistical inference, we typically use .orangered_style[sample data] to make inferences about the .orangered_style[population of interest].

* We will not typically be able to collect data from every single individual in our population of interest.

* Our hope is that our sample data is representative of the population of interest.

The results we obtain with our sample data provide us with estimates of the characteristics of the population of interest.

We can use statistical tools and techniques to quantify the level of uncertainty in our estimates.

* For example, we might like to know how accurate our .seagreen_style[sample mean] `$\overline{X}$` is as an estimate of the .seagreen_style[population mean] `$\mu$`.

---

# Hypothesis Testing

A .orangered_style[hypothesis] is a specific statement of prediction.

It describes in concrete terms (rather than abstract terms) what you expect will happen in your study.

In statistical hypothesis testing, we consider two options - a .orangered_style[Null Hypothesis] `$(H_0)$` and an .orangered_style[Alternate Hypothesis] `$(H_1)$`.

We always begin our testing with the assumption that our null hypothesis is correct.

The null hypothesis is typically based on existing research, while the alternate hypothesis considers the possibility that `$H_0$` may be inaccurate.

Our intention is to use our sample data to test if the alternate hypothesis provides a more accurate description of the phenomenon of interest.

---

# Example Scenario - Cat Weight

To help introduce statistical hypothesis testing concepts, we will walk through the following simple example.

Suppose we would like to explore whether adding fish into a cat's normal daily diet leads to an increase in weight over time. `$^\star$`
                                                                                                                                                                                                                    
--

.pull-left[

.caption_style[
Image by [Thea Xiaa](https://www.pexels.com/@xiaa/) from [Pexels](https://www.pexels.com/)
]

]

.pull-right[
  * Let us assume that the average weight of a domestic shorthair cat is 4.5kgs.
{{content}}
]

* Suppose we sample 20 Victorian families, each of which has a cat that eats fish in their normal daily diet.
{{content}}

* We record each cat's *Weight*, and these 20 measurements form our sample data set
{{content}}

`$^\star$` *A more sophisticated measurement would be body condition score, and we may return to this later in the semester*

---

# Hypothesis Options - Cat Weight

For the cat example scenario, we have the following hypothesis options, with `$\mu$` denoting the population average weight of domestic shorthair cats:

.pull-left[
.bold_style[The Null Hypothesis] `$(H_0)$`:  
.seagreen_style[Eating fish] as part of their regular diet .orangered_style[has no effect] on the .orangered_style[average weight] of the cats
{{content}}
]

Based on this null hypothesis, we expect the  population *average weight* of the cats who have fish in their diet to be:
{{content}}

* `$\mu = 4.5$`

.pull-right[
.bold_style[The Alternate Hypothesis] `$(H_1)$`: .seagreen_style[Eating fish] as part of their regular diet .orangered_style[has an effect] on the .orangered_style[average weight] of the cats
{{content}}
]

For the alternate hypothesis, we can specify the population **average weight**  of the cats who have fish in their diet to be:

* `$\mu < 4.5$` or `$\mu > 4.5$` or `$\mu \neq 4.5$` 
 
 (which is most appropriate?)

---

# Directional vs Non-Directional Hypotheses

Regardless of scenario, our null hypothesis `$H_0$` will typically take the form of a parameter .bold_style[being equal to] something (e.g. a number or another parameter).

* For the .seagreen_style[cat example], we have `$H_0: \mu = 4.5$`
 
 
 
--

For our alternative hypothesis `$H_1$`, we are considering that `$\mu$` could be different to the null hypothesis value.

.pull-left[
* We have several options here: 
{{content}}
]

* `$\mu <$` the `$H_0$` value, or 
{{content}}

* `$\mu >$` the `$H_0$` value, or 
{{content}}

* `$\mu \neq$` the `$H_0$` value

.pull-right[
For the .seagreen_style[cat example]:

* `$H_1: \mu < 4.5$`

* `$H_1: \mu > 4.5$`

* `$H_1: \mu \neq 4.5$`

]

Our choice of `$H_1$` is extremely important and will affect our subsequent inference.

---

# Directional vs Non-Directional Hypotheses

.pull-left[
 ### .orangered_style_light[Directional Alternate Hypothesis]
 ### (One-Sided or One-Tailed Test) 
]

.pull-right[

A directional `$H_1$`  will assume the parameter of interest is .bold_style[either] greater than .bold_style[or] less than the parameter's value under the null hypothesis.
]

.pull-left[
 ### .orangered_style_light[Non-Directional Alternate Hypothesis] 
 ### (Two-Sided or Two-Tailed Test) 
]

.pull-right[

A non-directional `$H_1$` will assume the parameter of interest is .bold_style[not equal to] the parameter's value under the null hypothesis.
]

In practice, non-directional alternate hypotheses are generally preferred, as they are .seagreen_style[unbiased] in terms of the predicted direction of the results.

We would only select a directional alternate hypothesis if we had compelling prior evidence or expert opinion to support our choice.

---

# Sample Data -> Inference

When we compute our .orangered_style[sample mean], this is based on the information in our .orangered_style[sample data set].

However, we cannot simply compare our sample mean to our null hypothesis value to conclude if `$H_0$` is accurate.

* Suppose we conduct an experiment twice:
  
  We collect different sample data and compute a different sample mean each time 
  (e.g. 4.2kgs and 4.5kgs for the .seagreen_style[cat example])
  
--

* .bold_style[Which sample mean is better?] 
  
--

* Under our null hypothesis, we assume that the sample mean is a random variable and follows a certain .orangered_style[probability distribution].
  
--

* This will provide us with a range of potential sample mean values

* Some values will be more likely than others

* The specific distribution will depend on the test being conducted

---

# Sample Data -> Inference

When we talk about .seagreen_style[statistical inference], generally we are using .orangered_style[sample data] to *infer* something about a .orangered_style[population of interest].

Our motivation for conducting a hypothesis test is to determine:

* Given our sample data, is `$H_0$` likely to be accurate/true?
  
--

To answer this question, we will need to conduct a formal statistical test (with several steps), using statistics from our data, e.g.:

* the sample mean `$\overline{X}$`, 
  
  * the sample standard deviation SD,
  
  * the sample size `$n$`

We will use these values, along with our `$H_0$` value for our parameter of interest, to compute a .orangered_style[Test Statistic].

---

class: middle

# Test Statistics

Test statistics are random variables which we use when trying to decide if we can reject `$H_0$`.

Different statistical tests will have different test statistic equations and details, but the way in which we use test statistics is typically .seagreen_style[test-agnostic].

What we will focus on here is an overview on .bold_style[why] we use test statistics, .bold_style[how] they tie in with our hypothesis testing, and .bold_style[how to use them for statistical inference].

For the time being, it may help to think of a test statistic as being a bit like a .pink_style[smoothie].

* It is created using multiple ingredients (e.g. `$\overline{X}$`, SD, `$n$`), and we are expecting it to taste a certain way/be close to a certain value under `$H_0$` (typically 0).

*We will cover various statistical tests and corresponding test statistics in detail in subsequent DA topics.*

---

Example Test Statistic Distribution

Recall that the height of a probability distribution denotes the .seagreen_style[likelihood] of observing the corresponding x-axis value.

.pull-left[
<img src="data:image/png;base64,#BIO2POS_DA_Lecture_Topic_1B_files/figure-html/unnamed-chunk-2-1.svg" style="display: block; margin: auto;" />
]

.pull-right[

In this example:

* we are likely to observe test statistic values close to 0

* we are unlikely to observe test statistic values far from 0
]
---

# `$p$`-values

If our null hypothesis is correct, our test statistic should be a value that we are .bold_style[likely to see], based on our null hypothesis probability distribution.

Given a test statistic, we can compute a value known as a .seagreen_style[*p*-value].

The `$p$`-value can be used to determine if we should reject or not reject `$H_0$`.

.bold_style[Formal Definition]: The `$p$`-value is the probability, assuming `$H_0$` is true, that we would obtain a test statistic as extreme as or more extreme than the test statistic we have computed.

.bold_style[Informal Definition]: The `$p$`-value is the probability that the null hypothesis is correct.

* If the .seagreen_style[*p*-value] is large, we probably do not have enough evidence to reject `$H_0$`
    
    * We have a high probability that `$H_0$` is true
    
--

* If the .seagreen_style[*p*-value] is small, we may have enough evidence to reject `$H_0$`
    
    * It is unlikely we would observe a test statistic with a small `$p$`-value if `$H_0$` were true

---

As an example, suppose our test statistic equals 1:

* the .bold_style[light blue shaded area] denotes the probability we would observe a test statistic less than or equal to 1 in magnitude

* the .bold_style[orange shaded areas] together denote the probability we would observe a test statistic greater than or equal to 1 in magnitude -  **this is our `$p$`-value**

---

# Rejection Decision

We base our decision to reject or not reject `$H_0$` on our specified level of significance, `$\alpha$`.

We can think of `$\alpha$` as the percentage or level of error we are willing to accept, when making our decision.

* We want to keep this low - typically `$\alpha = 0.05$`
  
--

### Decision:

* If our `$p$`-value is less than `$\alpha$`, we have sufficient evidence to reject `$H_0$`
  
--

* If our `$p$`-value is greater than `$\alpha$`, we cannot reject `$H_0$` - we do not have enough evidence that `$H_0$` is false
    * Note we do not say that we 'accept `$H_0$`'
    
---

### Example: Rejection Region for `$\alpha = 0.05$`

* Test statistics in the rejection region are unlikely under `$H_0$`, and will have `$p$`-values less than `$\alpha$`, meaning we can reject `$H_0$`.
  
--

* Test statistics not in the rejection region will have `$p$`-values greater than `$\alpha$`, meaning we cannot reject `$H_0$`.
---

# 3. Directional Hypotheses

If we have a directional alternate hypothesis, there will only be one rejection region.

.pull-left[
<img src="data:image/png;base64,#BIO2POS_DA_Lecture_Topic_1B_files/figure-html/unnamed-chunk-5-1.svg" style="display: block; margin: auto;" />
]

.pull-right[
<img src="data:image/png;base64,#BIO2POS_DA_Lecture_Topic_1B_files/figure-html/unnamed-chunk-6-1.svg" style="display: block; margin: auto;" />
]

---

# Depiction of scientific method

---

# Confidence Intervals

Another method for determining whether we should reject or not reject `$H_0$` is to use a .orangered_style[Confidence Interval (CI)].

.bold_style[Informal Definition]: A `$(1-\alpha)\times100\%$` .orangered_style[Confidence Interval (CI)] is an interval of values within which the value of the true population parameter will fall with `$(1-\alpha)\times100\%$` certainty.

We construct CIs using our sample data and our chosen `$\alpha$`.

* `$95\%$` CIs are the most commonly reported `$(\alpha = 0.05)$`

CIs help us to clarify the amount of uncertainty around our estimate of the population mean.

* Remember, our sample mean is an estimate, based on our sample data - the true population mean might be quite different from our sample mean

---

# Confidence Intervals

`$$CI = \overline{X} \pm \text{Margin of Error (MoE)}$$`

---

# Confidence Interval Interpretation

Using CIs, we can infer statistical significance at the chosen level of significance `$(\alpha)$`.

* If we are testing a specific parameter value under `$H_0$`, we simply check if that value lies in our CI

Let us consider the cat weight example:

---

If the `$H_0$` value lies within the CI, we .bold_style[do not] have enough evidence to reject `$H_0$` at the `$\alpha$` level of significance.

.pull-left[
.bold_style[Question]: Is the mean likely to be 4.5?
{{content}}
]

* I cannot prove it is not 4.5, based on this CI

.pull-right[
.bold_style[Conclusion]: Do not reject `$H_0$`
{{content}}
]

---

If the `$H_0$` value .bold_style[does not] lie within the CI, we have enough evidence to reject `$H_0$` at the `$\alpha$` level of significance.

.pull-left[
.bold_style[Question]: Is the mean likely to be 4.5?
{{content}}
]

* 4.5 does not lie within the CI

.pull-right[
.bold_style[Conclusion]: Reject `$H_0$`
{{content}}
]
  
---

# Summary

To make inferences about a population of interest, we collect and analyse sample data.

We construct null and alternate hypotheses to test a scientific claim.

* We assume `$H_0$` is true initially

* We assess our data and decide if we have enough evidence to reject `$H_0$`

We determine .orangered_style[statistical significance] using a chosen `$\alpha$` value and results computed from our sample data (e.g. .orangered_style[test statistic, *p*-value, CI])

* If the `$p$`-value `$< \alpha$`, we can reject `$H_0$` at the `$\alpha \times 100\%$` level of significance; otherwise we cannot reject `$H_0$`
 
--

* if the `$H_0$` value lies outside our CI, we can reject `$H_0$` at the `$\alpha \times 100\%$` level of significance; otherwise we cannot reject `$H_0$`

---

# Where to next?

---

# End

That concludes our Introduction to Data-Based Decision Making in Science lecture.

### What to do next:

* .seagreen_style[Quick Kahoot revision quiz]: Please go to [kahoot.it](kahoot.it) and type in the code shown

* Make sure to attend the first DA computer lab

* If you have any questions, check the LMS, email us or ask in the computer labs

### Optional further reading:

* Kokoska (2020) Chapter 1: An introduction to statistics and statistical inference

* Kokoska (2020) Chapter 3: Numerical summary measures

* LMS Online Learning Activity
---

# References

* Kokoska, S. (2020). Introductory statistics: a problem-solving approach (Third edition..). W H FREEMAN.

* The jamovi project. (2022). *Jamovi [Computer Software]*.[https://www.jamovi.org](https://www.jamovi.org).

---
class: middle

These notes have been prepared by Rupert Kuveke, Amanda Shaker, and other members of the Department of Mathematical and Physical Sciences. The copyright for the material in these notes resides with the authors named above, with the Department of Mathematical and Physical Sciences and with the Department of Environment and Genetics and with La Trobe University. Copyright in this work is vested in La Trobe University including all La Trobe University branding and naming. Unless otherwise stated, material within this work is licensed under a Creative Commons Attribution-Non Commercial-Non Derivatives License 
<a href = "https://creativecommons.org/licenses/by-nc-nd/4.0/" target="_blank"> BY-NC-ND. </a>