Year 1 stats spin-off

Dr Jens Roeser

Learning outcomes

After completing this lecture and the workshop you should have had the opportunity to …

revisiting inferential statistics covered during year 1 statistics.
practice your understanding of inferential statistics in a quiz (people may call this a mock exam).
use an app to explore the relationship between inferential tests and data visualisation tools (as well as sample size).

Overview

No new materials
Repetition: inferential tests in a nutshell
Quiz: discuss the questions and answers in the workshop
I’ll be referring to this quiz a lot.
Also this app will allow you to explore test statistics.

Inferential data analysis: what and why?

We use data to make important decisions about theories, treatments, interventions.
Are our effects “real” or possibly specific to our sample?
“Statistically significant” is when a difference / relationship is deemed reliable.
Reliability: if we would redo our experiment, would we expect a similar outcome.
Significance level \(\alpha = 0.05\): similar outcome in 95% of potential replications.
- If \(p < \alpha\), we can make claims beyond our sample.

Inferential data analysis: what and why?

\(\alpha = 0.05\) implies that in 5% of replications we will…:
- observe an effect that isn’t real (Type I error)
- fail to find an effect that does exist (Type II error)

The p-value

p-value: how plausible are our data if there was actually no effect (H\(_0\) = True)?
If p < 0.05, our data are unexpected under H\(_0\).
- Reject H\(_0\) in favour of the alternative hypothesis.
If p > 0.05, our data are not inconsistent with H\(_0\).
- Fail to reject H\(_0\) and don’t learn anything.

Choosing an inferential tests

What’s research question you want to answer like?
What’s you design / data like?
Comparing two or more groups?
- t-test
- ANOVA
Relationship between two variables?
- Pearson’s correlation
- Linear regression

Decision tree

Quiz

Attempt the Choosing a statistical test and p-value section of this quiz.
Links can be found on website of this lecture (on NOW).
There will be room for discussing the answers during the workshop.

Independent t-test

Difference between groups
More extreme t \(> |2|\) indicates less overlap between groups; hence smaller p-value.
\(t = \frac{\text{group a} - \text{group b}}{\text{variability in the data}}\)
Comparing two independent groups:
- Experts / novice
- Football players / rugby players
- Treatment / control groups
- rts of young and old adults (Blomkvist et al., 2017)

Independent t-test

library(psyntur)
t_test(rt ~ age_2, data = blomkvist)


    Two Sample t-test

data:  rt by age_2
t = 15.918, df = 704, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 177.1910 227.0489
sample estimates:
  mean in group old-ish mean in group young-ish 
               746.4182                544.2982

Paired samples t-test

Paired observations: each participant contributes data to both groups
Performance before and after intervention
Comparing response times of the dominant hand and non dominant hand

Paired samples t-test

select(blomkvist_wide, id, dominant, nondominant)

# A tibble: 353 x 3
      id dominant nondominant
   <dbl>    <dbl>       <dbl>
 1     1     702.        780.
 2     2     471.        497 
 3     3     639.        638 
 4     4     708         639.
 5     5     607.        652 
 6     6     542.        499.
 7     7     571.        527.
 8     8     509.        547.
 9     9     737.        865.
10    10     550.        569.
# … with 343 more rows

Paired samples t-test

library(psyntur)
paired_t_test(y1 = dominant, y2 = nondominant, data = blomkvist_wide)


    Paired t-test

data:  vec_1 and vec_2
t = -0.31154, df = 352, p-value = 0.7556
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -14.49468  10.53057
sample estimates:
mean of the differences 
              -1.982059

Simulation app and quiz

Explore the simulation for t-test outcomes using this app.
Attempt the t-test section of this quiz.
Links can be found on website of this lecture (on NOW).
There will be room for discussing the quiz during the workshop.

One-way ANOVA

ANalysis Of VAriance
Comparing 3 or more independent groups (between-subjects design)
- Mild, moderate, severe symptoms
- Beginner, intermediate, expert
- Monolingual, bilingual, multilingual
F-ratio: how much more variablity in the data is due to differences between conditions / groups as opposed to the normal variability in the data.
\(F = \frac{\text{variability between groups}}{\text{variability across groups}}\)

One-way ANOVA

select(blomkvist, id, rt, age_3)

# A tibble: 706 x 3
      id    rt age_3    
   <dbl> <dbl> <ord>    
 1     1  702. old-ish  
 2     1  780. old-ish  
 3     2  471. young-ish
 4     2  497  young-ish
 5     3  639. middle   
 6     3  638  middle   
 7     4  708  old-ish  
 8     4  639. old-ish  
 9     5  607. old-ish  
10     5  652  old-ish  
# … with 696 more rows

One-way ANOVA

model <- aov(rt ~ age_3, data = blomkvist)
summary(model)

             Df   Sum Sq Mean Sq F value Pr(>F)    
age_3         2  8173425 4086713   151.9 <2e-16 ***
Residuals   703 18910154   26899                   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

groups <- 3
groups - 1 # Df 1

[1] 2

nrow(blomkvist) - groups # Df 2

[1] 703

One-way ANOVA

(tm <- tidy(model)) # broom package for tidy()

# A tibble: 2 x 6
  term         df     sumsq   meansq statistic   p.value
  <chr>     <dbl>     <dbl>    <dbl>     <dbl>     <dbl>
1 age_3         2  8173425. 4086713.      152.  1.45e-55
2 Residuals   703 18910154.   26899.       NA  NA

tm$sumsq / tm$df

[1] 4086712.58   26899.22

tm$meansq[1] / tm$meansq[2]

[1] 151.9268

Simulation app and quiz

Explore the simulation for ANOVA outcomes using this app
Attempt the ANOVA section of this quiz
Links can be found on website of this lecture (on NOW)
There will be room for discussing the quiz during the workshop.

Correlation

Association between two continuous variables
Pearson’s correlation coefficient r (between -1 and +1)

Positive correlation

No correlation

Negative correlation

Correlation

Sign (\(+\), \(-\)) indicates type of relationship
Value indicates the strength of relationship
- \(<\) 0.2 none or a very weak correlation
- \(>\) 0.2 weak
- \(>\) 0.4 moderate
- \(>\) 0.6 strong
- \(>\) 0.8 very strong

Correlation

select(blomkvist_wide, id, dominant, nondominant)

# A tibble: 353 x 3
      id dominant nondominant
   <dbl>    <dbl>       <dbl>
 1     1     702.        780.
 2     2     471.        497 
 3     3     639.        638 
 4     4     708         639.
 5     5     607.        652 
 6     6     542.        499.
 7     7     571.        527.
 8     8     509.        547.
 9     9     737.        865.
10    10     550.        569.
# … with 343 more rows

Correlation

with(blomkvist_wide, cor.test(dominant, nondominant, method = "pearson"))


    Pearson's product-moment correlation

data:  dominant and nondominant
t = 26.574, df = 351, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 0.7794094 0.8492324
sample estimates:
      cor 
0.8172993

Simulation app and quiz

Explore the simulation for correlation outcomes using this app.
Attempt the Correlation section of this quiz.
Links can be found on website of this lecture (on NOW).
There will be room for discussing the quiz during the workshop.

Linear regression

Relationship between a outcome variable and a predictor variable.
Change in the outcome variable (can be DV) for every unit increase in the predictor variable (can be IV).
Intercept \(\alpha\), slope \(\beta\), residual error \(\epsilon\), \(R^2\)
The outcome \(y\) can be predicted as linear function of \(x\).
\(y = \alpha + \beta \cdot x + \epsilon\)

Linear regression

Linear regression: intercept

Linear regression: positive slope \(\beta\)

Linear regression: negative slope \(\beta\)

Linear regression: no change in slope \(\beta\)

Linear regression: \(R^2\)

How well does the model (regression line) represent the data
Strength of relationship (goes from 0 to 1)
% variance explained \(= R^2 \cdot 100\)

Linear regression

select(blomkvist, id, rt, age)

# A tibble: 706 x 3
      id    rt   age
   <dbl> <dbl> <dbl>
 1     1  702.    84
 2     1  780.    84
 3     2  471.    37
 4     2  497     37
 5     3  639.    62
 6     3  638     62
 7     4  708     85
 8     4  639.    85
 9     5  607.    73
10     5  652     73
# … with 696 more rows

Linear regression

model <- lm(rt ~ age, data = blomkvist) 
summary(model)


Call:
lm(formula = rt ~ age, data = blomkvist)

Residuals:
    Min      1Q  Median      3Q     Max 
-374.81  -93.02  -27.81   54.43 1290.35 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 346.3363    17.3053   20.01   <2e-16 ***
age           5.2929     0.2942   17.99   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 162.3 on 704 degrees of freedom
Multiple R-squared:  0.315, Adjusted R-squared:  0.314 
F-statistic: 323.7 on 1 and 704 DF,  p-value: < 2.2e-16

Simulation app and quiz

Explore the simulation for linear regression outcomes using this app.
Attempt the Linear regression section of this quiz.
Links can be found on website of this lecture (on NOW).
There will be room for discussing the quiz during the workshop.

Model assumptions and evaluation

How well does the model capture the data?
Inferential results might be misleading when violations are severe.

Model assumptions

Parametric tests
Normal distribution
Continuous data
Samples must be independent and identically distributed (iid)
Central limit theorem

Central limit theorem

Distribution of sample means approaches normality as the number of participants increases.
iid applies (independent and identically distributed)

LINE

Linearity
Independence
Normality
Equality of variance

What are residuals?

Residuals are the unexplained (residual) variance: error in the modelling results.
Distance between observed and predicted rt.
The closer the residuals are to 0, the lower the prediction error.

Linearity, Independency, Equality of variance (see Faraway, 2015, p. 74)

Normality: Skewness

Negatively skewed: skew < -2
Normal: skew \(\approx\) 0
Positively skewed: skew > 2

Normality: Kurtosis

Equality of variance

Simulation app and quiz

Attempt the Model assumptions and evaluation section of this quiz.
Explore the simulation for Distribution using this app.
Links can be found on website of this lecture (on NOW).
There will be room for discussing the quiz during the workshop.

Learning outcomes

Revisited inferential statistics
- Why inferential statistics?
- p-values
- Which test when?
- t-test and ANOVA
- Correlation and linear regression
- Model assumptions and evaluation
Quiz with practice exam questions
- Use the workshop to discuss your answers in the moch quiz.
Simulation app to explore test statistics and how they related to data visualisation and sample size.

References

Blomkvist, A. W., Eika, F., Rahbek, M. T., Eikhof, K. D., Hansen, M. D., Søndergaard, M., Ryg, J., Andersen, S., & Jørgensen, M. G. (2017). Reference data on reaction time and aging using the Nintendo Wii Balance Board: A cross-sectional study of 354 subjects from 20 to 99 years of age. PLoS One, 12(12), e0189598. https://doi.org/10.1371/journal.pone.0189598

Faraway, J. J. (2015). Linear models with R (Vol. 2). CRC press.