class: middle background-image: url(data:image/png;base64,#LTU_logo.jpg) background-position: top left background-size: 30% # STM1001 [Topic 7](https://bookdown.org/a_shaker/STM1001_Topic_7/) Lecture ## One-way ANOVA ### La Trobe University This lecture complements the [Topic 7 readings](https://bookdown.org/a_shaker/STM1001_Topic_7/) --- # Topic 7: Related Links ## Readings [Topic 7 readings](https://bookdown.org/a_shaker/STM1001_Topic_7/) ## Notation [Notation for Topic 7: One-way ANOVA](https://bookdown.org/a_shaker/STM1001_Topic_0/notation-summary.html#topic-7-one-way-anova) --- # Topic 7: One-way ANOVA **Overview** <iframe src="https://bookdown.org/a_shaker/STM1001_Topic_7/" width="100%" height="400px" data-external="1"></iframe> --- name: stat class: middle background-image: url(data:image/png;base64,#slide_1.png) background-size: 110% --- name: stat class: middle background-image: url(data:image/png;base64,#slide_10.png) background-size: 100% --- # One-way ANOVA * Recall that we used the independent samples `\(t\)`-test to test for a difference between ***two independent groups*** * What if we want to test for a difference between ***two or more independent groups***? -- * Then, we can use one-way ANOVA * ANOVA is short for "Analysis of Variance" --- # Is personality associated with region of the United States? * Today we will be considering an example adapted from Hartnett (2020) and based on [this research study](https://www.apa.org/pubs/journals/releases/psp-a0034434.pdf) (Rentfrow, Gosling, Jokela, Stillwell, Kosinski, and Potter, 2013) * In the study, a group of researchers gave a personality test to over 1.5 million Americans to study whether different regions of the US are associated with different personality traits -- * The research was reported on by Time magazine (Wilson and Cluger, 2013) - you can read the article [here](https://time.com/7612/americas-mood-map-an-interactive-guide-to-the-united-states-of-attitude/) ---  .center[[Time magazine](https://time.com/7612/americas-mood-map-an-interactive-guide-to-the-united-states-of-attitude/) (Wilson and Cluger, 2013)] * The researchers identified three regions of the US based on personality -- * Click on [this link](https://time.com/7612/americas-mood-map-an-interactive-guide-to-the-united-states-of-attitude/) if you want to take the abbreviated test for yourself and see which state you match with --- # Is personality associated with region of the United States? * The personality test was based on the Big Five framework (Costa and McCrae, 1992; Goldberg, 1992), which contains five main personality traits: * Conscientiousness * Agreeableness * Neuroticism * Openness * Extraversion -- Watch [this video](https://www.youtube.com/watch?v=sUrV6oZ3zsk&t=195s) (3:15 to 4:30) if you would like to learn more about the Big Five personality traits --- # Is personality associated with region of the United States? * The data set we will consider has a measurement of each of the Big Five personality traits for each US state * The measurements take values between 0-100, so that high scores indicate high values of the Big Five trait (e.g. high agreeableness), low scores indicate a strong result at the other end of the scale (e.g. high disagreeableness), and 50 indicates the score is in-between the two opposites of the scale -- * States are categorised into four regions: South (S), West (W), Midwest (MW) and Northeast (NE) -- * We will be focussing on the *Extraversion* trait: in particular, we wish to determine whether there is an association between *Extraversion* and *Region* * We will be carrying out a One-way ANOVA to study this question --- # The Big Five personality data set <div class="datatables html-widget html-fill-item" id="htmlwidget-65b25fdbc74dff6f8358" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-65b25fdbc74dff6f8358">{"x":{"filter":"none","vertical":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49"],["Alabama","Arizona","Arkansas","California","Colorado","Connecticut","Delaware","District of Columbia","Florida","Georgia","Idaho","Illinois","Indiana","Iowa","Kansas","Kentucky","Louisiana","Maine","Maryland","Massachusetts","Michigan","Minnesota","Mississippi","Missouri","Montana","Nebraska","Nevada","New Hampshire","New Jersey","New Mexico","New York","North Carolina","North Dakota","Ohio","Oklahoma","Oregon","Pennsylvania","Rhode Island","South Carolina","South Dakota","Tennessee","Texas","Utah","Vermont","Virginia","Washington","West Virginia","Wisconsin","Wyoming"],["27,690","33,569","13,941","177,085","28,995","17,769","4,957","6,774","81,002","52,631","9,314","72,011","36,483","19,821","18,039","21,791","17,414","8,963","30,568","38,270","59,221","38,597","10,479","34,558","5,052","12,102","10,196","8,076","39,549","9,408","86,855","45,940","4,808","61,683","19,515","25,403","67,938","5,322","17,375","4,581","29,738","116,094","20,715","4,118","45,898","44,150","8,349","40,731","3,166"],[55.5,50.6,49.9,51.4,45.3,57.6,47,64.8,60.9,63.2,40.7,62.5,48.9,62.8,45.5,53.4,52.2,44.2,35.2,44.4,55.2,52.9,56.8,62.9,33.1,60,46.4,40.2,59.9,32.4,47,51,52.4,54.6,39.7,30.9,54.6,43.3,60,58.7,51.3,55.2,55.8,26.5,48.5,30.6,38.5,69.8,46],[52.7,46.6,52.7,49,47.5,38.6,38.8,21.4,50.7,60,52.9,48.3,50.2,56.6,48.9,48.1,49.7,32.8,37.3,40.7,54.7,61.6,63.3,59.3,52.3,62.9,31.8,53.5,44.6,45.4,29.8,63.6,52.4,45.9,54.3,59.1,42.8,35.2,55.4,56.7,66.59999999999999,52.3,69.40000000000001,60,47.4,55.8,51.8,57.8,40.7],[55.5,58.4,41,43.2,58.8,34.2,36.5,44.1,62.7,68.8,44.5,50.9,56.2,52.2,50.8,51.3,45,24,37.5,32.2,53,52.5,59.7,60.8,56.1,64.3,55.8,38,40.8,58.5,37.7,68.40000000000001,51.4,46.5,54.7,45.8,52.4,48.5,69.59999999999999,55.8,62,54.4,54.5,38.2,45.3,45,42.6,47.6,42.4],[48.7,38.1,56.2,39.1,34.3,53.4,62.4,41.6,40.8,38,44.2,51.2,59.3,49.1,49,62.5,60.4,71,49.4,63.8,48.6,43.4,52,48.3,43,41.6,44,61.8,56.4,51.6,62.7,44.8,49.6,58.5,52.1,39.5,61.4,61.9,45.7,36.1,49,44.3,30.4,55.8,44.7,36.9,79.2,48.6,46.1],[42.7,54.7,40.3,65,57.9,53.9,42.7,77.5,61,56.9,44.7,55.2,44.9,33.7,40.1,43,53.7,50.8,56.6,59.6,43.4,38.5,46.2,45.7,55,34.3,61.3,48.7,57.6,62,64.5,49.6,21.8,46,42.2,58.8,49.6,59.4,55.3,41.9,50.1,52.7,47.7,54.2,57.1,56.6,36.8,35.7,42.4],["S","W","S","W","W","NE","S","S","S","S","W","MW","MW","MW","MW","S","S","NE","S","NE","MW","MW","S","MW","W","MW","W","NE","NE","W","NE","S","MW","MW","W","W","NE","NE","S","MW","S","S","W","NE","S","W","S","MW","W"]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>State<\/th>\n <th>Sample.size<\/th>\n <th>E<\/th>\n <th>A<\/th>\n <th>C<\/th>\n <th>N<\/th>\n <th>O<\/th>\n <th>Region<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"pageLength":7,"columnDefs":[{"className":"dt-right","targets":[3,4,5,6,7]},{"orderable":false,"targets":0},{"name":" ","targets":0},{"name":"State","targets":1},{"name":"Sample.size","targets":2},{"name":"E","targets":3},{"name":"A","targets":4},{"name":"C","targets":5},{"name":"N","targets":6},{"name":"O","targets":7},{"name":"Region","targets":8}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[7,10,25,50,100]}},"evals":[],"jsHooks":[]}</script> --- # One-way ANOVA * First, we need to set up our hypotheses: `$$H_0: \mu_1 = \mu_2 = \mu_3 = \mu_4 \text{ versus } H_1: \text{not all } \mu_i\text{'s are equal,}$$` where: -- * `\(\mu_1\)` denotes the population mean extraversion for South * `\(\mu_2\)` denotes the population mean extraversion for West * `\(\mu_3\)` denotes the population mean extraversion for Midwest * `\(\mu_4\)` denotes the population mean extraversion for Northeast -- * The test tells us whether there is evidence that at least one of the `\(\mu_i\)`'s is significantly different from the others * It does not tell us which groups, or how many, are different from each other - only whether ***at least one group*** is different from the others * However, post-hoc tests can be carried out for further analysis and we will consider these later in this lecture --- # One-way ANOVA In general terms, a one-way ANOVA can be set up as follows: `$$H_0:\mu_1 = \mu_2 = \ldots = \mu_k \;\;\text{versus}\;\;H_1: \text{not all }\mu_i\text{'s are equal},$$` where: * For some number of `\(k\)` groups, `\(\mu_1 = \mu_2 = \ldots = \mu_k\)` denote the population means for Group 1, Group 2, ..., and Group `\(k\)` respectively. --- # One-way ANOVA * What type of variables are required for a one-way ANOVA? -- .content-box-blue[ .center[ A one-way ANOVA will always involve two variables: ] 1. The ***dependent*** variable, sometimes also called the *response* variable. This should be a numeric, continuous variable. 2. The ***independent*** variable. This should be a categorical variable with ***two or more categories***. ] -- * So our ***dependent*** variable is extraversion * Our ***independent*** variable is region --- # One-way ANOVA Assumptions: The assumptions of the one-way ANOVA test can be summarised as follows: .content-box-blue[ .center[ **One-way ANOVA Assumptions:** ] 1. The data are numeric 2. Independence of observations 3. The random errors are normally distributed 4. The population variances for each group are equal. ] * As we can see, these assumptions bear some similarity to those of the independent samples `\(t\)`-test * However, here we will be checking for normality of the random errors, which will be discussed later * In order to do this, we need to carry out the test first and will therefore check the assumptions *after* carrying out the one-way ANOVA test --- # Visualising the data As usual, we begin with descriptive statistics and plots: | | | | | |:---------|:---------------|:--------|:------| |**Group** |**Sample size** |**Mean** |**SD** | |South |16 |52.71 |8.08 | |West |12 |41.91 |8.69 | |Midwest |12 |57.18 |6.87 | |Northeast |9 |46.41 |10.17 | --- <img src="data:image/png;base64,#Topic_7_Lecture_files/figure-html/unnamed-chunk-4-1.svg" width="80%" style="display: block; margin: auto;" /> --- # Visualising the data From the descriptive statistics and plots, we can observe the following: 1. The boxplots and sample means suggest that the population mean Extraversion may be different between groups. When we carry out the one-way ANOVA, we will see whether or not this difference is ***statistically significant*** -- 1. From the boxplots and standard deviations, there appears to be some difference in variability between groups but the difference may not be strong. We will check the assumption of equal variance more formally later using the Levene's test. -- 1. The sample size in each group is 12, 9, 16 and 12 respectively, for a total sample size of 49 --- #Defining the `\(F\)`-distribution * Recall that for `\(t\)`-tests, we calculated a test statistic and then evaluated how extreme this was by using the `\(t\)`-distribution -- * However, for ANOVA tests, we use the `\(F\)`-distribution. -- * To define the `\(F\)`-distribution, we need to know two degrees of freedom: -- * `\(d_1 = k - 1\)`, where `\(k\)` is the number of groups. This is the "between group" degrees of freedom. * `\(d_2 = N - k\)`, where `\(N\)` is the total sample size. This is the "within group" degrees of freedom. -- * Recall that in our example we have `\(k = 4\)` groups and total sample size of `\(N = 49\)`. We therefore have: * `\(d_1 = k - 1 = 4 - 1 = 3\)` * `\(d_2 = N - k = 49 - 4 = 45\)` -- * We define the `\(F\)` distribution using notation `\(F_{d_1, d_2}\)`, so the distribution that will be used for our example is the `\(F_{3, 45}\)` distribution --- #Defining the `\(F\)`-distribution * Depending on the degrees of freedom, `\(d_1\)` and `\(d_2\)`, the shape of the distribution will look different: <img src="data:image/png;base64,#Topic_7_Lecture_files/figure-html/unnamed-chunk-5-1.svg" style="display: block; margin: auto;" /> --- #The test statistic * For a one-way ANOVA, the test statistic, or the `\(F\)` value, is calculated by estimating the ratio of ***between group variation*** to ***within group variation***: `$$F_{df_1, df_2} = \displaystyle \frac{\text{between group variation}}{\text{within group variation}}$$` -- * The ***between group variation*** is a measure of how much the sample means for each group vary. * The ***within group variation*** is a measure of how much individual sample values within a group vary from their group sample mean. -- * If the ***between group variation*** is much larger than the ***within group variation***, then the `\(F\)`-statistic will be very large and lead to a statistically significant result. * On the other hand, if the ***between group variation*** is not large compared to the ***within group variation***, then the `\(F\)`-statistic will not be large and subsequently will not lead to a statistically significant result. We will now carry out the one-way ANOVA test, and will assume (for now) that the assumptions have been met. --- name: menti class: middle background-image: url(data:image/png;base64,#menti.jpg) background-size: 115% # Kahoot ## Go to [www.kahoot.it](https://www.kahoot.it) and use ## the code provided --- # One-way ANOVA output ``` r Df Sum Sq Mean Sq F value Pr(>F) Region 3 1638 546.2 7.785 0.000272 *** Residuals 45 3157 70.2 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 ``` --- # One-way ANOVA output ``` r Df Sum Sq Mean Sq F value `Pr(>F)` Region 3 1638 546.2 7.785 `0.000272` *** Residuals 45 3157 70.2 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 ``` * The ** `\(p\)`-value** is `\(p < 0.001\)`, which is much less than 0.05, so we reject `\(H_0\)`. That is, we have enough evidence to conclude that there is a statistically significant difference between groups --- # One-way ANOVA output ``` r Df Sum Sq Mean Sq `F value` `Pr(>F)` Region 3 1638 546.2 `7.785` `0.000272` *** Residuals 45 3157 70.2 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 ``` * The ** `\(p\)`-value** is `\(p < 0.001\)`, which is much less than 0.05, so we reject `\(H_0\)`. That is, we have enough evidence to conclude that there is a statistically significant difference between groups * The test statistic (<code>F value</code>) is `\(F = 7.785\)` --- # One-way ANOVA output ``` r `Df` Sum Sq Mean Sq `F value` `Pr(>F)` Region `3` 1638 546.2 `7.785` `0.000272` *** Residuals 45 3157 70.2 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 ``` * The ** `\(p\)`-value** is `\(p < 0.001\)`, which is much less than 0.05, so we reject `\(H_0\)`. That is, we have enough evidence to conclude that there is a statistically significant difference between groups * The test statistic (<code>F value</code>) is `\(F = 7.785\)` * `\(d_1 = 3\)` (read from the <code>Df</code> column, <code>Region</code> row) --- # One-way ANOVA output ``` r `Df` Sum Sq Mean Sq `F value` `Pr(>F)` Region `3` 1638 546.2 `7.785` `0.000272` *** Residuals `45` 3157 70.2 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 ``` * The ** `\(p\)`-value** is `\(p < 0.001\)`, which is much less than 0.05, so we reject `\(H_0\)`. That is, we have enough evidence to conclude that there is a statistically significant difference between groups * The test statistic (<code>F value</code>) is `\(F = 7.785\)` * `\(d_1 = 3\)` (read from the <code>Df</code> column, <code>Region</code> row) * `\(d_2 = 45\)` (read from the <code>Df</code> column, <code>Residuals </code> row) -- * To summarise, we can write: There was a significant difference in mean Extraversion [F(3, 45) = 7.785, `\(p < .001\)`] between regions. --- # Checking assumptions The two assumptions we need to check for here are: 1. Equality of variances 1. Normality of the random errors --- # Levene's test for equality of variances Recall the following null and alternative hypotheses for the Levene's test: * `\(H_0 : \text{The groups have equal variances}\)` * `\(H_1 : \text{The groups do not have equal variances}.\)` Also recall the following summary: .content-box-blue[ .center[ **Levene's test for equality of variances:** ] * If `\(p\)` < 0.05, equal variances cannot be assumed * If `\(p\)` > 0.05, equal variances can be assumed ] --- # Levene's test for equality of variances Let's carry out the Levene's test for our example: ``` Levene's Test for Homogeneity of Variance (center = median) Df F value Pr(>F) group 3 0.3488 0.7902 45 ``` Since `\(p = 0.7902\)` is greater than 0.05, equal variances can be assumed. --- # Checking for normality * When checking for normality for a one-way ANOVA, this needs to be done for the random errors (rather than the response variable) -- * The "random errors" can be approximated by the "residuals" -- * For a one-way ANOVA, each observation will have a corresponding ***residual***, which is the difference between the observed value of the dependent variable for that observation, and the mean of the dependent variable for that group -- * For example, the first state (Alabama) in the data set is from the South group, which has an average extraversion score of 52.71 * Alabama's extraversion score is 55.5, so its residual is `\(55.5 - 52.71 = 2.79\)`. -- * Similarly, the residual value can be calculated for all 49 observations * We will check for normality using these 49 residual values by considering the histogram, Normal Q-Q plots, and Shapiro-Wilk test --- # The Big Five personality data set with residuals <div class="datatables html-widget html-fill-item" id="htmlwidget-c05d5be7d6f505b25493" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-c05d5be7d6f505b25493">{"x":{"filter":"none","vertical":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49"],["Alabama","Arizona","Arkansas","California","Colorado","Connecticut","Delaware","District of Columbia","Florida","Georgia","Idaho","Illinois","Indiana","Iowa","Kansas","Kentucky","Louisiana","Maine","Maryland","Massachusetts","Michigan","Minnesota","Mississippi","Missouri","Montana","Nebraska","Nevada","New Hampshire","New Jersey","New Mexico","New York","North Carolina","North Dakota","Ohio","Oklahoma","Oregon","Pennsylvania","Rhode Island","South Carolina","South Dakota","Tennessee","Texas","Utah","Vermont","Virginia","Washington","West Virginia","Wisconsin","Wyoming"],[55.5,50.6,49.9,51.4,45.3,57.6,47,64.8,60.9,63.2,40.7,62.5,48.9,62.8,45.5,53.4,52.2,44.2,35.2,44.4,55.2,52.9,56.8,62.9,33.1,60,46.4,40.2,59.9,32.4,47,51,52.4,54.6,39.7,30.9,54.6,43.3,60,58.7,51.3,55.2,55.8,26.5,48.5,30.6,38.5,69.8,46],["S","W","S","W","W","NE","S","S","S","S","W","MW","MW","MW","MW","S","S","NE","S","NE","MW","MW","S","MW","W","MW","W","NE","NE","W","NE","S","MW","MW","W","W","NE","NE","S","MW","S","S","W","NE","S","W","S","MW","W"],[52.71249999999987,41.90833333333333,52.71250000000001,41.90833333333333,41.90833333333333,46.41111111111111,52.71250000000001,52.71250000000001,52.71250000000001,52.71250000000001,41.90833333333333,57.18333333333333,57.18333333333334,57.18333333333333,57.18333333333334,52.71250000000001,52.71250000000001,46.41111111111111,52.71250000000001,46.41111111111111,57.18333333333333,57.18333333333333,52.71250000000001,57.18333333333333,41.90833333333334,57.18333333333333,41.90833333333333,46.41111111111111,46.41111111111111,41.90833333333334,46.41111111111111,52.71250000000001,57.18333333333333,57.18333333333333,41.90833333333333,41.90833333333334,46.41111111111111,46.41111111111111,52.71250000000001,57.18333333333333,52.71250000000001,52.71250000000001,41.90833333333333,46.41111111111111,52.71250000000001,41.90833333333334,52.71250000000001,57.18333333333334,41.90833333333333],[2.787500000000127,8.691666666666672,-2.812500000000008,9.491666666666664,3.391666666666665,11.18888888888889,-5.712500000000007,12.08749999999999,8.187499999999991,10.48749999999999,-1.208333333333329,5.316666666666668,-8.283333333333335,5.616666666666665,-11.68333333333333,0.6874999999999905,-0.5125000000000055,-2.211111111111108,-17.5125,-2.011111111111112,-1.98333333333333,-4.283333333333334,4.087499999999989,5.716666666666667,-8.808333333333335,2.816666666666668,4.491666666666667,-6.211111111111107,13.48888888888889,-9.508333333333338,0.588888888888889,-1.712500000000009,-4.783333333333334,-2.583333333333331,-2.208333333333329,-11.00833333333334,8.18888888888889,-3.111111111111114,7.287499999999993,1.516666666666671,-1.412500000000011,2.487499999999994,13.89166666666666,-19.91111111111111,-4.212500000000007,-11.30833333333334,-14.21250000000001,12.61666666666666,4.091666666666669]],"container":"<table class=\"display\">\n <thead>\n <tr>\n <th> <\/th>\n <th>State<\/th>\n <th>E<\/th>\n <th>Region<\/th>\n <th>Group.mean<\/th>\n <th>Residual<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"pageLength":7,"columnDefs":[{"targets":4,"render":"function(data, type, row, meta) {\n return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\", null);\n }"},{"targets":5,"render":"function(data, type, row, meta) {\n return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\", null);\n }"},{"className":"dt-right","targets":[2,4,5]},{"orderable":false,"targets":0},{"name":" ","targets":0},{"name":"State","targets":1},{"name":"E","targets":2},{"name":"Region","targets":3},{"name":"Group.mean","targets":4},{"name":"Residual","targets":5}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[7,10,25,50,100]}},"evals":["options.columnDefs.0.render","options.columnDefs.1.render"],"jsHooks":[]}</script> --- # Checking for normality .pull-left[ <img src="data:image/png;base64,#Topic_7_Lecture_files/figure-html/unnamed-chunk-14-1.svg" style="display: block; margin: auto;" /> **Shapiro-Wilk test:** ``` Shapiro-Wilk normality test data: residuals W = 0.97963, p-value = 0.5498 ``` ] -- .pull-right[ * Considering the histogram, Normal Q-Q plot, Shapiro-Wilk test result and sample size, we can safely conclude the normality assumption has been met. * Note that we have used the ***unstandardised*** residuals to check for normality here. Some people prefer to use the ***standardised*** residuals so that values can be easily interpreted. ] --- # What if the assumptions are not met? If the assumptions have not been met, there are several options available, such as: * Use of the Welch ANOVA, which does not require equal variances * Use of a non-parametric alternative such as the Kruskall-Wallis test * Transformation of the dependent variable. However for a one-way ANOVA analysis, these techniques are beyond the scope of this subject. --- #Post-hoc tests * When a one-way ANOVA test leads to a significant result, it is common to then follow up with post-hoc tests to see which particular groups are significantly different from each other * We can carry out pairwise `\(t\)`-tests to test for differences between each pair of categories -- * However, each time we carry out a hypothesis test, we have a chance of making a Type I error (the probability of this occurring is `\(\alpha\)`, normally 0.05), so our chance of making a Type I error naturally increases for each `\(t\)`-test -- * For that reason, we need to adjust `\(p\)`-values to account for this -- * There are different adjustment methods available, some of which are more common in different disciplines. We will be learning more about `\(p\)`-value adjustments in our next topic. -- * For our purposes, we will be using the Tukey adjustment. --- #Post-hoc tests ``` r Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = E ~ Region, data = personality) $Region diff lwr upr p adj W-S -10.804167 -19.337077 -2.2712564 0.0079753 MW-S 4.470833 -4.062077 13.0037436 0.5073225 NE-S -6.301389 -15.611557 3.0087795 0.2841492 MW-W 15.275000 6.152935 24.3970648 0.0003003 NE-W 4.502778 -5.350178 14.3557339 0.6181424 NE-MW -10.772222 -20.625178 -0.9192661 0.0272102 ``` --- #Post-hoc tests ``` r Tukey multiple comparisons of means 95% family-wise confidence level $Region diff lwr upr p adj `W-S -10.804167 -19.337077 -2.2712564 0.0079753` MW-S 4.470833 -4.062077 13.0037436 0.5073225 NE-S -6.301389 -15.611557 3.0087795 0.2841492 MW-W 15.275000 6.152935 24.3970648 0.0003003 NE-W 4.502778 -5.350178 14.3557339 0.6181424 NE-MW -10.772222 -20.625178 -0.9192661 0.0272102 ``` * Comparing West and South, there is a 10.8 difference in mean extraversion score (read from the `diff` column) -- * The 95% confidence interval is (-19.34, -2.27) (read from the <code>lwr</code> and <code>upr</code> columns) * Since this confidence interval does not include zero, we can conclude that the difference between these two groups is significantly different -- * Since `\(p = 0.008\)` is less than 0.05 (see <code>p adj</code> column), this leads us to the same conclusion -- * See if you can interpret the remaining pairwise results --- # Effect sizes * For a one-way ANOVA, we can use a measure called "eta squared" -- * "Eta" is a Greek letter: `\(\eta\)` -- * The eta squared ( `\(\eta^2\)` ) value is a measure of the proportion of variation in the response variable that can be attributed to the independent variable -- * For interpretation of eta squared, the following conventions apply (Cohen, 1988) .content-box-blue[ .center[ **Guidelines for interpreting eta squared effect sizes:** ] * `\(\eta^2 < 0.01\)`: "negligible" * `\(0.01 \leq \eta^2 < 0.06\)`: "small" * `\(0.06 \leq \eta^2 < 0.14\)`: "medium" * `\(\eta^2 \geq 0.14\)`: "large" ] --- # Effect sizes The results of the effect size calculation for our example are: ``` eta.sq eta.sq.part Region 0.3416801 0.3416801 ``` * The above output provides both the eta squared value (<code>eta.sq</code>), and the "partial eta squared" value (<code>eta.sq.part</code>) * For a one-way ANOVA, both values are equivalent. For more complicated types of ANOVAs (where there is more than one independent variable), the "partial eta squared" should be used. * As we can see, the effect size was 0.34, which is considered large --- # References Cohen, J. (1988). _Statistical power analysis for the behavioral sciences_. 2nd edition. New York: Academic Press. Costa, P. T. and R. R. McCrae (1992). _Neo personality inventory-revised (NEO PI-R)_. Psychological Assessment Resources Odessa, FL. Goldberg, L. R. (1992). "The development of markers for the Big-Five factor structure." In: _Psychological assessment_ 4.1, p. 26. Hartnett, J. (2020). _Not awful and boring ideas for teaching statistics_. URL: [https://notawfulandboring.blogspot.com/2020/04/online-day-6-one-way-anova-example.html](https://notawfulandboring.blogspot.com/2020/04/online-day-6-one-way-anova-example.html). Rentfrow, P. J., S. D. Gosling, M. Jokela, et al. (2013). "Divided we stand: three psychological regions of the United States and their political, economic, social, and health correlates." In: _Journal of personality and social psychology_ 105.6, p. 996. Wilson, C. and J. Cluger (2013). _America's Mood Map: An Interactive Guide to the United States of Attitude_. URL: [https://time.com/7612/americas-mood-map-an-interactive-guide-to-the-united-states-of-attitude/](https://time.com/7612/americas-mood-map-an-interactive-guide-to-the-united-states-of-attitude/). --- background-image: url(data:image/png;base64,#computerlab.jpg) background-position: bottom background-size: 75% class: center # See you in the computer labs! --- class: middle <font color = "grey"> These notes have been prepared by Amanda Shaker. The copyright for the material in these notes resides with the authors named above, with the Department of Mathematics and Statistics and with La Trobe University. Copyright in this work is vested in La Trobe University including all La Trobe University branding and naming. Unless otherwise stated, material within this work is licensed under a Creative Commons Attribution-Non Commercial-Non Derivatives License <a href = "https://creativecommons.org/licenses/by-nc-nd/4.0/" target="_blank"> BY-NC-ND. </a> </font>