INDEPENDENT T-TEST & MANN-WHITNEY U TEST

HYPOTHESIS TESTED:

Used to test if there is a difference between the means of two groups.

NULL HYPOTHESIS (H0)

The null hypothesis below is ALWAYS used.

There is no difference between the scores of Group A and Group B.

ALTERNATE HYPOTHESIS (H1)

Choose ONE of the three options below (based on your research scenario):

1) NON-DIRECTIONAL ALTERNATE HYPOTHESIS: There is a difference between the scores of Group A and Group B.

2) DIRECTIONAL ALTERNATE HYPOTHESES ONE: Group A has higher scores than Group B.

3) DIRECTIONAL ALTERNATE HYPOTHESIS TWO: Group B has higher scores than Group A.

QUESTION

What are the null and alternate hypotheses for YOUR research scenario?

H0:There is no difference between the scores of Human and AI.

H1:Human has higher scores than AI.

IMPORT EXCEL FILE

Purpose: Import your Excel dataset into R to conduct analyses.

INSTALL REQUIRED PACKAGE

If never installed, remove the hashtag before the install code.

If previously installed, leave the hashtag in front of the code.

#install.packages("readxl")

LOAD THE PACKAGE

Always reload the package you want to use.

library(readxl)

IMPORT EXCEL FILE INTO R STUDIO

Download the Excel file from One Drive and save it to your desktop.

In RStudio, replace the example path below with your actual path.

Replace backslashes with forward slashes / or double them //:

✘ WRONG “C:.xlsx”

✔ CORRECT “C:/Users/Joseph/Desktop/mydata.xlsx”

✔ CORRECT “C:\Users\Joseph\Desktop\mydata.xlsx”

Replace “dataset” with the name of your excel data (without the .xlsx)

A6R2 <- read_excel("/Users/mac/Downloads/A6R2.xlsx")

DESCRIPTIVE STATISTICS

PURPOSE: Calculate the mean, median, SD, and sample size for each group.

Human : mean(7.42), median(8), SD(1.43), and sample size(100).

AI : mean(3.60), median(3), SD(1.69), and sample size(100).

INSTALL REQUIRED PACKAGE

If never installed, remove the hashtag before the install code.

If previously installed, leave the hashtag in front of the code.

#install.packages(“dplyr”)

LOAD THE PACKAGE

Always reload the package you want to use.

library(dplyr)

## 
## Attaching package: 'dplyr'

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

filter, lag

The following objects are masked from ‘package:base’:

intersect, setdiff, setequal, union

CALCULATE THE DESCRIPTIVE STATISTICS

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

NOTE: Do NOT edit “group_by”

A6R2 %>%
  group_by(ServiceType) %>%
  summarise(
    Mean = mean(SatisfactionScore, na.rm = TRUE),
    Median = median(SatisfactionScore, na.rm = TRUE),
    SD = sd(SatisfactionScore, na.rm = TRUE),
    N = n()
  )

## # A tibble: 2 × 5
##   ServiceType  Mean Median    SD     N
##   <chr>       <dbl>  <dbl> <dbl> <int>
## 1 AI           3.6       3  1.60   100
## 2 Human        7.42      8  1.44   100

HISTOGRAMS

Purpose: Visually check the normality of the scores for each group.

#Both are not normal

CREATE THE HISTOGRAMS

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

Replace “Group1” with the R code name for your first group (example: USA)

Replace “Group2” with the R code name for your second group (example: India)

hist(A6R2$SatisfactionScore[A6R2$ServiceType == "Human"],
     main = "Histogram of Human Scores",
     xlab = "Value",
     ylab = "Frequency",
     col = "lightblue",
     border = "black",
     breaks = 20)

hist(A6R2$SatisfactionScore[A6R2$ServiceType == "AI"],
     main = "Histogram of AI Scores",
     xlab = "Value",
     ylab = "Frequency",
     col = "lightgreen",
     border = "black",
     breaks = 20)

# QUESTIONS # Answer the questions below as comments within the R script:

Q1) Check the SKEWNESS of the VARIABLE 1 histogram. In your opinion, does the histogram look symmetrical, positively skewed, or negatively skewed?

#negatively skewed

Q2) Check the KURTOSIS of the VARIABLE 1 histogram. In your opinion, does the histogram look too flat, too tall, or does it have a proper bell curve?

#tall

Q3) Check the SKEWNESS of the VARIABLE 2 histogram. In your opinion, does the histogram look symmetrical, positively skewed, or negatively skewed?

#Positively skewed

Q4) Check the KUROTSIS of the VARIABLE 2 histogram. In your opinion, does the histogram look too flat, too tall, or does it have a proper bell curve?

#tall

SHAPIRO-WILK TEST

Purpose: Check the normality for each group’s score statistically.

The Shapiro-Wilk Test is a test that checks skewness and kurtosis at the same time.

The test is checking “Is this variable the SAME as normal data (null hypothesis) or DIFFERENT from normal data (alternate hypothesis)?”

For this test, if p is GREATER than .05 (p > .05), the data is NORMAL.

If p is LESS than .05 (p < .05), the data is NOT normal.

not normal

CONDUCT THE SHAPIRO-WILK TEST

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

Replace “Group1” with the R code name for your first group (example: USA)

Replace “Group2” with the R code name for your second group (example: India)

shapiro.test(A6R2$SatisfactionScore[A6R2$ServiceType == "Human"])

## 
##  Shapiro-Wilk normality test
## 
## data:  A6R2$SatisfactionScore[A6R2$ServiceType == "Human"]
## W = 0.93741, p-value = 0.0001344

shapiro.test(A6R2$SatisfactionScore[A6R2$ServiceType == "AI"])

## 
##  Shapiro-Wilk normality test
## 
## data:  A6R2$SatisfactionScore[A6R2$ServiceType == "AI"]
## W = 0.91143, p-value = 5.083e-06

QUESTION

Answer the questions below as a comment within the R script:

Was the data normally distributed for Variable 1?

#not normal # Was the data normally distributed for Variable 2? #not normal

If p > 0.05 (P-value is GREATER than .05) this means the data is NORMAL. Continue to the box-plot test below.

If p < 0.05 (P-value is LESS than .05) this means the data is NOT normal (switch to Mann-Whitney U).

BOXPLOT

Purpose: Check for any outliers impacting the mean for each group’s scores.

#yes

INSTALL REQUIRED PACKAGE

If previously installed, put a hashtag in front of the code.

#install.packages(“ggplot2”) #install.packages(“ggpubr”)

LOAD THE PACKAGE

Always reload the package you want to use.

library(ggplot2)
library(ggpubr)

CREATE THE BOXPLOT

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

ggboxplot(A6R2, x = "ServiceType", y = "SatisfactionScore",
          color = "ServiceType",
          palette = "jco",
          add = "jitter")

# QUESTION # Answer the questions below as a comment within the R script: # Q1) Were there any dots outside of the boxplots? These dots represent participants with extreme scores. #yes, there are more outside dots

Q2) If there are outliers, in your opinion are the scores of those dots changing the mean so much that the mean no longer accurately represents the average score?

#yes

If there were no extreme outliers, this means the data is NORMAL. Continue to the Independent t-test.

If there WERE any extreme outliers, this means the data is NOT abnormal. Switch to the Mann-Whitney U test.

MANN-WHITNEY U TEST

PURPOSE: Test if there was a difference between the distributions of the two groups.

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

wilcox.test(SatisfactionScore ~ ServiceType, data = A6R2, exact = FALSE)

## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  SatisfactionScore by ServiceType
## W = 497, p-value < 2.2e-16
## alternative hypothesis: true location shift is not equal to 0

DETERMINE STATISTICAL SIGNIFICANCE

If results were statistically significant (p < .05), continue to effect size section below.

NOTE: The Mann-Whitney U test is used when your data is abnormally distributed

or when assumptions of the t-test are not met.

It is not chosen based on whether the t-test was significant.

EFFECT-SIZE

PURPOSE: Determine how big of a difference there was between the group distributions.

INSTALL REQUIRED PACKAGE

If never installed, remove the hashtag before the install code.

If previously installed, leave the hashtag in front of the code.

#install.packages(“effectsize”)

LOAD THE PACKAGE

Always load the package you want to use.

library(effectsize)

CALCULATE EFFECT SIZE (R VALUE)

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

mw_results <- wilcox.test(SatisfactionScore ~ ServiceType, data = A6R2, exact = FALSE)

#rank_biserial(mw_results) #rb(mw_results)

rank_biserial(SatisfactionScore ~ ServiceType, data = A6R2)

## r (rank biserial) |         95% CI
## ----------------------------------
## -0.90             | [-0.93, -0.87]

QUESTIONS

Answer the questions below as a comment within the R script:

Q1) What is the size of the effect?

The effect means how big or small was the difference between the two groups.

± 0.00 to 0.10 = ignore

± 0.10 to 0.30 = small

± 0.30 to 0.50 = moderate

± 0.50 to + = large

Example 1) A rank-biserial correlation of 0.05 indicates the difference between the groups was not meaningful. There was no effect.

Example 2) A rank-biserial correlation of 0.32 indicates the difference between the groups was moderate.

#That means there is a very strong difference between groups.(The negative sign just tells you the direction — which group had higher scores.) #THere is very strong dufference between the groups.

Q2) Which group had the higher average rank?

The Mann-Whitney U test does not compare means directly. Instead, it looks at whether one group tends to have higher scores than the other.

To determine which group ranked higher, look at the group means or medians in your dataset.

#Human grouped ranked higher, the mean and median are higher compared to the AI.

WRITTEN REPORT FOR MANN-WHITNEY U TEST

Write a paragraph summarizing your findings.

1) REVIEW YOUR OUTPUT

Collect the information below from your output:

1. The name of the inferential test used (Mann-Whitney U test)

2. The names of the IV and DV (their proper names, not their R code names).

3. The sample size for each group (labeled as “n”).

4. Whether the inferential test results were statistically significant (p < .05) or not (p > .05).

5. The median for each group’s score on the DV (rounded to two places after the decimal).

6. U statistic (from output).

7. EXACT p-value to three decimals. NOTE: If p > .05, just report p > .05 If p < .001, just report p < .001

8. Effect size (rank-biserial correlation) ** Only if the results were significant.

2) REPORT YOUR DATA AS A PARAGRAPH

An example report is provided below. You should copy the paragraph and just edit/ replace words with your information.

This is not considered plagiarizing because science has a specific format for reporting information.

EXAMPLE

A Mann-Whitney U test was conducted to compare

exam scores between students who attended a review session (n = 60) and students who did not (n = 60).

Students who attended the review session had significantly higher median scores (Mdn = 86.00) than

students who did not attend a review session (Mdn = 78.00), U = 1280, p = 0.01.

The effect size was moderate (r = 0.32), indicating a meaningful difference between student exam scores.

Overall, attending the review session resulted in higher exam performance.

Summary Paragraph: The Mann-Whitney U test was selected to show a comparison of the level of satisfaction between those participants who engaged with Human service (n = 100) and those participants who would engage an AI service (n = 100). Findings showed a statistically significant difference between the two groups and W = 497, and the p <. 001. The respondents in the Human service condition had a much higher level of satisfaction (Median = 8) than those in the AI service condition (Median = 3). This effect size was quite big (r = -0.90), which is a clear difference in the ratings of satisfaction, whereby Human service had the highest rating of satisfaction ratings compared against AI service.

INDEPENDENT T-TEST & MANN-WHITNEY U TEST

HYPOTHESIS TESTED:

Used to test if there is a difference between the means of two groups.

NULL HYPOTHESIS (H0)

The null hypothesis below is ALWAYS used.

There is no difference between the scores of Group A and Group B.

ALTERNATE HYPOTHESIS (H1)

Choose ONE of the three options below (based on your research scenario):

1) NON-DIRECTIONAL ALTERNATE HYPOTHESIS: There is a difference between the scores of Group A and Group B.

2) DIRECTIONAL ALTERNATE HYPOTHESES ONE: Group A has higher scores than Group B.

3) DIRECTIONAL ALTERNATE HYPOTHESIS TWO: Group B has higher scores than Group A.

QUESTION

What are the null and alternate hypotheses for YOUR research scenario?

H0:There is no difference between the scores of Human and AI.

H1:Human has higher scores than AI.

IMPORT EXCEL FILE

Purpose: Import your Excel dataset into R to conduct analyses.

INSTALL REQUIRED PACKAGE

If never installed, remove the hashtag before the install code.

If previously installed, leave the hashtag in front of the code.

LOAD THE PACKAGE

Always reload the package you want to use.

IMPORT EXCEL FILE INTO R STUDIO

Download the Excel file from One Drive and save it to your desktop.

Right-click the Excel file and click “Copy as path” from the menu.

In RStudio, replace the example path below with your actual path.

Replace backslashes with forward slashes / or double them //:

✘ WRONG “C:.xlsx”

✔ CORRECT “C:/Users/Joseph/Desktop/mydata.xlsx”

✔ CORRECT “C:\Users\Joseph\Desktop\mydata.xlsx”

Replace “dataset” with the name of your excel data (without the .xlsx)

DESCRIPTIVE STATISTICS

PURPOSE: Calculate the mean, median, SD, and sample size for each group.

Human : mean(7.42), median(8), SD(1.43), and sample size(100).

AI : mean(3.60), median(3), SD(1.69), and sample size(100).

INSTALL REQUIRED PACKAGE

If never installed, remove the hashtag before the install code.

If previously installed, leave the hashtag in front of the code.

LOAD THE PACKAGE

Always reload the package you want to use.

Attaching package: ‘dplyr’

The following objects are masked from ‘package:stats’:

filter, lag

The following objects are masked from ‘package:base’:

intersect, setdiff, setequal, union

CALCULATE THE DESCRIPTIVE STATISTICS

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

NOTE: Do NOT edit “group_by”

HISTOGRAMS

Purpose: Visually check the normality of the scores for each group.

CREATE THE HISTOGRAMS

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

Replace “Group1” with the R code name for your first group (example: USA)

Replace “Group2” with the R code name for your second group (example: India)

Q1) Check the SKEWNESS of the VARIABLE 1 histogram. In your opinion, does the histogram look symmetrical, positively skewed, or negatively skewed?

Q2) Check the KURTOSIS of the VARIABLE 1 histogram. In your opinion, does the histogram look too flat, too tall, or does it have a proper bell curve?

Q3) Check the SKEWNESS of the VARIABLE 2 histogram. In your opinion, does the histogram look symmetrical, positively skewed, or negatively skewed?

Q4) Check the KUROTSIS of the VARIABLE 2 histogram. In your opinion, does the histogram look too flat, too tall, or does it have a proper bell curve?

SHAPIRO-WILK TEST

Purpose: Check the normality for each group’s score statistically.

The Shapiro-Wilk Test is a test that checks skewness and kurtosis at the same time.

The test is checking “Is this variable the SAME as normal data (null hypothesis) or DIFFERENT from normal data (alternate hypothesis)?”

For this test, if p is GREATER than .05 (p > .05), the data is NORMAL.

If p is LESS than .05 (p < .05), the data is NOT normal.

not normal

CONDUCT THE SHAPIRO-WILK TEST

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

Replace “Group1” with the R code name for your first group (example: USA)

Replace “Group2” with the R code name for your second group (example: India)

QUESTION

Answer the questions below as a comment within the R script:

Was the data normally distributed for Variable 1?

If p > 0.05 (P-value is GREATER than .05) this means the data is NORMAL. Continue to the box-plot test below.

If p < 0.05 (P-value is LESS than .05) this means the data is NOT normal (switch to Mann-Whitney U).