Scenario 2 A customer service firm wants to test whether customer satisfaction scores differ between those served by human agents versus those served by an AI chatbot. After interactions, customers rate their satisfaction (single-item rating scale). Is there a difference in the average satisfaction scores of the two groups? # INDEPENDENT T-TEST & MANN-WHITNEY U TEST

HYPOTHESIS TESTED:

Used to test if there is a difference between the means of two groups.

NULL HYPOTHESIS (H0)

The null hypothesis below is ALWAYS used.

There is no difference between the scores of Group A and Group B.

ALTERNATE HYPOTHESIS (H1)

Choose ONE of the three options below (based on your research scenario):

1) NON-DIRECTIONAL ALTERNATE HYPOTHESIS: There is a difference between the scores of Group A and Group B.

2) DIRECTIONAL ALTERNATE HYPOTHESES ONE: Group A has higher scores than Group B.

3) DIRECTIONAL ALTERNATE HYPOTHESIS TWO: Group B has higher scores than Group A.

QUESTION

What are the null and alternate hypotheses for YOUR research scenario?

H0:

H1:

IMPORT EXCEL FILE

Purpose: Import your Excel dataset into R to conduct analyses.

INSTALL REQUIRED PACKAGE

If never installed, remove the hashtag before the install code.

If previously installed, leave the hashtag in front of the code.

options(repos = c(CRAN = "https://cloud.r-project.org"))
install.packages("readxl")
## Installing package into 'C:/Users/N Geetha Shivani/AppData/Local/R/win-library/4.5'
## (as 'lib' is unspecified)
## package 'readxl' successfully unpacked and MD5 sums checked
## 
## The downloaded binary packages are in
##  C:\Users\N Geetha Shivani\AppData\Local\Temp\RtmpIvZKnB\downloaded_packages

LOAD THE PACKAGE

Always reload the package you want to use.

library(readxl)

IMPORT EXCEL FILE INTO R STUDIO

Download the Excel file from One Drive and save it to your desktop.

Right-click the Excel file and click “Copy as path” from the menu.

In RStudio, replace the example path below with your actual path.

Replace backslashes  with forward slashes / or double them //:

✘ WRONG “C:.xlsx”

✔ CORRECT “C:/Users/Joseph/Desktop/mydata.xlsx”

✔ CORRECT “C:\Users\Joseph\Desktop\mydata.xlsx”

Replace “dataset” with the name of your excel data (without the .xlsx)

dataset <- read_excel("C:\\Users\\N Geetha Shivani\\Downloads\\A6R2.xlsx")

DESCRIPTIVE STATISTICS

PURPOSE: Calculate the mean, median, SD, and sample size for each group.

INSTALL REQUIRED PACKAGE

If never installed, remove the hashtag before the install code.

If previously installed, leave the hashtag in front of the code.

install.packages("dplyr")
## Installing package into 'C:/Users/N Geetha Shivani/AppData/Local/R/win-library/4.5'
## (as 'lib' is unspecified)
## package 'dplyr' successfully unpacked and MD5 sums checked
## 
## The downloaded binary packages are in
##  C:\Users\N Geetha Shivani\AppData\Local\Temp\RtmpIvZKnB\downloaded_packages

LOAD THE PACKAGE

Always reload the package you want to use.

library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

CALCULATE THE DESCRIPTIVE STATISTICS

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

NOTE: Do NOT edit “group_by”

dataset %>%
  group_by(ServiceType) %>%
  summarise(
    Mean = mean(SatisfactionScore, na.rm = TRUE),
    Median = median(SatisfactionScore, na.rm = TRUE),
    SD = sd(SatisfactionScore, na.rm = TRUE),
    N = n()
  )
## # A tibble: 2 × 5
##   ServiceType  Mean Median    SD     N
##   <chr>       <dbl>  <dbl> <dbl> <int>
## 1 AI           3.6       3  1.60   100
## 2 Human        7.42      8  1.44   100

HISTOGRAMS

Purpose: Visually check the normality of the scores for each group.

CREATE THE HISTOGRAMS

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

Replace “Group1” with the excel group name for your first group (example: USA)

Replace “Group2” with the excel group name for your second group (example: India)

hist(dataset$SatisfactionScore[dataset$ServiceType == "Human"],
     main = "Histogram of Human  Scores",
     xlab = "Value",
     ylab = "Frequency",
     col = "lightblue",
     border = "black",
     breaks = 20)

hist(dataset$SatisfactionScore[dataset$ServiceType == "AI"],
     main = "Histogram of AI  Scores",
     xlab = "Value",
     ylab = "Frequency",
     col = "lightgreen",
     border = "black",
     breaks = 20)

QUESTIONS

Answer the questions below as comments within the R script:

Q1) Check the SKEWNESS of the VARIABLE 1 histogram. In your opinion, does the histogram look symmetrical, positively skewed, or negatively skewed?

Q2) Check the KURTOSIS of the VARIABLE 1 histogram. In your opinion, does the histogram look too flat, too tall, or does it have a proper bell curve?

Q3) Check the SKEWNESS of the VARIABLE 2 histogram. In your opinion, does the histogram look symmetrical, positively skewed, or negatively skewed?

Q4) Check the KUROTSIS of the VARIABLE 2 histogram. In your opinion, does the histogram look too flat, too tall, or does it have a proper bell curve?

SHAPIRO-WILK TEST

Purpose: Check the normality for each group’s score statistically.

The Shapiro-Wilk Test is a test that checks skewness and kurtosis at the same time.

The test is checking “Is this variable the SAME as normal data (null hypothesis) or DIFFERENT from normal data (alternate hypothesis)?”

For this test, if p is GREATER than .05 (p > .05), the data is NORMAL.

If p is LESS than .05 (p < .05), the data is NOT normal.

CONDUCT THE SHAPIRO-WILK TEST

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

Replace “Group1” with the R code name for your first group (example: USA)

Replace “Group2” with the R code name for your second group (example: India)

shapiro.test(dataset$SatisfactionScore[dataset$ServiceType == "Human"])
## 
##  Shapiro-Wilk normality test
## 
## data:  dataset$SatisfactionScore[dataset$ServiceType == "Human"]
## W = 0.93741, p-value = 0.0001344
shapiro.test(dataset$SatisfactionScore[dataset$ServiceType == "AI"])
## 
##  Shapiro-Wilk normality test
## 
## data:  dataset$SatisfactionScore[dataset$ServiceType == "AI"]
## W = 0.91143, p-value = 5.083e-06

MANN-WHITNEY U TEST

PURPOSE: Test if there was a difference between the distributions of the two groups.

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable excel name (example: USD)

Replace “group” with your independent variable excel name (example: Country)

wilcox.test(SatisfactionScore ~ ServiceType, data = dataset, exact = FALSE)
## 
##  Wilcoxon rank sum test with continuity correction
## 
## data:  SatisfactionScore by ServiceType
## W = 497, p-value < 2.2e-16
## alternative hypothesis: true location shift is not equal to 0

DETERMINE STATISTICAL SIGNIFICANCE

If results were statistically significant (p < .05), continue to effect size section below.

If results were NOT statistically significant (p > .05), skip to reporting section below.

NOTE: The Mann-Whitney U test is used when your data is abnormally distributed

or when assumptions of the t-test are not met.

It is not chosen based on whether the t-test was significant.

EFFECT-SIZE

PURPOSE: Determine how big of a difference there was between the group distributions.

INSTALL REQUIRED PACKAGE

If never installed, remove the hashtag before the install code.

If previously installed, leave the hashtag in front of the code.

install.packages("effectsize")
## Installing package into 'C:/Users/N Geetha Shivani/AppData/Local/R/win-library/4.5'
## (as 'lib' is unspecified)
## package 'effectsize' successfully unpacked and MD5 sums checked
## 
## The downloaded binary packages are in
##  C:\Users\N Geetha Shivani\AppData\Local\Temp\RtmpIvZKnB\downloaded_packages

LOAD THE PACKAGE

Always load the package you want to use.

library(effectsize)

CALCULATE EFFECT SIZE (R VALUE)

Replace “dataset” with your excel dataset name (without .xlsx)

Replace “score” with your dependent variable excel name (example: USD)

Replace “group” with your independent variable excel name (example: Country)

rank_biserial(SatisfactionScore ~ ServiceType, data = dataset, exact = FALSE)
## r (rank biserial) |         95% CI
## ----------------------------------
## -0.90             | [-0.93, -0.87]

QUESTIONS

Answer the questions below as a comment within the R script:

Q1) What is the size of the effect?

The effect means how big or small was the difference between the two groups.

± 0.00 to 0.10 = ignore

± 0.10 to 0.30 = small

± 0.30 to 0.50 = moderate

± 0.50 to + = large

Example 1) A rank-biserial correlation of 0.05 indicates the difference between the groups was not meaningful. There was no effect.

Example 2) A rank-biserial correlation of 0.32 indicates the difference between the groups was moderate.

Q2) Which group had the higher average rank?

The Mann-Whitney U test does not compare means directly. Instead, it looks at whether one group tends to have higher scores than the other.

To determine which group ranked higher, look at the group means or medians in your dataset.

WRITTEN REPORT FOR MANN-WHITNEY U TEST

Write a paragraph summarizing your findings.

1) REVIEW YOUR OUTPUT

Collect the information below from your output:

1. The name of the inferential test used (Mann-Whitney U test)

2. The names of the IV and DV (their proper names, which may not be their excel names).

3. The sample size for each group (labeled as “n”).

4. Whether the inferential test results were statistically significant (p < .05) or not (p > .05).

5. The median for each group’s score on the DV (rounded to two places after the decimal).

6. U statistic (from output).

7. EXACT p-value to three decimals. NOTE: If p > .05, just report p > .05 If p < .001, just report p < .001

8. Effect size (rank-biserial correlation) ** Only if the results were significant.

2) REPORT YOUR DATA AS A PARAGRAPH

An example report is provided below. You should copy the paragraph and just edit/ replace words with your information.

This is not considered plagiarizing because science has a specific format for reporting information.

EXAMPLE

A Mann-Whitney U test was conducted to compare

exam scores between students who attended a review session (n = 60) and students who did not (n = 60).

Students who attended the review session had significantly higher median scores (Mdn = 86.00) than

students who did not attend a review session (Mdn = 78.00), U = 1280, p = 0.01.

The effect size was moderate (r = 0.32), indicating a meaningful difference between student exam scores.

Overall, attending the review session resulted in higher exam performance.

QUESTION

Answer the questions below as a comment within the R script:

Was the data normally distributed for Variable 1?

Was the data normally distributed for Variable 2?

If p > 0.05 (P-value is GREATER than .05) this means the data is NORMAL.

If p < 0.05 (P-value is LESS than .05) this means the data is NOT normal.

BOXPLOT

Purpose: Check for any outliers impacting the mean for each group’s scores.

INSTALL REQUIRED PACKAGE

If previously installed, put a hashtag in front of the code.

install.packages("ggplot2")
## Installing package into 'C:/Users/N Geetha Shivani/AppData/Local/R/win-library/4.5'
## (as 'lib' is unspecified)
## package 'ggplot2' successfully unpacked and MD5 sums checked
## 
## The downloaded binary packages are in
##  C:\Users\N Geetha Shivani\AppData\Local\Temp\RtmpIvZKnB\downloaded_packages
install.packages("ggpubr")
## Installing package into 'C:/Users/N Geetha Shivani/AppData/Local/R/win-library/4.5'
## (as 'lib' is unspecified)
## package 'ggpubr' successfully unpacked and MD5 sums checked
## 
## The downloaded binary packages are in
##  C:\Users\N Geetha Shivani\AppData\Local\Temp\RtmpIvZKnB\downloaded_packages

LOAD THE PACKAGE

Always reload the package you want to use.

library(ggplot2)
library(ggpubr)

CREATE THE BOXPLOT

Replace “dataset” with your dataset name (without .xlsx)

Replace “score” with your dependent variable R code name (example: USD)

Replace “group” with your independent variable R code name (example: Country)

ggboxplot(dataset, x = "ServiceType", y = "SatisfactionScore",
          color = "ServiceType",
          palette = "jco",
          add = "jitter")

QUESTION

Answer the questions below as a comment within the R script. Answer the questions for EACH boxplot:

Q1) Were there any dots outside of the boxplot? Are these dots close to the whiskers of the boxplot (check if there are any dots past the lines on the boxes) or are they very far away?

If there are no dots, continue with Independent t-test.

If there are a few dots (two or less), and they are close to the whiskers, continue with the Independent t-test.

If there are a few dots (two or less), and they are far away from the whiskers, consider switching to Mann Whitney U test.

If there are many dots (more than one or two) and they are very far away from the whiskers, you should switch to the Mann Whitney U test.