Homework2_Applied Data Analysis_StephanieYang
Stephanie Yang
2024-04-18
Adding necessary packages to library
library(pastecs)
library(ggplot2)
library(tidyr)##
## Attaching package: 'tidyr'## The following object is masked from 'package:pastecs':
##
## extractlibrary(car)## Loading required package: carDatalibrary(psych)##
## Attaching package: 'psych'## The following object is masked from 'package:car':
##
## logit## The following objects are masked from 'package:ggplot2':
##
## %+%, alphalibrary(ggpubr)
library(rstatix)##
## Attaching package: 'rstatix'## The following object is masked from 'package:stats':
##
## filterlibrary(dplyr)##
## Attaching package: 'dplyr'## The following object is masked from 'package:car':
##
## recode## The following objects are masked from 'package:pastecs':
##
## first, last## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, unionlibrary(effectsize)##
## Attaching package: 'effectsize'## The following objects are masked from 'package:rstatix':
##
## cohens_d, eta_squared## The following object is masked from 'package:psych':
##
## phi1. Research Question
Is there a difference in the medical costs billed by health insurance between smokers and non-smokers?
We will need to look at the charges and the smoker status of each unit.
I predict that smokers will have higher charges than non-smokers.
2 and 3. Importing Dataset
#Read in data
myData <- read.table("./Medical_Cost.csv", header = TRUE, sep = ",", dec = ".")4. Display of my Data
head(myData)## age sex bmi children smoker region charges
## 1 19 female 27.900 0 yes southwest 16884.924
## 2 18 male 33.770 1 no southeast 1725.552
## 3 28 male 33.000 3 no southeast 4449.462
## 4 33 male 22.705 0 no northwest 21984.471
## 5 32 male 28.880 0 no northwest 3866.855
## 6 31 female 25.740 0 no southeast 3756.6225. Explaining the Data
Variables:
- age: Age of the primary beneficiary (years)
- sex: Gender of primary beneficiary
- bmi: Body mass index, an index that compares height to weight (kg / m ^ 2)
- children: Number of children covered by health insurance / Number of dependents
- smoker: Whether beneficiary smokes or not
- region: The beneficiary’s residential area in the US
- charges: Individual medical costs billed by health insurance ($USD)
Unit: A single beneficiary (person with insurance)
Sample size: 1338 beneficiaries
6. Source of Data
Source Link: https://www.kaggle.com/datasets/pritambhakat/medical-cost-dataset
7. Cleaning the data
#Cast Gender, smoker, and region as factor
myData$GenderFactor <- as.factor(myData$sex)
myData$SmokerFactor <- as.factor(myData$smoker)
myData$RegionFactor <- as.factor(myData$region)
str(myData)## 'data.frame': 1338 obs. of 10 variables:
## $ age : int 19 18 28 33 32 31 46 37 37 60 ...
## $ sex : chr "female" "male" "male" "male" ...
## $ bmi : num 27.9 33.8 33 22.7 28.9 ...
## $ children : int 0 1 3 0 0 0 1 3 2 0 ...
## $ smoker : chr "yes" "no" "no" "no" ...
## $ region : chr "southwest" "southeast" "southeast" "northwest" ...
## $ charges : num 16885 1726 4449 21984 3867 ...
## $ GenderFactor: Factor w/ 2 levels "female","male": 1 2 2 2 2 1 1 1 2 1 ...
## $ SmokerFactor: Factor w/ 2 levels "no","yes": 2 1 1 1 1 1 1 1 1 1 ...
## $ RegionFactor: Factor w/ 4 levels "northeast","northwest",..: 4 3 3 2 2 3 3 2 1 2 ...#Removing NAs (there are no NAs)
cleanedData <- drop_na(myData)
#View Cleaned Data
head(cleanedData)## age sex bmi children smoker region charges GenderFactor
## 1 19 female 27.900 0 yes southwest 16884.924 female
## 2 18 male 33.770 1 no southeast 1725.552 male
## 3 28 male 33.000 3 no southeast 4449.462 male
## 4 33 male 22.705 0 no northwest 21984.471 male
## 5 32 male 28.880 0 no northwest 3866.855 male
## 6 31 female 25.740 0 no southeast 3756.622 female
## SmokerFactor RegionFactor
## 1 yes southwest
## 2 no southeast
## 3 no southeast
## 4 no northwest
## 5 no northwest
## 6 no southeast8. Hypothesis Test
Independent t-test with Welch correction
We have to preform an independent t-test with Welch corrections because we are comparing the mean charge amount for 2 groups of units that do not overlap. You cannot be a smoker and non-smoker. The hypothesises are below:
H0: µSmoker - µNonSmoker = 0
H1: µSmoker - µNonSmoker ≠ 0
Assumptions:
- Variable is numeric
- Normality: The distribution of the variable is normal in both populations
- The data must come from two independent populations
- Variable has the same variance in both populations
describeBy(cleanedData$charges, cleanedData$SmokerFactor)##
## Descriptive statistics by group
## group: no
## vars n mean sd median trimmed mad min max range
## X1 1 1064 8434.27 5993.78 7345.41 7599.76 5477.15 1121.87 36910.61 35788.73
## skew kurtosis se
## X1 1.53 3.12 183.75
## ------------------------------------------------------------
## group: yes
## vars n mean sd median trimmed mad min max
## X1 1 274 32050.23 11541.55 34456.35 31782.89 15167.19 12829.46 63770.43
## range skew kurtosis se
## X1 50940.97 0.13 -1.05 697.25From the descriptive statistics, you can see that smokers have a higher mean charge than non-smokers. You can also see that the standard deviation is higher for smokers than non-smokers.
Check for Normality
Smoker_no <- ggplot(cleanedData[cleanedData$SmokerFactor == "no", ], aes(x = charges)) +
theme_linedraw() +
geom_histogram() +
ggtitle("non-smoker")
Smoker_yes <- ggplot(cleanedData[cleanedData$SmokerFactor == "yes", ], aes(x = charges)) +
theme_linedraw() +
geom_histogram() +
ggtitle("smoker")
ggarrange(Smoker_no, Smoker_yes,
ncol = 2, nrow = 1)## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
It looks like neither smokers or non-smokers have a normal distribution. Non-smokers are skewed right and smokers seem to have a bimodal distribution of charges. We can check using a Shapiro-Wilk test.
Shapiro-Wilk Test
H0: The distribution of charges is normally distributed
H1: The distribution of charges is not normally distributed
cleanedData %>%
group_by(SmokerFactor) %>%
shapiro_test(charges)## # A tibble: 2 × 4
## SmokerFactor variable statistic p
## <fct> <chr> <dbl> <dbl>
## 1 no charges 0.873 1.45e-28
## 2 yes charges 0.940 3.62e- 9Results of Shapiro-Wilk test:
Both p-values for smokers and non-smokers are less than .001. This means we reject the null hypothesis for both non-smokers and smokers at p < 0.001. This means they are not normally distributed and we use a non-parametric test.
Wilcoxon Rank Sum Test
H0: Locations of distribution of charges is the same for non-smokers and smokers.
H1: Locations of distribution of charges is not the same for non-smokers and smokers.
wilcox.test(cleanedData$charges ~ cleanedData$SmokerFactor,
paired = FALSE,
correct = FALSE,
exact = FALSE,
alternative = "two.sided")##
## Wilcoxon rank sum test
##
## data: cleanedData$charges by cleanedData$SmokerFactor
## W = 7403, p-value < 2.2e-16
## alternative hypothesis: true location shift is not equal to 0Results of Wilcoxon Rank Sum Test
The p-value is equal to 2.2 * 10^-16. We reject the null hypothesis at p < 0.001. This means that there is a significant difference between the medical insurance charges between non-smoker and smokers.
Effect Size
effectsize(wilcox.test(cleanedData$charges ~ cleanedData$SmokerFactor,
paired = FALSE,
correct = FALSE,
exact = FALSE,
alternative = "two.sided"))## r (rank biserial) | 95% CI
## ----------------------------------
## -0.95 | [-0.96, -0.94]interpret_rank_biserial(0.95)## [1] "very large"
## (Rules: funder2019)Based off of the calculated effect size, you can see that smokers have higher charges and the difference in distributions between smokers and non-smokers is very large.
Conclusion
Based on the sample data, I find that non-smokers and smokers significantly differ in terms of individual medical costs billed by health insurance at p < 0.001. Smokers pay more in charges than non-smokers. The difference in distribution is very large (r = .95)