RPubs link information

• RPubs link - https://rpubs.com/Nawazsyed123/S4239096

Introduction

For insurers, policymakers and individuals, medical costs becomes one of the major concerns. Analysing and understanding what drivers expenditure of healthcare that supports with pricing, risk assessment and preventive health strategies. This analysis uses the medical cost personal dataset to analyse and investigate different factors associated with individual medical charges.

Introduction Cont.

• Focused key question for this analysis includes,

  1. Do smokers have a different mean medical charge than non-smokers?

  2. How well do smoking status, age and BMI together explain charges?

  The term medical charges focuses on individual medical cost which as billed by the health insurance. BMI (Body Mass Index) is a measure of body weight that is relative to height. Smoking status represents whether the beneficiary is non-smoker or smoker. The rationale is to quantify the financial impact of smoking, risk factor and assess on how well simple linear model in charges.

Problem Statement

• Do smokers have a different mean medical charge than non-smokers?
• How well do smoking status, age and BMI together explain charges?
• Tasks 5 and 7 were chosen

Data

• The selected dataset is collected from Kaggle which is an open dataset “Medical Cost Personal Datasets”. The dataset was compiled for the Machine Learning with R by Brett Lantz. The data has 1338 observations and 7 variables(Choi, 2026).

• Used two sample t-test which supports to compare mean charges, though used multiple linear regression model using log transformed charges. This evaluates integrated relationship between medical cost and explanatory variables.

# insurance dataset
insurance_csv_dt <- read.csv("insurance.csv", stringsAsFactors = TRUE)
dim(insurance_csv_dt)

## [1] 1338    7

Data Cont.

• The variables are sex, age, BMI, region, smoker, children and charges. Each observation indicates individuals information regarding medical insurance.
• The main variable is charges which indicates medical cost that billed by health insurance. The explanatory variables are smoker (yes/ no), age (years) and BMI (body mass index in kg/m²). The other variable shows residential region and demographic characteristics.
• The dataset was imported into R and checked the missing values and determined that there is no missing values found. Checked and removed duplicate observations. Smoking status was releveled so that non-smokers were formed the category for analysis. Prepared steps for descriptive analysis, hypothesis and regression modelling.

Descriptive Statistics and Visualisation

• Summary statistics for numeric variables determines that average age is 39.22 years, mean BMI is 30.66 and the mean charges are $13,279 and median charge is $9,386 which is lower than mean. This represents right skewness(Soetewey, 2020).

# missing values
colSums(is.na(insurance_csv_dt))

##      age      sex      bmi children   smoker   region  charges 
##        0        0        0        0        0        0        0

• There are no missing values were found. Removed duplicate rows

# duplicate rows
duplicate_rows <- sum(duplicated(insurance_csv_dt))

# remove duplicates
insurance_csv_dt <- distinct(insurance_csv_dt)

# non-smoker
insurance_csv_dt$smoker <- relevel(insurance_csv_dt$smoker, ref = "no")  

# dataset summary
summary(insurance_csv_dt[, c("age", "bmi", "children", "charges")])

##       age             bmi           children        charges     
##  Min.   :18.00   Min.   :15.96   Min.   :0.000   Min.   : 1122  
##  1st Qu.:27.00   1st Qu.:26.29   1st Qu.:0.000   1st Qu.: 4746  
##  Median :39.00   Median :30.40   Median :1.000   Median : 9386  
##  Mean   :39.22   Mean   :30.66   Mean   :1.096   Mean   :13279  
##  3rd Qu.:51.00   3rd Qu.:34.70   3rd Qu.:2.000   3rd Qu.:16658  
##  Max.   :64.00   Max.   :53.13   Max.   :5.000   Max.   :63770

• The histogram shows that non-smoker has high right skewed distribution which is below $20,000, though smokers has high and flatter distribution that extends beyond $40,000. So, determined that smoking is associated with high medical charges(Emily Nordmann et al., 2022.

# Medical charges basedon smoker
ggplot(insurance_csv_dt, aes(x = charges)) +
  geom_histogram(bins = 30, fill = "steelblue", colour = "white") +
  facet_wrap(~ smoker, labeller = label_both, scales = "free_y") +
  labs(x = "Medical charges", y = "Count") +
  theme_minimal()

• The boxplot shows that median charge for smokers are around $34,457 compared to non-smokers of $7,346. The interquartile range for smokers is larger and represents high variability in charges. Also, there are high value outlier among non-smokers but still below the smokers charges.

ggplot(insurance_csv_dt, aes(x = smoker, y = charges, fill = smoker)) +
  geom_boxplot(show.legend = FALSE) +
  labs(x = "Smoker", y = "Medical charges") +
  theme_minimal()

• The scatterplot shows that charges increase with age for both but there is a slop steeper for smokers. Cluster of smokers are at high charge level across all ages. Even young smokers around 18 to 25 exceeds charges of $20,000, though young non-smokers exceeds $10,000.

# charges based on ages
ggplot(insurance_csv_dt, aes(age, charges, colour = smoker)) +
  geom_point(alpha = .5) + theme_minimal() + labs(x = "Age", y = "Charges")

• The scatter plot shows that charges stays relatively low for non-smokers and flat across BMI range. At only slight increase at high BMI values. Though, there is a positive relationship for smokers includes chargers, BMI increases particularly above 30. The highest charges is above $50,000 which exclusively among smokers with high BMI.

# charges based on BMI
ggplot(insurance_csv_dt, aes(bmi, charges, colour = smoker)) +
  geom_point(alpha = .5) + theme_minimal() + labs(x = "BMI", y = "Charges")

Decsriptive Statistics Cont.

# descriptive statistics
insurance_csv_dt %>%
  group_by(smoker) %>%
  summarise(n = n(),Min = min(charges),Q1 = quantile(charges, .25),
            Median = median(charges),
            Q3 = quantile(charges, .75),
            Max = max(charges),
            Mean = mean(charges),
            SD = sd(charges)) -> table1
kable(table1, digits = 0)

• The bar chart indicates that smokers has higher mean charges in every region of northeast, southeast, southwest and northwest. The southeast has high mean charges for both non-smokers and smokers but there is a gap stays consistently across all region.

# mean charges based on regions
insurance_csv_dt %>%
  group_by(region, smoker) %>%
  summarise(mean_charges = mean(charges), .groups = "drop") %>%
  ggplot(aes(x = region, y = mean_charges, fill = smoker)) +
  geom_col(position = "dodge") +
  labs(x = "Region", y = "Mean charges") +
  theme_minimal()

• The boxplot by smoker status and BMI category (Normal, Underweight, overweight and obsess) shows clear gradient. The median charges are low across all BMI categories around $7000 - $10,000 for non-smokers. The median charges increases BMI category of normal weight around $20,000 for smokers. Though, obese BMI category has median charges that exceeds $40,000 and has highest outlier.

# charges by BMI category and smoking status
insurance_csv_dt %>%
  mutate(bmi_group = cut(bmi, breaks = c(0, 18.5, 25, 30, Inf),
                         labels = c("Underweight", "Normal", "Overweight", "Obese"),
                         right = FALSE)) %>%
  ggplot(aes(x = bmi_group, y = charges, fill = smoker)) +
  geom_boxplot() +
  labs(x = "BMI category", y = "Charges") +
  theme_minimal()

Hypothesis Testing

• The p-value is small (<0.05), so, rejected H₀. There is a strong evidence where the smokers has various mean medical charge than non-smokers. The confidence interval of 95% represents that smokers pay between $22,191 and $25,028 on average. The t-test analysis provides p value of 6.78e-234 which confirms robustness.

par(mfrow = c(1, 2))
for (g in c("no", "yes")) {
  x <- insurance_csv_dt$charges[insurance_csv_dt$smoker == g]
  qqnorm(x, main = paste("Smoker:", g)); qqline(x)
}

par(mfrow = c(1, 1))

smk <- insurance_csv_dt$charges[insurance_csv_dt$smoker == "yes"]
non_smk <- insurance_csv_dt$charges[insurance_csv_dt$smoker == "no"]
tt_results <- t.test(smk, non_smk, alternative = "two.sided", conf.level = 0.95)
tt_results

## 
##  Welch Two Sample t-test
## 
## data:  smk and non_smk
## t = 32.742, df = 311.88, p-value < 2.2e-16
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  22190.79 25028.35
## sample estimates:
## mean of x mean of y 
##  32050.23   8440.66

# robustness check on log scale
t.test(log(charges) ~ smoker, data = insurance_csv_dt)$p.value

## [1] 6.784987e-234

Regression Model

The log transformation has been used where fitted plot vs residual represents roughly constant variance, and Q-Q plot shows normality and scale location plot has no strong pattern. All three predictors significant statistically. The model obtained 74.4% of variations. The charge per unit increase age increases by about 3.6, BMI increase by 1.1 and smokeryes increase charge by 368.9(Datanovia, 2026).

lm_mdl <- lm(log(charges) ~ age + bmi + smoker, data = insurance_csv_dt)
summary(lm_mdl)

## 
## Call:
## lm(formula = log(charges) ~ age + bmi + smoker, data = insurance_csv_dt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -1.22166 -0.21732 -0.03983  0.10344  2.07125 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 7.0761244  0.0715858   98.85  < 2e-16 ***
## age         0.0350930  0.0009117   38.49  < 2e-16 ***
## bmi         0.0107853  0.0020982    5.14 3.15e-07 ***
## smokeryes   1.5451927  0.0315190   49.02  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.465 on 1333 degrees of freedom
## Multiple R-squared:  0.7443, Adjusted R-squared:  0.7438 
## F-statistic:  1294 on 3 and 1333 DF,  p-value: < 2.2e-16

# Model Diagnostics
par(mfrow = c(2, 2)); plot(lm_mdl); par(mfrow = c(1, 1))

# percentage change in charges per unit increase
round((exp(coef(lm_mdl)[-1]) - 1) * 100, 1)

##       age       bmi smokeryes 
##       3.6       1.1     368.9

Discussion

Major Findings

1. Smokers significantly have high mean medical charges than the non-smokers with confidence interval of 95% of $22,191 to $25,028.
2. Smoker age, BMI, status together shows around 74% of variation in log charges.
3. Smoking has high effect where being smoker raises the charges by around 369%, also exceeding the effect of age and MBI of 3.6% per year and 1.1% per unit.

Strength

The strength of dataset is reproducible and open, the analysis uses required transformation and assumptions and the sample size is large.

Limitations

There is a unknown sampling method so generalisation is limited. Missing variables includes exercise, diet and pre-existing conditions.

Future Direction

Future direction may focus in additional variables like lifestyle and health and use of more advanced models and better interaction.

Conclusion

Smoking is connected with intensely higher medical charges. The take home message is that smoking is most significant predictor of high medical charges, there is a increasing cost around 370% compared to non-smokers.

References

• Choi, M. (2026). Medical cost personal datasets. Kaggle: The World’s AI Proving Ground. https://www.kaggle.com/datasets/mirichoi0218/insurance/data
• Datanovia. (2026, June 23). Linear regression assumptions & diagnostics in R. https://www.datanovia.com/learn/biostatistics/regression/linear-regression-assumptions-and-diagnostics-in-r
• Emily Nordmann, Phil McAleer, Wilhelmiina Toivo, Helena Paterson, & Lisa M. DeBruine. (2022). Author response for “Data visualization using R for researchers who do not use R”. Advances in Methods and Practices in Psychological Science, 5(2), 1-36. https://doi.org/10.1177/25152459221074654/v2/response1
• Soetewey, A. (2020, January 22). Descriptive statistics in R. Stats and R. https://statsandr.com/blog/descriptive-statistics-in-r/