I used the Online Shoppers Purchasing Intention Dataset, which tracks user behavior on an e-commerce site. E-commerce analytics is critical in business because it helps companies understand what drives purchases and which types of sessions convert visitors into buyers.
The dataset includes session-related metrics such as the number of administrative and product-related pages viewed, bounce rates, exit rates, page values, whether the visit occurred on a weekend, and whether the visitor completed a purchase.
Business Problem: In a competitive online retail environment, understanding customer behavior is crucial. I want to identify behavioral patterns that lead to conversions, helping e-commerce sites make data-driven decisions that improve revenue and customer experience.
Goal: Use the dataset to build a predictive model that determines the likelihood of a customer making a purchase during their visit based on session data.
Key Question: Can I accurately predict whether a user will make a purchase based on features like page values, bounce rates, and weekend visits?
This section loads the dataset from a CSV file and provides a summary. I want to understand the structure, size, and types of data before doing any modeling. I begin by loading the dataset and exploring the structure. This step allows us to understand the data types, number of observations, and preview the variables.
shoppers <- read_csv("shoppers.csv")
## Rows: 12330 Columns: 18
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (2): Month, VisitorType
## dbl (14): Administrative, Administrative_Duration, Informational, Informatio...
## lgl (2): Weekend, Revenue
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
head(shoppers)
## # A tibble: 6 × 18
## Administrative Administrative_Duration Informational Informational_Duration
## <dbl> <dbl> <dbl> <dbl>
## 1 0 0 0 0
## 2 0 0 0 0
## 3 0 0 0 0
## 4 0 0 0 0
## 5 0 0 0 0
## 6 0 0 0 0
## # ℹ 14 more variables: ProductRelated <dbl>, ProductRelated_Duration <dbl>,
## # BounceRates <dbl>, ExitRates <dbl>, PageValues <dbl>, SpecialDay <dbl>,
## # Month <chr>, OperatingSystems <dbl>, Browser <dbl>, Region <dbl>,
## # TrafficType <dbl>, VisitorType <chr>, Weekend <lgl>, Revenue <lgl>
summary(shoppers)
## Administrative Administrative_Duration Informational
## Min. : 0.000 Min. : 0.00 Min. : 0.0000
## 1st Qu.: 0.000 1st Qu.: 0.00 1st Qu.: 0.0000
## Median : 1.000 Median : 7.50 Median : 0.0000
## Mean : 2.315 Mean : 80.82 Mean : 0.5036
## 3rd Qu.: 4.000 3rd Qu.: 93.26 3rd Qu.: 0.0000
## Max. :27.000 Max. :3398.75 Max. :24.0000
## Informational_Duration ProductRelated ProductRelated_Duration
## Min. : 0.00 Min. : 0.00 Min. : 0.0
## 1st Qu.: 0.00 1st Qu.: 7.00 1st Qu.: 184.1
## Median : 0.00 Median : 18.00 Median : 598.9
## Mean : 34.47 Mean : 31.73 Mean : 1194.8
## 3rd Qu.: 0.00 3rd Qu.: 38.00 3rd Qu.: 1464.2
## Max. :2549.38 Max. :705.00 Max. :63973.5
## BounceRates ExitRates PageValues SpecialDay
## Min. :0.000000 Min. :0.00000 Min. : 0.000 Min. :0.00000
## 1st Qu.:0.000000 1st Qu.:0.01429 1st Qu.: 0.000 1st Qu.:0.00000
## Median :0.003112 Median :0.02516 Median : 0.000 Median :0.00000
## Mean :0.022191 Mean :0.04307 Mean : 5.889 Mean :0.06143
## 3rd Qu.:0.016813 3rd Qu.:0.05000 3rd Qu.: 0.000 3rd Qu.:0.00000
## Max. :0.200000 Max. :0.20000 Max. :361.764 Max. :1.00000
## Month OperatingSystems Browser Region
## Length:12330 Min. :1.000 Min. : 1.000 Min. :1.000
## Class :character 1st Qu.:2.000 1st Qu.: 2.000 1st Qu.:1.000
## Mode :character Median :2.000 Median : 2.000 Median :3.000
## Mean :2.124 Mean : 2.357 Mean :3.147
## 3rd Qu.:3.000 3rd Qu.: 2.000 3rd Qu.:4.000
## Max. :8.000 Max. :13.000 Max. :9.000
## TrafficType VisitorType Weekend Revenue
## Min. : 1.00 Length:12330 Mode :logical Mode :logical
## 1st Qu.: 2.00 Class :character FALSE:9462 FALSE:10422
## Median : 2.00 Mode :character TRUE :2868 TRUE :1908
## Mean : 4.07
## 3rd Qu.: 4.00
## Max. :20.00
After loading the data, I observe that the dataset contains over 12,000 rows and several key variables related to user behavior on an e-commerce site. The ‘Revenue’ variable, which indicates if a purchase was made, will be my target for prediction.
I clean the dataset and prepare it for modeling. This includes selecting relevant predictors and converting the response variable to a factor.
model_data <- shoppers %>%
select(PageValues, BounceRates, ExitRates, Weekend, Revenue) %>%
mutate(Revenue = factor(Revenue))
set.seed(123)
split <- createDataPartition(model_data$Revenue, p = 0.7, list = FALSE)
train_data <- model_data[split, ]
test_data <- model_data[-split, ]
model <- glm(Revenue ~ ., data = train_data, family = "binomial")
This section evaluates how well the logistic model fits the training data by plotting the distribution of predicted probabilities for both purchase outcomes. I visualize how the model performs on the training data by examining the distribution of predicted probabilities for purchases vs. non-purchases.
train_data$predicted <- predict(model, train_data, type = "response")
train_data$predicted_class <- ifelse(train_data$predicted > 0.5, "TRUE", "FALSE")
ggplot(train_data, aes(x = predicted, fill = Revenue)) +
geom_histogram(bins = 30, alpha = 0.6, position = "identity") +
labs(title = "Predicted Probabilities (Training Set)") +
theme_minimal()
From the training set histogram, I can see a clear separation between predicted values for purchases and non-purchases, suggesting the model is learning some signal from the data.
This plot shows how well the model’s predictions generalize to new data in the test set. I am looking for similar separation of predicted probabilities here. Here I evaluate how the model generalizes to new data using the test set. I compare the predicted probabilities for conversion.
test_data$predicted <- predict(model, test_data, type = "response")
test_data$predicted_class <- ifelse(test_data$predicted > 0.5, "TRUE", "FALSE")
ggplot(test_data, aes(x = predicted, fill = Revenue)) +
geom_histogram(bins = 30, alpha = 0.6, position = "identity") +
labs(title = "Predicted Probabilities (Test Set)") +
theme_minimal()
In the test set, the predictions are more mixed compared to the training set, which is expected. However, there is still a visible difference in prediction distributions between those who purchased and those who didn’t.
I print the summary of my logistic regression model. This gives us insights into which variables are statistically significant in predicting purchases.
summary(model)
##
## Call:
## glm(formula = Revenue ~ ., family = "binomial", data = train_data)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -1.722515 0.070036 -24.595 < 2e-16 ***
## PageValues 0.077026 0.002684 28.700 < 2e-16 ***
## BounceRates -1.337592 3.823274 -0.350 0.726
## ExitRates -21.782472 2.674724 -8.144 3.83e-16 ***
## WeekendTRUE 0.086479 0.083125 1.040 0.298
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 7439.1 on 8631 degrees of freedom
## Residual deviance: 5304.0 on 8627 degrees of freedom
## AIC: 5314
##
## Number of Fisher Scoring iterations: 7
I evaluate model assumptions by plotting the residuals. Deviance residuals help us identify any systematic errors.
residuals <- residuals(model, type = "deviance")
plot(residuals, main = "Residual Plot", ylab = "Deviance Residuals")
abline(h = 0, col = "red", lty = 2)
To further evaluate the model’s ability to distinguish between buyers and non-buyers, I use a ROC curve and calculate the AUC score. These are common evaluation metrics in classification problems.
library(pROC)
roc_curve <- roc(test_data$Revenue, test_data$predicted)
plot(roc_curve, main = "ROC Curve for Logistic Regression", col = "blue")
auc(roc_curve)
## Area under the curve: 0.8667
The ROC curve plots the true positive rate against the false positive rate. The closer the curve follows the left-hand border and then the top border of the ROC space, the better the model. The AUC (Area Under the Curve) quantifies this; values closer to 1 indicate a better model. In this case, the AUC shows that the model performs moderately well in distinguishing between sessions that result in purchases and those that don’t.
While the improved model may offer slightly better classification rates, the gains are minimal. This tells me that logistic regression handles this problem well even in its simpler form, and more advanced methods like decision trees may be needed to capture additional nuance.
My analysis demonstrates that PageValues is the
strongest predictor of purchase behavior. High bounce rates are
associated with lower conversion rates. I also saw modest improvement
after incorporating interaction terms and standardizing variables.
This type of model can help e-commerce businesses prioritize improvements in site content and user experience to increase the likelihood of a successful sale.
I visualize how this model can be used to segment users for retargeting based on predicted purchase probability. Visitors with high or low predicted probabilities can be grouped for targeted actions.
One key application of this analysis is in
retargeting — using behavioral data to identify
visitors who are unlikely to convert in real time. For example, a
visitor with a low PageValues score and high
BounceRates might be flagged for a targeted promotion,
pop-up offer, or follow-up email campaign. Conversely, visitors with
high PageValues but no purchase could be retargeted through
ads or reminders, as they have shown clear intent.
The logistic model provides a probability of conversion, which can be used to segment users into different engagement strategies and improve marketing efficiency.
test_data %>%
mutate(segment = case_when(
predicted >= 0.75 ~ "High Intent",
predicted >= 0.50 ~ "Medium Intent",
TRUE ~ "Low Intent"
)) %>%
count(segment) %>%
ggplot(aes(x = segment, y = n, fill = segment)) +
geom_col() +
labs(title = "User Segments Based on Predicted Purchase Probability",
x = "Segment", y = "Number of Users") +
theme_minimal()
This chart shows how many users fall into each predicted intent
category. High intent users might need reminders or abandoned cart
emails. Low intent users might benefit from an incentive, such as a
coupon or limited-time offer. that PageValues is the
strongest predictor of purchase behavior. High bounce rates are
associated with lower conversion rates. I also saw modest improvement
after incorporating interaction terms and standardizing variables.
This type of model can help e-commerce businesses prioritize improvements in site content and user experience to increase the likelihood of a successful sale.