Executive Summary

This project studies whether premium-priced Sephora skincare products actually generate stronger customer response, or whether customer response is better explained by product positioning variables such as ingredients, brand, category, exclusivity, online availability, and review sentiment.

The business question is:

Should Sephora emphasize premium pricing as a signal of quality, or should it prioritize ingredient-based and segment-based merchandising when recommending skincare products?

The analysis uses the Kaggle dataset Sephora Products and Skincare Reviews, which includes product attributes, ratings, prices, ingredients, and customer review data. With the upgraded Posit Cloud memory limit, the review analysis is designed to use a larger review sample instead of relying mainly on product-level averages.

item value
Expected data folder /cloud/project/archive/sephora
Source data available TRUE
Analysis script available TRUE
Processed output folder /cloud/project/data/processed/sephora

If Source data available is FALSE, download the Kaggle dataset and unzip it into archive/sephora/. Expected files include product_info.csv and reviews_*.csv.

Dataset and Variables

The unit of analysis appears in two forms:

Key variables used in the analysis:

metric value
product_file product_info.csv
review_files 5
max_review_rows_per_file 75000
product_rows 8494
review_rows 349977
product_columns 27
review_columns 20

Data Quality Treatment

Check Value
Price outlier quantile 0.99
Price outlier cutoff $275.00
Highest source price $1,900.00
Highest analysis price $275.00
Source product rows 8,494
Analysis product rows 8,412
Price outlier products removed 82

The price analysis excludes products above the 99th-percentile price cutoff. The pre-cutoff maximum price is shown in the audit and Product Overview tables so the cutoff is transparent: the rule removes only the most extreme prices while preserving nearly the entire product assortment. A stricter cutoff, such as the 95th percentile, would remove many legitimate premium skincare products and weaken the business question. A looser cutoff would leave unusually expensive items in the plots and models, where they can distort the visual scale and overstate the role of price. The raw cleaned dataset is still preserved; the cutoff is applied only to the analysis dataset used for charts, summaries, clustering, and models.

The review sample is also capped for compute stability. The analysis uses up to 75,000 rows from each review file, which materially expands coverage beyond the smaller starting sample while keeping the report reproducible in Posit Cloud. More expensive steps remain capped separately: sentiment analysis uses up to 75,000 review rows, and the recommendation model uses a reproducible sample of up to 50,000 review rows. This keeps the descriptive tables broad without letting NLP or regression steps dominate memory and runtime.

Product Overview

Metric Value
Products 8,412
Source products 8,494
Price outlier cutoff $275
Highest source price before cutoff $1,900
Highest analysis price after cutoff $275
Price outlier products excluded 82
Brands 303
Primary categories 9
Median price $35
Median rating 4.29
Median reviews 124
Share online only 21.8%
Share Sephora exclusive 28.1%
Share out of stock 7.4%

This section establishes the overall product landscape: number of products, number of brands, median price, median rating, review volume, and channel-positioning variables. Price-sensitive analysis excludes products above the 99th-percentile price cutoff so a small number of extreme products do not distort the charts, models, or segment profiles.

Review Sample

Metric Value
Reviews analyzed 447,257
Products reviewed 1,767
Median review rating 5
Recommendation rate 84.3%
Median helpfulness 0.92
Earliest review 2008-08-28
Latest review 2023-03-21

This table confirms the review-level sample used for customer recommendation and sentiment analysis. The review-level outcome is more useful than product rating alone because it measures whether customers would recommend a product after using it.

Price and Customer Ratings

This chart tests the basic managerial assumption behind premium beauty pricing: if price is a quality signal, higher-priced products should consistently receive stronger customer ratings. If the relationship is weak or flat, then price alone is not a reliable indicator of customer satisfaction.

Ingredient Positioning

ingredient_flag products median_price median_rating median_reviews
Retinol 129 69.0 4.462 154.0
Hyaluronic 737 40.0 4.339 173.0
Vitamin C 545 45.0 4.315 146.0
Niacinamide 530 42.5 4.314 166.0
Fragrance 4171 38.0 4.308 117.0
Peptide 723 49.0 4.270 155.5
Spf 224 38.0 4.255 223.5

This section evaluates whether recognizable skincare ingredients are associated with stronger customer response. This is more actionable than price alone because Sephora can use ingredient-based merchandising, filters, product badges, and recommendation modules.

Brand Positioning

brand_name products median_price median_rating total_reviews total_loves share_exclusive
SEPHORA COLLECTION 351 15.0 4.182 120986 12527343 0.912
Benefit Cosmetics 71 25.0 4.233 112163 3813091 0.014
Anastasia Beverly Hills 94 25.0 4.340 106085 7941142 0.202
Urban Decay 58 27.0 4.273 104549 5730058 0.000
Too Faced 61 28.0 4.200 100891 4864537 0.016
CLINIQUE 177 32.5 4.223 98449 3618316 0.034
Lancôme 76 37.5 4.408 87754 2164858 0.053
NARS 60 30.0 4.107 80900 7431271 0.133
Fenty Beauty by Rihanna 81 28.0 4.118 77185 9369443 0.000
KVD Beauty 36 25.0 4.117 68248 2347195 0.000
tarte 126 24.0 4.039 67529 4983541 0.230
Dior 133 60.0 4.344 61015 6224155 0.090
Laura Mercier 55 36.0 4.256 59396 3384861 0.109
Hourglass 84 39.0 4.379 57938 3843272 0.012
Tatcha 50 54.0 4.387 57075 3713079 0.800

This view separates brand popularity from brand quality. A brand with high review volume but ordinary ratings may be broadly visible but not necessarily highly loved. A brand with fewer products and higher ratings may be a stronger candidate for premium placement.

Recommendation Behavior

Price tier Reviews Rec. rate Median rating
Under $25 86,939 80.6% 5
$25-$49 151,498 84.8% 5
$50-$99 132,894 86.0% 5
$100+ 20,514 85.5% 5

Ingredient signal Reviews Recommendation rate Median review rating Median product price
Retinol 18,582 89.2% 5 $88
Fragrance 94,711 84.8% 5 $38
Hyaluronic 37,460 84.6% 5 $45
Peptide 30,410 84.3% 5 $58
Niacinamide 32,084 83.6% 5 $48
Vitamin C 56,591 83.1% 5 $32
Spf 14,034 80.1% 5 $38

The ingredient view moves the project beyond “expensive versus cheap” and tests whether skincare-specific product signals line up with customer advocacy.

Vitamin C is a useful cautionary example: it has a strong raw recommendation rate of 83.1%, but the logistic regression odds ratio is 0.88 (p = 0.049), suggesting about 12% lower recommendation odds after controlling for review rating, price, skin type, and other product signals. In business terms, Vitamin C is popular, but it does not appear to add independent recommendation lift by itself.

Skin type Price tier Reviews Recommendation rate Median review rating Median product price
combination $25-$49 42,134 84.8% 5 $38
combination $50-$99 37,533 87.3% 5 $69
combination Under $25 28,581 80.6% 5 $20
dry $25-$49 14,915 83.8% 5 $38
dry $50-$99 13,075 86.6% 5 $68
normal $25-$49 10,383 83.7% 5 $38
dry Under $25 10,153 79.8% 5 $20
normal $50-$99 9,511 85.8% 5 $69
oily $25-$49 9,425 83.1% 5 $38
normal Under $25 7,431 78.8% 5 $20
oily $50-$99 7,386 85.6% 5 $69
oily Under $25 6,614 79.5% 5 $20
combination $100+ 4,888 85.7% 5 $140
dry $100+ 1,753 86.8% 5 $140
normal $100+ 1,306 86.4% 5 $145

This customer-segment table checks whether the price/recommendation relationship is consistent across customer skin types. It is useful managerially because skincare recommendations are often personalized.

Variable Estimate Odds ratio Std. error p-value
Review rating 3.506 33.33 0.046 <0.001
Ingredient: hyaluronic 0.154 1.17 0.075 0.041
Ingredient: retinol 0.218 1.24 0.110 0.047
Ingredient: vitamin c -0.127 0.88 0.064 0.049
Skin type: oily -0.159 0.85 0.083 0.055
Sephora exclusive 0.115 1.12 0.062 0.065
Skin type: normal -0.132 0.88 0.083 0.110
Online only 0.122 1.13 0.082 0.135
Ingredient: peptide -0.118 0.89 0.081 0.145
Ingredient: niacinamide -0.096 0.91 0.076 0.210

The review-level model estimates which variables are associated with a customer recommending a product. The dependent variable is the recommendation flag, and the independent variables include review rating, product price, skin type, category, ingredient flags, exclusivity, and online-only status. The model uses a reproducible capped sample of review rows for rendering stability, while the descriptive review tables use the larger cleaned review sample. Very unstable coefficients with extremely large standard errors are omitted from the display table because they are not interpretable.

Interpretation guide:

Model Evaluation and Comparison

Model Model type Business question Evaluation basis Best use
Logistic regression Supervised classification Which product and customer signals are associated with recommendation behavior? Odds ratios, p-values, coefficient stability, and interpretability Identify variables associated with stronger or weaker customer advocacy
K-means clustering Unsupervised segmentation What groups of products have similar price, rating, review, channel, and ingredient profiles? Segment size, segment profiles, median price, median rating, review volume, and managerial usefulness Create merchandising segments and compare product positioning groups
Sentiment analysis Text-based exploratory analysis What language appears most often in positive and negative customer reviews? Positive/negative word patterns by sentiment and price tier Explain the customer language behind numeric ratings and recommendation behavior

The models are evaluated differently because they answer different questions. The logistic regression is the main explanatory model for the recommendation outcome, so its usefulness comes from interpretable odds ratios and statistical evidence. K-means clustering does not predict a yes/no outcome; it is evaluated by whether the resulting product groups are large enough, distinct enough, and useful for merchandising decisions. Sentiment analysis is included as a text-based exploratory layer that helps explain what customers say, rather than only whether they recommend a product. Together, the models provide complementary evidence: regression identifies drivers of advocacy, clustering describes product positioning groups, and sentiment adds customer-language context.

Product Segmentation

Segment Products Median price Median rating Median reviews Median loves Exclusive Online only
Affordable mass-appeal favorites 126 $68 4.46 154 10,839 30.2% 18.3%
Mid-price high-engagement products 485 $42 4.31 166 11,006 38.8% 20.2%
Online-first niche products 2,862 $39 4.25 18 3,666 18.8% 50.7%
Premium hero products 4,664 $32 4.30 282 20,460 33.0% 2.6%

The segmentation model uses k-means clustering with k = 4. Each product is represented by standardized product features: log price, rating, log review count, log loves count, Sephora exclusivity, online-only status, limited-edition status, out-of-stock status, and ingredient flags. The numeric cluster IDs were renamed after inspecting their profiles:

Review Sentiment

Price tier Positive words Negative words Sentiment words Net sentiment
Under $25 561 500 1,061 5.7%
$25-$49 201 150 351 14.5%
$50-$99 307 172 479 28.2%
brand_name positive negative total_sentiment_words net_sentiment
LANEIGE 182 82 264 0.379
Clarins 307 172 479 0.282
Supergoop! 201 150 351 0.145
The INKEY List 219 207 426 0.028
Dr. Jart+ 156 211 367 -0.150

Sentiment analysis adds qualitative context to the quantitative model. It helps identify what customers actually talk about when they describe positive and negative skincare experiences, and whether premium tiers receive meaningfully different language from customers.

Managerial Recommendations

Area Recommended action
Premium pricing Do not assume higher price automatically means stronger customer advocacy. Use recommendation rate and sentiment as performance checks for premium placement.
Ingredient merchandising Prioritize ingredient-led merchandising for signals with stronger recommendation rates, especially in search filters, product badges, and category pages.
Customer personalization Use skin-type differences to personalize recommendations instead of showing the same premium products to every shopper.
Review monitoring Track net sentiment by brand and price tier to catch products that look strong by rating but generate negative review language.

The key managerial implication is that Sephora should evaluate products using customer advocacy and customer language, not only product price and average rating. Premium products should earn their placement through recommendation behavior, review sentiment, and ingredient fit.

Limitations

This analysis is observational. It can identify associations, not prove that price, ingredients, or exclusivity cause higher ratings or recommendation rates.

Important limitations:

Appendix: Reproducibility

The analysis script writes processed CSVs to:

output_type folder
Processed CSVs /cloud/project/data/processed/sephora
Figures /cloud/project/figures/sephora

To rerun the full project:

source("Sephora_skincare_analysis.R")
rmarkdown::render("Sephora.Rmd")

AI disclosure: I used OpenAI Codex to help structure the R workflow, draft reusable code, and prepare the report template. All analysis, interpretation, and conclusions are my own.