Brief Overview

This project evaluates how AI brand, subscription model, and platform compatibility influence customer preference. The analysis combines data cleaning, exploratory analysis, conjoint modeling, attribute-importance calculations, profile prediction, model evaluation, and business recommendations.

1. Decision Problem

Primary research question

Which combination of AI assistant features produces the strongest customer preference?

Business objective

Use respondent ratings to recommend an AI assistant configuration that a company could prioritize for product development and marketing.

2. Data Ingestion

file_path <- "ai_preferences_survey.csv"

if (!file.exists(file_path)) {
  stop(
    paste0(
      "The file '", file_path, "' was not found. ",
      "Place the CSV in the same folder as this R Markdown file, ",
      "or update file_path in the ingestion chunk."
    )
  )
}

raw_data <- read.csv(
  file_path,
  stringsAsFactors = FALSE,
  skip = 3,
  check.names = TRUE
)

cat("Raw data dimensions:", nrow(raw_data), "rows x",
    ncol(raw_data), "columns\n")

## Raw data dimensions: 240 rows x 7 columns

glimpse(raw_data)

## Rows: 240
## Columns: 7
## $ Student.ID                         <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,…
## $ Student.Name                       <chr> "Abajian, Sevag", "Abajian, Sevag",…
## $ Profile.ID                         <int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, …
## $ AI.Brand                           <chr> "ChatGPT", "ChatGPT", "ChatGPT", "C…
## $ Subscription.Model                 <chr> "Free", "Monthly Subscription", "An…
## $ Compatibility.with.Other.Platforms <chr> "AI platform only", "Integrates wit…
## $ Rating..1.7.                       <int> NA, NA, NA, NA, NA, NA, NA, NA, NA,…

3. Data Cleaning

3.1 Rename and select variables

clean_data <- raw_data %>%
  rename(
    student_id = Student.ID,
    student_name = Student.Name,
    profile_id = Profile.ID,
    ai_brand = AI.Brand,
    subscription_model = Subscription.Model,
    compatibility = Compatibility.with.Other.Platforms,
    rating = Rating..1.7.
  ) %>%
  select(
    student_id, student_name, profile_id,
    ai_brand, subscription_model, compatibility, rating
  )

3.2 Standardize data types and remove identifiers

clean_data <- clean_data %>%
  mutate(
    across(where(is.character), ~ na_if(str_squish(.x), "")),
    student_id = as.factor(student_id),
    profile_id = as.factor(profile_id),
    ai_brand = as.factor(ai_brand),
    subscription_model = as.factor(subscription_model),
    compatibility = as.factor(compatibility),
    rating = suppressWarnings(as.numeric(rating))
  ) %>%
  select(-student_name)

3.3 Missing values and valid rating rules

For conjoint analysis, missing ratings should not be replaced with the overall mean because doing so creates artificial preferences and reduces the natural variation in the data. This report excludes records with missing attributes or ratings outside the valid 1–7 range.

missing_before <- clean_data %>%
  summarise(across(everything(), ~ sum(is.na(.x)))) %>%
  pivot_longer(
    everything(),
    names_to = "variable",
    values_to = "missing_count"
  ) %>%
  mutate(
    missing_percent = missing_count / nrow(clean_data)
  ) %>%
  arrange(desc(missing_count))

missing_before %>%
  mutate(missing_percent = percent(missing_percent, accuracy = .1)) %>%
  kbl(caption = "Missing Values Before Final Filtering") %>%
  kable_styling(full_width = FALSE)

Missing Values Before Final Filtering
variable	missing_count	missing_percent
rating	15	6.2%
student_id	0	0.0%
profile_id	0	0.0%
ai_brand	0	0.0%
subscription_model	0	0.0%
compatibility	0	0.0%

3.4 Data-quality checks

quality_summary <- tibble(
  Metric = c(
    "Raw rows",
    "Usable analysis rows",
    "Excluded or duplicate rows",
    "Unique respondents",
    "Unique profiles",
    "Minimum rating",
    "Maximum rating"
  ),
  Value = c(
    nrow(clean_data),
    nrow(analysis_data),
    nrow(clean_data) - nrow(analysis_data),
    n_distinct(analysis_data$student_id),
    n_distinct(analysis_data$profile_id),
    min(analysis_data$rating),
    max(analysis_data$rating)
  )
)

quality_summary %>%
  kbl(caption = "Final Data-Quality Summary") %>%
  kable_styling(full_width = FALSE)

Final Data-Quality Summary
Metric	Value
Raw rows	240
Usable analysis rows	225
Excluded or duplicate rows	15
Unique respondents	19
Unique profiles	12
Minimum rating	1
Maximum rating	7

Respondents

Profiles

225

Usable ratings

3.95

Average rating

4. Exploratory Data Analysis

4.1 Overall rating distribution

rating_distribution <- analysis_data %>%
  count(rating) %>%
  complete(rating = 1:7, fill = list(n = 0))

ggplot(rating_distribution, aes(x = factor(rating), y = n, fill = factor(rating))) +
  geom_col(width = .72, show.legend = FALSE) +
  geom_text(
    aes(label = n),
    vjust = -.4,
    color = "#E8F7FF",
    fontface = "bold"
  ) +
  scale_fill_manual(values = cyber_colors) +
  scale_y_continuous(expand = expansion(mult = c(0, .12))) +
  labs(
    title = "Distribution of AI Preference Ratings",
    subtitle = paste0(
      "Mean = ", round(mean(analysis_data$rating), 2),
      " | Median = ", median(analysis_data$rating)
    ),
    x = "Preference rating",
    y = "Number of responses"
  )

4.2 Average preference by AI brand

brand_summary <- analysis_data %>%
  group_by(ai_brand) %>%
  summarise(
    average_rating = mean(rating),
    standard_error = sd(rating) / sqrt(n()),
    responses = n(),
    .groups = "drop"
  ) %>%
  arrange(average_rating)

ggplot(
  brand_summary,
  aes(x = fct_inorder(ai_brand), y = average_rating, fill = ai_brand)
) +
  geom_col(width = .68, show.legend = FALSE) +
  geom_errorbar(
    aes(
      ymin = average_rating - standard_error,
      ymax = average_rating + standard_error
    ),
    width = .14,
    color = "#E8F7FF"
  ) +
  geom_text(
    aes(label = round(average_rating, 2)),
    hjust = -.18,
    color = "#E8F7FF",
    fontface = "bold"
  ) +
  coord_flip() +
  scale_fill_manual(values = cyber_colors) +
  scale_y_continuous(
    limits = c(0, 7),
    breaks = 0:7,
    expand = expansion(mult = c(0, .08))
  ) +
  labs(
    title = "Average Preference by AI Brand",
    subtitle = "Error bars show plus or minus one standard error",
    x = NULL,
    y = "Average rating"
  )

4.3 Average preference by subscription model

subscription_summary <- analysis_data %>%
  group_by(subscription_model) %>%
  summarise(
    average_rating = mean(rating),
    standard_error = sd(rating) / sqrt(n()),
    responses = n(),
    .groups = "drop"
  ) %>%
  arrange(average_rating)

ggplot(
  subscription_summary,
  aes(
    x = fct_inorder(subscription_model),
    y = average_rating,
    fill = subscription_model
  )
) +
  geom_col(width = .68, show.legend = FALSE) +
  geom_errorbar(
    aes(
      ymin = average_rating - standard_error,
      ymax = average_rating + standard_error
    ),
    width = .14,
    color = "#E8F7FF"
  ) +
  geom_text(
    aes(label = round(average_rating, 2)),
    hjust = -.18,
    color = "#E8F7FF",
    fontface = "bold"
  ) +
  coord_flip() +
  scale_fill_manual(values = cyber_colors) +
  scale_y_continuous(
    limits = c(0, 7),
    breaks = 0:7,
    expand = expansion(mult = c(0, .08))
  ) +
  labs(
    title = "Average Preference by Subscription Model",
    x = NULL,
    y = "Average rating"
  )

4.4 Average preference by compatibility

compatibility_summary <- analysis_data %>%
  group_by(compatibility) %>%
  summarise(
    average_rating = mean(rating),
    standard_error = sd(rating) / sqrt(n()),
    responses = n(),
    .groups = "drop"
  ) %>%
  arrange(average_rating)

ggplot(
  compatibility_summary,
  aes(
    x = fct_inorder(compatibility),
    y = average_rating,
    fill = compatibility
  )
) +
  geom_col(width = .68, show.legend = FALSE) +
  geom_errorbar(
    aes(
      ymin = average_rating - standard_error,
      ymax = average_rating + standard_error
    ),
    width = .14,
    color = "#E8F7FF"
  ) +
  geom_text(
    aes(label = round(average_rating, 2)),
    hjust = -.18,
    color = "#E8F7FF",
    fontface = "bold"
  ) +
  coord_flip() +
  scale_fill_manual(values = cyber_colors) +
  scale_y_continuous(
    limits = c(0, 7),
    breaks = 0:7,
    expand = expansion(mult = c(0, .08))
  ) +
  labs(
    title = "Average Preference by Compatibility",
    x = NULL,
    y = "Average rating"
  )

4.5 Profile ranking

profile_summary <- analysis_data %>%
  group_by(
    profile_id, ai_brand, subscription_model, compatibility
  ) %>%
  summarise(
    average_rating = mean(rating),
    standard_deviation = sd(rating),
    responses = n(),
    .groups = "drop"
  ) %>%
  mutate(
    profile_label = paste(
      "Profile", profile_id, "—", ai_brand,
      subscription_model, compatibility, sep = " "
    )
  ) %>%
  arrange(average_rating)

ggplot(
  profile_summary,
  aes(
    x = fct_inorder(profile_label),
    y = average_rating,
    fill = ai_brand
  )
) +
  geom_col(width = .7) +
  geom_text(
    aes(label = round(average_rating, 2)),
    hjust = -.12,
    color = "#E8F7FF",
    fontface = "bold",
    size = 3.5
  ) +
  coord_flip() +
  scale_fill_manual(values = cyber_colors) +
  scale_y_continuous(
    limits = c(0, 7),
    breaks = 0:7,
    expand = expansion(mult = c(0, .08))
  ) +
  labs(
    title = "Average Rating of Each Tested AI Profile",
    subtitle = "This identifies the strongest and weakest observed combinations",
    x = NULL,
    y = "Average rating",
    fill = "AI brand"
  )

4.6 Brand and subscription interaction heatmap

interaction_summary <- analysis_data %>%
  group_by(ai_brand, subscription_model) %>%
  summarise(
    average_rating = mean(rating),
    .groups = "drop"
  )

ggplot(
  interaction_summary,
  aes(
    x = subscription_model,
    y = ai_brand,
    fill = average_rating
  )
) +
  geom_tile(color = "#07111F", linewidth = 1) +
  geom_text(
    aes(label = round(average_rating, 2)),
    color = "#FFFFFF",
    fontface = "bold",
    size = 4.5
  ) +
  scale_fill_gradientn(
    colors = c("#182448", "#9B5DE5", "#42F5E9", "#FEE440"),
    limits = c(1, 7)
  ) +
  labs(
    title = "Brand × Subscription Preference Heatmap",
    subtitle = "Higher values indicate more attractive combinations",
    x = "Subscription model",
    y = "AI brand",
    fill = "Average rating"
  ) +
  theme(panel.grid = element_blank())

4.7 Respondent preference heatmap

heatmap_data <- analysis_data %>%
  mutate(
    student_label = paste0("Student ", student_id),
    profile_label = paste0("P", profile_id)
  )

ggplot(
  heatmap_data,
  aes(
    x = profile_label,
    y = fct_rev(student_label),
    fill = rating
  )
) +
  geom_tile(color = "#07111F", linewidth = .35) +
  scale_fill_gradientn(
    colors = c("#182448", "#9B5DE5", "#42F5E9", "#FEE440"),
    limits = c(1, 7),
    breaks = 1:7
  ) +
  labs(
    title = "Preference Patterns Across Respondents",
    subtitle = "The chart reveals agreement and individual differences",
    x = "Tested profile",
    y = NULL,
    fill = "Rating"
  ) +
  theme(
    panel.grid = element_blank(),
    axis.text.y = element_text(size = 8)
  )

5. Conjoint Model Development

An additive conjoint model estimates the separate contribution of each attribute level. Respondent fixed effects are included because some students may consistently use higher or lower portions of the rating scale.

analysis_data <- analysis_data %>%
  mutate(
    student_id = droplevels(student_id),
    ai_brand = droplevels(ai_brand),
    subscription_model = droplevels(subscription_model),
    compatibility = droplevels(compatibility)
  )

conjoint_model <- lm(
  rating ~ ai_brand + subscription_model + compatibility + student_id,
  data = analysis_data
)

model_summary <- glance(conjoint_model)

model_summary %>%
  select(r.squared, adj.r.squared, sigma, statistic, p.value) %>%
  rename(
    `R-squared` = r.squared,
    `Adjusted R-squared` = adj.r.squared,
    `RMSE` = sigma,
    `F statistic` = statistic,
    `Model p-value` = p.value
  ) %>%
  mutate(across(where(is.numeric), ~ round(.x, 4))) %>%
  kbl(caption = "Conjoint Model Evaluation") %>%
  kable_styling(full_width = FALSE)

Conjoint Model Evaluation
R-squared	Adjusted R-squared	RMSE	F statistic	Model p-value
0.1925	0.0911	1.8911	1.8977	0.0085

5.1 Attribute significance

anova_results <- anova(conjoint_model) %>%
  broom::tidy() %>%
  filter(term %in% c(
    "ai_brand",
    "subscription_model",
    "compatibility"
  )) %>%
  transmute(
    Attribute = recode(
      term,
      ai_brand = "AI Brand",
      subscription_model = "Subscription Model",
      compatibility = "Compatibility"
    ),
    `F statistic` = statistic,
    `p-value` = p.value,
    Significant = if_else(`p-value` < .05, "Yes", "No")
  )

anova_results %>%
  mutate(
    `F statistic` = round(`F statistic`, 3),
    `p-value` = format.pval(`p-value`, digits = 3, eps = .001)
  ) %>%
  kbl(caption = "Overall Statistical Significance of Product Attributes") %>%
  kable_styling(full_width = FALSE)

Overall Statistical Significance of Product Attributes
Attribute	F statistic	p-value	Significant
AI Brand	1.591	0.19269	No
Subscription Model	4.846	0.00881	Yes
Compatibility	1.524	0.22034	No

5.2 Calculate part-worth utilities

prediction_grid <- expand_grid(
  ai_brand = levels(analysis_data$ai_brand),
  subscription_model = levels(analysis_data$subscription_model),
  compatibility = levels(analysis_data$compatibility),
  student_id = levels(analysis_data$student_id)
) %>%
  mutate(
    ai_brand = factor(ai_brand, levels = levels(analysis_data$ai_brand)),
    subscription_model = factor(
      subscription_model,
      levels = levels(analysis_data$subscription_model)
    ),
    compatibility = factor(
      compatibility,
      levels = levels(analysis_data$compatibility)
    ),
    student_id = factor(
      student_id,
      levels = levels(analysis_data$student_id)
    ),
    predicted_rating = predict(conjoint_model, newdata = .)
  )

brand_utility <- prediction_grid %>%
  group_by(ai_brand) %>%
  summarise(
    mean_prediction = mean(predicted_rating),
    .groups = "drop"
  ) %>%
  mutate(
    attribute = "AI Brand",
    level = as.character(ai_brand),
    utility = mean_prediction - mean(mean_prediction)
  ) %>%
  select(attribute, level, utility)

subscription_utility <- prediction_grid %>%
  group_by(subscription_model) %>%
  summarise(
    mean_prediction = mean(predicted_rating),
    .groups = "drop"
  ) %>%
  mutate(
    attribute = "Subscription Model",
    level = as.character(subscription_model),
    utility = mean_prediction - mean(mean_prediction)
  ) %>%
  select(attribute, level, utility)

compatibility_utility <- prediction_grid %>%
  group_by(compatibility) %>%
  summarise(
    mean_prediction = mean(predicted_rating),
    .groups = "drop"
  ) %>%
  mutate(
    attribute = "Compatibility",
    level = as.character(compatibility),
    utility = mean_prediction - mean(mean_prediction)
  ) %>%
  select(attribute, level, utility)

utilities <- bind_rows(
  brand_utility,
  subscription_utility,
  compatibility_utility
)

utility_plot_data <- utilities %>%
  mutate(
    level_label = paste(attribute, level, sep = " — "),
    direction = if_else(
      utility >= 0,
      "Increases preference",
      "Decreases preference"
    )
  ) %>%
  arrange(utility) %>%
  mutate(level_label = fct_inorder(level_label))

ggplot(
  utility_plot_data,
  aes(x = level_label, y = utility, fill = direction)
) +
  geom_hline(
    yintercept = 0,
    color = "#A6C8DA",
    linetype = "dashed"
  ) +
  geom_col(width = .68) +
  coord_flip() +
  scale_fill_manual(
    values = c(
      "Increases preference" = "#42F5E9",
      "Decreases preference" = "#F15BB5"
    )
  ) +
  labs(
    title = "Conjoint Part-Worth Utilities",
    subtitle = "Positive utility increases predicted preference",
    x = NULL,
    y = "Centered utility",
    fill = NULL
  )

5.3 Attribute importance

attribute_importance <- utilities %>%
  group_by(attribute) %>%
  summarise(
    utility_range = max(utility) - min(utility),
    .groups = "drop"
  ) %>%
  mutate(
    importance = utility_range / sum(utility_range),
    importance_label = percent(importance, accuracy = .1)
  ) %>%
  arrange(importance)

ggplot(
  attribute_importance,
  aes(
    x = fct_reorder(attribute, importance),
    y = importance,
    fill = attribute
  )
) +
  geom_col(width = .68, show.legend = FALSE) +
  geom_text(
    aes(label = importance_label),
    hjust = -.18,
    color = "#E8F7FF",
    fontface = "bold",
    size = 5
  ) +
  coord_flip() +
  scale_fill_manual(values = cyber_colors) +
  scale_y_continuous(
    labels = percent_format(),
    limits = c(0, max(attribute_importance$importance) * 1.22)
  ) +
  labs(
    title = "Relative Importance of AI Product Attributes",
    subtitle = "Importance is calculated from each attribute's utility range",
    x = NULL,
    y = "Relative importance"
  )

6. Predicted Product Configurations

The following analysis predicts every possible combination of the observed attribute levels rather than limiting the recommendation to only the profiles included in the survey.

all_combinations <- expand_grid(
  ai_brand = levels(analysis_data$ai_brand),
  subscription_model = levels(analysis_data$subscription_model),
  compatibility = levels(analysis_data$compatibility),
  student_id = levels(analysis_data$student_id)
) %>%
  mutate(
    ai_brand = factor(ai_brand, levels = levels(analysis_data$ai_brand)),
    subscription_model = factor(
      subscription_model,
      levels = levels(analysis_data$subscription_model)
    ),
    compatibility = factor(
      compatibility,
      levels = levels(analysis_data$compatibility)
    ),
    student_id = factor(
      student_id,
      levels = levels(analysis_data$student_id)
    ),
    predicted_rating = predict(conjoint_model, newdata = .)
  ) %>%
  group_by(ai_brand, subscription_model, compatibility) %>%
  summarise(
    predicted_rating = mean(predicted_rating),
    .groups = "drop"
  ) %>%
  arrange(desc(predicted_rating)) %>%
  mutate(
    configuration = paste(
      ai_brand, subscription_model, compatibility, sep = " | "
    )
  )

top_predicted <- all_combinations %>%
  slice_head(n = min(10, nrow(all_combinations)))

top_predicted %>%
  mutate(predicted_rating = round(predicted_rating, 3)) %>%
  kbl(caption = "Top Predicted AI Assistant Configurations") %>%
  kable_styling(full_width = FALSE)

Top Predicted AI Assistant Configurations
ai_brand	subscription_model	compatibility	predicted_rating	configuration
Claude	Free	Integrates with common apps	4.997	Claude \| Free \| Integrates with common apps
Claude	Free	Full cross-platform compatibility	4.985	Claude \| Free \| Full cross-platform compatibility
ChatGPT	Free	Integrates with common apps	4.962	ChatGPT \| Free \| Integrates with common apps
ChatGPT	Free	Full cross-platform compatibility	4.950	ChatGPT \| Free \| Full cross-platform compatibility
Gemini	Free	Integrates with common apps	4.656	Gemini \| Free \| Integrates with common apps
Gemini	Free	Full cross-platform compatibility	4.644	Gemini \| Free \| Full cross-platform compatibility
Claude	Free	AI platform only	4.507	Claude \| Free \| AI platform only
ChatGPT	Free	AI platform only	4.472	ChatGPT \| Free \| AI platform only
Copilot	Free	Integrates with common apps	4.337	Copilot \| Free \| Integrates with common apps
Copilot	Free	Full cross-platform compatibility	4.325	Copilot \| Free \| Full cross-platform compatibility

ggplot(
  top_predicted,
  aes(
    x = fct_reorder(configuration, predicted_rating),
    y = predicted_rating,
    fill = ai_brand
  )
) +
  geom_col(width = .68) +
  geom_text(
    aes(label = round(predicted_rating, 2)),
    hjust = -.12,
    color = "#E8F7FF",
    fontface = "bold",
    size = 3.8
  ) +
  coord_flip() +
  scale_fill_manual(values = cyber_colors) +
  scale_y_continuous(
    limits = c(0, 7),
    breaks = 0:7,
    expand = expansion(mult = c(0, .08))
  ) +
  labs(
    title = "Top Predicted AI Assistant Configurations",
    subtitle = "Model-based ranking across all possible combinations",
    x = NULL,
    y = "Predicted rating",
    fill = "AI brand"
  )

6.1 Interactive profile explorer

interactive_data <- all_combinations %>%
  mutate(
    hover_text = paste0(
      "<b>", configuration, "</b>",
      "<br>Predicted rating: ", round(predicted_rating, 2)
    )
  )

interactive_plot <- ggplot(
  interactive_data,
  aes(
    x = reorder(configuration, predicted_rating),
    y = predicted_rating,
    fill = ai_brand,
    text = hover_text
  )
) +
  geom_col(width = .7) +
  coord_flip() +
  scale_fill_manual(values = cyber_colors) +
  scale_y_continuous(limits = c(0, 7)) +
  labs(
    title = "Interactive Configuration Explorer",
    x = NULL,
    y = "Predicted rating",
    fill = "AI brand"
  )

ggplotly(interactive_plot, tooltip = "text") %>%
  layout(
    paper_bgcolor = "#07111F",
    plot_bgcolor = "#07111F",
    font = list(color = "#E8F7FF")
  )

7. Model Evaluation and Validation

7.1 Actual versus predicted ratings

model_diagnostics <- analysis_data %>%
  mutate(
    predicted_rating = fitted(conjoint_model),
    residual = resid(conjoint_model)
  )

rmse <- sqrt(mean(model_diagnostics$residual^2))
mae <- mean(abs(model_diagnostics$residual))

evaluation_metrics <- tibble(
  Metric = c(
    "R-squared",
    "Adjusted R-squared",
    "RMSE",
    "Mean absolute error"
  ),
  Value = c(
    summary(conjoint_model)$r.squared,
    summary(conjoint_model)$adj.r.squared,
    rmse,
    mae
  )
)

evaluation_metrics %>%
  mutate(Value = round(Value, 3)) %>%
  kbl(caption = "Model Performance Metrics") %>%
  kable_styling(full_width = FALSE)

Model Performance Metrics
Metric	Value
R-squared	0.193
Adjusted R-squared	0.091
RMSE	1.779
Mean absolute error	1.457

trend_model <- lm(rating ~ predicted_rating, data = model_diagnostics)
summary(trend_model)

## 
## Call:
## lm(formula = rating ~ predicted_rating, data = model_diagnostics)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -3.8733 -1.4945 -0.1312  1.2100  4.5680 
## 
## Coefficients:
##                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      1.042e-14  5.542e-01   0.000        1    
## predicted_rating 1.000e+00  1.371e-01   7.291 5.29e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.786 on 223 degrees of freedom
## Multiple R-squared:  0.1925, Adjusted R-squared:  0.1889 
## F-statistic: 53.16 on 1 and 223 DF,  p-value: 5.286e-12

ggplot(
  model_diagnostics,
  aes(x = predicted_rating, y = rating)
) +
  geom_jitter(
    width = 0,
    height = .08,
    alpha = .55,
    color = "#42F5E9"
  ) +
  geom_smooth(
    method = "lm",
    se = FALSE,
    color = "#FEE440",
    linewidth = 1
  ) +
  geom_abline(
    slope = 1,
    intercept = 0,
    linetype = "dashed",
    color = "#F15BB5"
  ) +
  labs(
    title = "Actual versus Predicted Ratings",
    subtitle = "Points closer to the diagonal indicate more accurate predictions",
    x = "Predicted rating",
    y = "Actual rating"
  )

7.2 Residual diagnostic

ggplot(
  model_diagnostics,
  aes(x = predicted_rating, y = residual)
) +
  geom_hline(
    yintercept = 0,
    linetype = "dashed",
    color = "#FEE440"
  ) +
  geom_point(
    alpha = .55,
    color = "#42F5E9"
  ) +
  geom_smooth(
    se = FALSE,
    color = "#F15BB5",
    linewidth = 1
  ) +
  labs(
    title = "Residuals versus Predicted Ratings",
    subtitle = "A random pattern around zero supports the additive model",
    x = "Predicted rating",
    y = "Residual"
  )

7.3 Optional respondent-level cross-validation

This validation holds out one rating per respondent, fits the model to the remaining observations, and evaluates predictions on the held-out records.

set.seed(580)

holdout_data <- analysis_data %>%
  group_by(student_id) %>%
  slice_sample(n = 1) %>%
  ungroup()

training_data <- analysis_data %>%
  anti_join(
    holdout_data %>% select(student_id, profile_id),
    by = c("student_id", "profile_id")
  )

cv_model <- lm(
  rating ~ ai_brand + subscription_model + compatibility + student_id,
  data = training_data
)

cv_predictions <- holdout_data %>%
  mutate(
    predicted_rating = predict(cv_model, newdata = holdout_data),
    error = rating - predicted_rating
  )

cv_metrics <- tibble(
  Metric = c("Holdout RMSE", "Holdout MAE"),
  Value = c(
    sqrt(mean(cv_predictions$error^2, na.rm = TRUE)),
    mean(abs(cv_predictions$error), na.rm = TRUE)
  )
)

cv_metrics %>%
  mutate(Value = round(Value, 3)) %>%
  kbl(caption = "Respondent-Level Holdout Validation") %>%
  kable_styling(full_width = FALSE)

Respondent-Level Holdout Validation
Metric	Value
Holdout RMSE	1.604
Holdout MAE	1.308

8. Business Recommendations

best_configuration <- all_combinations %>% slice(1)

most_important <- attribute_importance %>%
  slice_max(importance, n = 1, with_ties = FALSE)

best_brand <- brand_utility %>%
  slice_max(utility, n = 1, with_ties = FALSE)

best_subscription <- subscription_utility %>%
  slice_max(utility, n = 1, with_ties = FALSE)

best_compatibility <- compatibility_utility %>%
  slice_max(utility, n = 1, with_ties = FALSE)

cat(
  '<div class="recommendation-box">',
  '<h3>Recommended AI Assistant Configuration</h3>',
  '<p><strong>AI brand:</strong> ', best_configuration$ai_brand, '</p>',
  '<p><strong>Subscription model:</strong> ',
  best_configuration$subscription_model, '</p>',
  '<p><strong>Compatibility:</strong> ',
  best_configuration$compatibility, '</p>',
  '<p><strong>Predicted preference:</strong> ',
  round(best_configuration$predicted_rating, 2), ' out of 7</p>',
  '<p><strong>Most influential attribute:</strong> ',
  most_important$attribute, ' (',
  percent(most_important$importance, accuracy = .1), ')</p>',
  '</div>'
)

Recommended AI Assistant Configuration

AI brand: 2

Subscription model: 2

Compatibility: 3

Predicted preference: 5 out of 7

Most influential attribute: Subscription Model ( 48.0% )

Recommended actions:

Position the preferred subscription option as the main customer offer.
Use the strongest predicted configuration in an A/B marketing test.
Avoid treating brand as the only source of value; the complete feature bundle drives preference.
Repeat the survey with a larger and more diverse sample before making a major investment decision.

9. Limitations

The sample is small and based primarily on classmates.
The survey measures stated preferences rather than actual purchases.
Only three attributes were examined.
The rating scale is ordinal, although the linear model treats it as approximately continuous.
Respondents may interpret subscription and compatibility descriptions differently.
Mean preference estimates may not apply to the broader AI-assistant market.

10. Optional Extensions

10.1 Interaction model comparison

This optional test determines whether specific combinations create effects beyond the additive conjoint model.

interaction_model <- lm(
  rating ~
    ai_brand * subscription_model +
    ai_brand * compatibility +
    subscription_model * compatibility +
    student_id,
  data = analysis_data
)

model_comparison <- anova(conjoint_model, interaction_model) %>%
  broom::tidy()

model_comparison %>%
  mutate(across(where(is.numeric), ~ round(.x, 4))) %>%
  kbl(
    caption = paste(
      "Additive Model versus Two-Way Interaction Model",
      "(use the interaction model only if it significantly improves fit)"
    )
  ) %>%
  kable_styling(full_width = FALSE)

Additive Model versus Two-Way Interaction Model (use the interaction model only if it significantly improves fit)
term	df.residual	rss	df	sumsq	statistic	p.value
rating ~ ai_brand + subscription_model + compatibility + student_id	199	711.6896	NA	NA	NA	NA
rating ~ ai_brand * subscription_model + ai_brand * compatibility + subscription_model * compatibility + student_id	195	706.2348	4	5.4548	0.3765	0.8252

10.2 Preference segmentation with clustering

This optional section groups respondents according to their average rating patterns across AI brands.

respondent_brand_matrix <- analysis_data %>%
  group_by(student_id, ai_brand) %>%
  summarise(
    average_rating = mean(rating),
    .groups = "drop"
  ) %>%
  pivot_wider(
    names_from = ai_brand,
    values_from = average_rating
  ) %>%
  drop_na()

if (nrow(respondent_brand_matrix) >= 4) {
  cluster_input <- respondent_brand_matrix %>%
    select(-student_id) %>%
    scale()

  k_value <- min(3, nrow(respondent_brand_matrix) - 1)

  set.seed(580)
  cluster_model <- kmeans(
    cluster_input,
    centers = k_value,
    nstart = 25
  )

  segmented_students <- respondent_brand_matrix %>%
    mutate(segment = factor(cluster_model$cluster))

  segment_profile <- segmented_students %>%
    pivot_longer(
      -c(student_id, segment),
      names_to = "ai_brand",
      values_to = "average_rating"
    ) %>%
    group_by(segment, ai_brand) %>%
    summarise(
      average_rating = mean(average_rating),
      .groups = "drop"
    )

  ggplot(
    segment_profile,
    aes(
      x = ai_brand,
      y = average_rating,
      group = segment,
      color = segment
    )
  ) +
    geom_line(linewidth = 1.2) +
    geom_point(size = 3) +
    scale_color_manual(values = cyber_colors) +
    scale_y_continuous(limits = c(1, 7), breaks = 1:7) +
    labs(
      title = "Optional Preference Segments",
      subtitle = "Respondents grouped by similar AI-brand rating patterns",
      x = "AI brand",
      y = "Average rating",
      color = "Segment"
    )
}

11. Final Conclusion

The analysis moves beyond simple averages by estimating the separate value of each AI assistant feature. The final recommendation should be based on the highest-utility attribute levels, the attribute-importance results, the predicted configuration ranking, and the model’s limitations.

Executive Summary

Our project used conjoint analysis to compare user preferences across four top AI models (ChatGPT, Claude, Copilot, and Gemini). This allowed us to create combinations of three attributes we found most important: AI brand, subscription model, and compatibility with other platforms. We chose these attributes because they are particularly useful for business recommendations, in that they provide data across several meaningful domains: perceived quality and trust, pricing psychology, and functional reach. The student survey we conducted allowed us to create a linear regression model, with the rating (1-7, 7 being most preferred) as the outcome and the attribute levels as predictors. In turn, this allowed us to generate every possible combination of the three attributes and predict a rating for each hypothetical product. By averaging predictions across all students, we could rank them and estimate the best configurations.

Our model captured had meaningful structure, but there’s still a lot of noise it couldn’t explain. The scatterplot in Section 7.1 shows the data points scattered pretty loosely around the trend line, not hugging it. That trend line does have a slope of exactly 1.0, but that’s not actually proof the model’s doing well — it’s just what happens whenever you plot a model’s predictions against the same data it was trained on, no matter how good or bad the model is. The numbers that actually show how accurate it is our R² = 0.193, RMSE = 1.779, and MAE = 1.457. In plain terms: the model explains about 19% of why people rated things the way they did, and its guesses are usually off by 1.5 to 2 points on the 7-point scale. So, it’s better at picking up on the big-picture trends, like which subscription type or brand people tend to like more, than at nailing any one person’s exact rating.

This analysis led us to conclude that the subscription model was the most statistically significant attribute (p-value = 0.00881). and it also mattered the most overall — 48% importance, compared to 30% for brand and 22% for compatibility. The proof is pretty clear: every single one of the top ten predicted configurations had a Free subscription, including the best one overall, Claude + Free + Integrates with common apps, which scored a 4.997 out of 7. Compatibility wasn’t statistically significant on its own, but it still helped — both “Integrates with common apps” and “Full cross-platform compatibility” bumped up predicted ratings compared to “AI platform only.” Thus, we recommend companies cater to what customers want most, a free subscription with basic inter-platform compatibility. But to maintain profitability, companies should limit this compatibility on the free level to only a few platforms, allowing them to charge for additional integrations. In the future, further larger-scale surveys should be conducted, including using A/B tests of the top configurations to validate our findings, before making expensive product decisions.

AI Preferences Survey

Conjoint Analysis of AI Assistant Features

Group 2: Nicholas Bridgman, Darren Ramos, Joshua Rivera, and Johnny Romero

August 04, 2026