A/B Testing Background

This method of analysis is used to compare two versions of something against each other to determine which one performs better. I chose to analyze the amount of Google Searches both Lincoln Financial and John Hancock recieved after the announcement of two of their marketing campaigns. Lincoln Financial’s Jalen Hurts Campaign, and John Hancock’s Vitality campaign.

library(readr)
data <- read_csv("Combined_Trends.csv")
## Rows: 64 Columns: 2
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## dbl (2): Ads, GoogleSearches
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
ls(data) # list the variables in the dataset
## [1] "Ads"            "GoogleSearches"
head(data) #list the first 6 rows of the dataset
## # A tibble: 6 × 2
##     Ads GoogleSearches
##   <dbl>          <dbl>
## 1     0             80
## 2     0             96
## 3     0             80
## 4     0             84
## 5     0             79
## 6     0             62
data$Ads <- factor(data$Ads)
is.factor(data$Ads)
## [1] TRUE
data$Ads[1:15]
##  [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
## Levels: 0 1
summary(lm(GoogleSearches~Ads, data = data))
## 
## Call:
## lm(formula = GoogleSearches ~ Ads, data = data)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -35.688 -10.125  -2.625   9.562  59.875 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   74.688      3.456  21.614  < 2e-16 ***
## Ads1         -34.562      4.887  -7.073 1.61e-09 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 19.55 on 62 degrees of freedom
## Multiple R-squared:  0.4465, Adjusted R-squared:  0.4376 
## F-statistic: 50.02 on 1 and 62 DF,  p-value: 1.61e-09

Interpretations

The P-Value for X is 1.61e-09, which is less than .05, we reject the null hypothesis in favor of the alternative hypothesis. The difference between the two groups is statistically significant.

The coefficient for Ads1 in the regression output is -34.562, which indicates that Lincoln’s advertisements are less effective than John Hancock’s. In comparison to John Hancock’s advertising campaign, Lincoln Financial’s google searches were on average 34.6 less.