A successful business is often looking to expand on its customer base by trying to acquire new customers through various marketing strategies. Depending on the company’s business model, they may choose from various marketing strategy methods. Therefore, it is imperative to understand which strategy works best in terms of generating success backed by evidence, not intuition.

Penetrating a New Market

Seeder is a company that sells electric scooters in California and operates as one of the most successfull producers of electric scooters in the country. They are trying to tap the Texas region by selling electric scooters in Texas.

The marketing team at Seeder employs the use of print media and designs a brochure to attract new customers. They distribute the brochure to 23 Texians and end up making 9 sales.

The Question

What Seeder management wants to know is how good is print media to market the new electric scooter? If the company went on to produce a large number of brochures, say in quantity of hundreds of thousands, what percentage of conversion should they expect to see?

The answer to the question seems pretty easy at first. Using the data collected by the marketing team, calculating the probability of success by divding number of sales by the total number of brochures distributed.

brochures_distributed = 23
new_sales = 9
probability_of_success = new_sales / brochures_distributed
probability_of_success
## [1] 0.3913043

We can see that the probability of making a sale is approximately 39%. Which means that for every 100 brochures distributed, Seeder should expect to sell 39 electric scooters.

This may seem as a good estimation given the data we have present to us, but it is highly uncertain. We have no information on which people were shown the brochure, what was the motivation behind their decision to buy the electric scooter and a plethora of information which may be deemed as siginifanctly affecting their decision to purchase the electric scooter. The small sample size also raises a red flag regarding the accuracy of the conversion estimate.

Using Bayesian Probability to Quantify Uncertainty

Bayesian probability allows us to quantify the unknown information using probability distributions. In order to be able to quantify the uncertainty in our study, we require three things.

  1. Data
  2. Generative Model
  3. Priors

Data

Data in our case will come from the pilot study we conducted. We know that out of the 23 Texians that were shown the brochure, 9 of them were converted into customers.

Generative Model

A generative model can be termed as a set of instructions that we pass to some parameter(s), say an underlying percentage of conversion, so that the model simulate data based on the parameter(s). For instance, we may assume that there is an underlying conversion of 40% and we show the brochure to randomly selected 50 Texians. Based on our assumed conversion rate, the generative model would estimate 20 sales to be made.

Based on the generative model assumed above, we have information regarding the number of sales we are likely to make but we are interested in the percentage at which the sales are likely to be made. Our generative model is simulating data but we already know what our data is. We need to reverse solve the generative model so that it outputs the conversion rate. In order to achieve this, what we need is the third item Priors.

Priors

Priors is the information the model has before seeing any data. In Bayesian terms, priors is a probability distribution used to represent the uncertainty in the model. In order to create Priors for our model, we will use the uniform probability distribution. By using the uniform probability distribution from 0 to 1, we are stating that before seeing any data, the model assumes that any rate of conversion between 0 and 1 (0% to 100%) is equally likely.

Fitting the Model

Following is the basic flow of how the model works.

  1. From our prior distribution, we draw a random sample for the parameter value. Parameter value in our case is the conversion rate.
# number of samples to draw from the prior distribution
n_size <- 100000

# drawing sample from the prior distribution - which in our case is uniform distribution
prior_dist <- runif(n_size, 0, 1)

# peeking at the histogram to verify the random sample was generated correctly.

hist(prior_dist, main = "Histogram of Prior Distribution", xlab = "Prior on the Conversion Rate", ylab = "Frequency")

  1. We take the parameter value from step 1, plug it into our generative model so that the model simulates some data. We repeat this process multiple times using a for loop.
# defining the generative model - a model or set of rules that we feed to parameters so that it simulates data based on those set of rules
generative_model <- function(rate) {
  sales <- rbinom(1, size = 23, prob = rate)
  sales
}

# simulating the data through our generative model
sales <- rep(NA, n_size)
for(i in 1:n_size) {
  sales[i] <- generative_model(prior_dist[i])
}
  1. Filter out all sample draws that are consistent with our data, i.e. conversion rate of 0.3913 or total sales equalling 9.
# filtering out values from the model that do not match our observed results
post_rate <- prior_dist[sales == 9]

# checking if there are any results left
length(post_rate)
## [1] 4183

What we are doing here is repeating the sampling process multiple times and generating a random conversion rate after each iteration. The reason for filtering out the sample draws is that we want to keep the data that we observed in reality. Which is, when the marketing team did the process, they generated the conversion rate of 0.3913 by making 9 sales.

What is the Real Conversion Rate?

The answer to this question is not a single number. It is a probability distribution of possible rates of conversion. The distribution of these possible conversion rates is presented below.

#plotting the posterior distribution
post_rate_hist = hist(post_rate, xlim = c(0,1), main = "Histogram of Posterior Distribution")

From the observed distribution of possible conversion rates, we can see that the most likely conversion rate should exist between 35 & 45 percent. We can also see, that the probability of conversion rate over 60% or below 20% is highly unlikely.

We can calculate the probabilities of the possible conversion rates by counting the frequecny of each bar and dividing it by the total number of draws.

To calculate the probability of conversion rate between 40% and 45%, we do the following maths

# sum of frequency of draws between 0.40 and 0.45 divided by total draws
post_rate_hist$counts[8]/length(post_rate)
## [1] 0.1365049

Here, we have a 20% probability that the conversion rate would fall between 40-45 percent.

Answering Some More Questions

Which Strategy is Better?

We can use the model we created to answer comparative questions regarding marketing strategies. For instance, we can compare conversion rates for two marketing strategies.

Say, the marketing team tells that the rate of conversion was 20% when they deployed email marketing. The management now wants to know the probability that print media marketing is better than email marketing.

From our probability distribution, we can calculate the frequency of the percentage of conversions greater than 20%, divide it by the total number of draws to get the probability of print media marketing achieving a conversion rate greater than 20%.

sum(post_rate > 0.20) / length(post_rate)
## [1] 0.9909156

Using the Confidence Interval

We can use confidence interval to calculate the conversion rates that cover 95% of the overall distribution.

quantile(post_rate, c(0.025, 0.975))
##      2.5%     97.5% 
## 0.2252910 0.5946871

Here, it implies that we are 95% confident that the true conversion rate falls somewhere between 22% and 60%.

Conclusion

Bayesian probability allows us to distinguish truth from noise by taking into account all of the information that we have prior to running a test. This is exactly what we did here. We took into account the fact that the probability of conversion from 0 to 1 had equal chance of occuring. We then ran a simulation and kept the results that were in accordance with the data that we observed when we distributed brochuers. The end result was not a single number, rather a distribution of probabilities.

Such distributions allow the stakeholders to input their domain knowledge and try to estimate what the right answer to their question could be under the influence of uncertainty.