5.6 A firm that sells software services has been piloting a new product and has records of 500 customers who have either bought the services or decided not to. The target value is the estimated profit from each sale (excluding sales costs). The global mean is $2128. However, the cost of the sales effort is not cheap—the company figures it comes to $2500 for each of the 500 customers (whether they buy or not). The firm developed a predictive model in hopes of being able to identify the top spenders in the future. The lift and decile charts for the validation set are shown in Figure 5.13.

  1. If the company begins working with a new set of 1000 leads to sell the same services, similar to the 500 in the pilot study, without any use of predictive modeling to target sales efforts, what is the estimated profit?

Since the global average of each sale is $2128 and the cost is $2500 for each person(cost of sales effots). We would expect them to lose $372000.((2500-2128)*1000)

  1. If the firm wants the average profit on each sale to at least double the sales effort cost, and applies an appropriate cutoff with this predictive model to a new set of 1000 leads, how far down the new list of 1000 should it proceed (how many deciles)?

According to Figure 5.13 if the firm wants the average profit to at least double the sales effort costs then it should should not go down the ranked list past the 100th entry.(include 1st decile)

  1. Still considering the new list of 1000 leads, if the company applies this predictive model with a lower cutoff of $2500, how far should it proceed down the ranked leads, in terms of deciles?

If we want the cutoff to be at $2500 then we pick the mean response to be above 1.174812 (2500/2128). In this case that means we stop at the 6th decile (it looks like that is a little more than 1.17) or proceed down the ranked leads up to the 60th record.

  1. Why use this two-stage process for predicting sales—why not simply develop a model for predicting profit for the 1000 new leads?

This two-stage process approach lets us use the model to maximize the likelihood of getting an outcome that we desire and allows us to easily determine what our cutoff should be depending on the scenario. While developing a model only takes in data and spits out a number, It doesn’t tell us much about how that number compared to the rest of the records.

5.7 Table 5.7 shows a small set of predictive model validation results for a classification model, with both actual values and propensities.

a <- c(0.03, 0, # just copied table 5.7 and did some manipulations
       0.52, 0,
       0.38, 0,
       0.82, 1,
       0.33, 0,
       0.42, 0,
       0.55, 1,
       0.59, 0,
       0.09, 0,
       0.21, 0,
       0.43, 0,
       0.04, 0,
       0.08, 0,
       0.13, 0,
       0.01, 0,
       0.79, 1,
       0.42, 0,
       0.29, 0,
       0.08, 0,
       0.02, 0)
b <- a[2*(1:20)]
c <- data.frame(prob=a[seq(from=1,to=40,by=2)],
                actual = b)
df <- c[order(-c$prob),] #show in descending order
print(df, row.names = FALSE) #get rid of that id column
 prob actual
 0.82      1
 0.79      1
 0.59      0
 0.55      1
 0.52      0
 0.43      0
 0.42      0
 0.42      0
 0.38      0
 0.33      0
 0.29      0
 0.21      0
 0.13      0
 0.09      0
 0.08      0
 0.08      0
 0.04      0
 0.03      0
 0.02      0
 0.01      0
  1. Calculate error rates, sensitivity, and specificity using cutoffs of 0.25, 0.5, and 0.75.

For the cutoff of 0.25 (that is if Propensity(1)>.25 -> belongs to group 1)

predict/actual 1 0
1 3 8
0 0 9

For the cutoff of 0.5 (that is if Propensity(1)>.5 -> belongs to group 1)

predict/actual 1 0
1 3 2
0 0 15

For the cutoff of 0.75 (that is if Propensity(1)>.75 -> belongs to group 1)

predict/actual 1 0
1 2 0
0 1 18
  1. Create a decile-wise lift chart in R.
gain <- gains::gains(df$actual, df$prob)
barplot(gain$mean.resp / mean(df$actual), 
        names.arg = gain$depth, 
        xlab = "Percentile", 
        ylab = "Mean Response", 
        main = "Decile-wise lift chart")