Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

BAF Analysis

Key Takeaways

Renewal by Decision Type
  1. NNLOA is hovering just above 50%, which is relatively low. Similar to AAF.
  2. Opt Out performance and volume is buoying overall renewal rate.
  3. Keep the 1.1 Ratio for NNLOA in mind throughout the analysis.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 61 32 29 1.1 52.5%
Opt Out 22 21 1 21.0 95.5%
Total 83 53 30 1.8 63.9%
Renewal Rate by Account Segment
  1. Some consistency in performance across segments.
  • Regional Public, Selective, and International all performing relatively similar
    • Large P&P suprisingly lower performing than other segments, with a large N
    • Regional Private: Struggling to maintain partners in this segment
Renewal Ratio by Account Segment
AccountSegment N ATL Dropped Ratio Perc_Renewed
International 7 5 2 2.5 71.4%
Large Public & Private 32 20 12 1.7 62.5%
Regional Private 9 2 7 0.3 22.2%
Regional Public 11 8 3 2.7 72.7%
Selective 24 18 6 3.0 75.0%
Renewal by Decision Type & Segment
  1. NNLOA in Large P&P lower than overall average.
  2. NNLOA in Regional Public and selectives is relatively solid. Meaning the bump from Opt Outs in those spaces accounts for less of the overall performance than we might have expected.
Renewal Ratio by Account Segment
AccountSegment DecisionType N ATL Dropped Ratio Perc_Renewed
International Need New LOA 4 2 2 1.0 50.0%
International Opt Out 3 3 0 Inf 100.0%
Large Public & Private Need New LOA 23 11 12 0.9 47.8%
Large Public & Private Opt Out 9 9 0 Inf 100.0%
Regional Private Need New LOA 9 2 7 0.3 22.2%
Regional Public Need New LOA 7 5 2 2.5 71.4%
Regional Public Opt Out 4 3 1 3.0 75.0%
Selective Need New LOA 18 12 6 2.0 66.7%
Selective Opt Out 6 6 0 Inf 100.0%
GRP 3-4
  1. Looks like BAF renews very well when the partner is of the 3-4 type.
Renewal Ratio by Navigate Bundling
GRP3-4 N ATL Dropped Ratio Perc_Renewed
0 73 43 30 1.4 58.9%
1 10 10 0 Inf 100.0%
Impact Interaction Volume Histogram
  1. Central Tendency is moving much closer to 3, but not 4!
  2. Still a decent number of renewals with 0-1 interactions in the year leading up to the renewal.

##### Impact Interaction Volume Histogram Split by Decision

  1. Opt Outs occupy an outsized portion of the 5+ range
  2. A lot of NNLOAs in the 0-1 range.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. At least 2 interactions are required to hit the average NNLOA rate.
  2. NNLOA climbs above the average around 4 interactions.
  3. Opt Outs with Low interactions will still renew at a high rate.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 11 3 8 0.4 27.3%
Need New LOA 1 11 4 7 0.6 36.4%
Need New LOA 2 9 5 4 1.2 55.6%
Need New LOA 3 14 7 7 1.0 50.0%
Need New LOA 4 6 4 2 2.0 66.7%
Need New LOA 5 3 2 1 2.0 66.7%
Need New LOA 6 3 3 0 Inf 100.0%
Need New LOA 7 1 1 0 Inf 100.0%
Need New LOA 9 1 1 0 Inf 100.0%
Need New LOA 11 2 2 0 Inf 100.0%
Opt Out 0 5 5 0 Inf 100.0%
Opt Out 2 5 5 0 Inf 100.0%
Opt Out 3 2 2 0 Inf 100.0%
Opt Out 4 2 2 0 Inf 100.0%
Opt Out 5 4 3 1 3.0 75.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 7 2 2 0 Inf 100.0%
Opt Out 9 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume, (Binned)
  1. Neater table of the above shows the same insight. 2+ to hit average, 4+ to be in good shape for NNLOA.
Renewal Ratio by Impact Interaction Volume (Binned)
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 11 3 8 0.4 27.3%
Need New LOA 1 11 4 7 0.6 36.4%
Need New LOA 2-3 23 12 11 1.1 52.2%
Need New LOA 4-6 12 9 3 3.0 75.0%
Need New LOA 7+ 4 4 0 Inf 100.0%
Opt Out 0 5 5 0 Inf 100.0%
Opt Out 2-3 7 7 0 Inf 100.0%
Opt Out 4-6 7 6 1 6.0 85.7%
Opt Out 7+ 3 3 0 Inf 100.0%
Renewal by Previous Yr Impact Interaction Volume & Decision
  1. 6+ interactions in the previous year looks like a winning ticket and may trump activity in the year leading up to the renewal decision.
  2. There is a decent amount of variation in the renewal rate below 5 interactions in the previous year. Suggesting other factors might play a more decisive role.
  3. Previous Year volume does not have an effect on Opt Outs.
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 4 1 3 0.3 25.0%
Need New LOA 1 12 6 6 1.0 50.0%
Need New LOA 2 13 4 9 0.4 30.8%
Need New LOA 3 7 5 2 2.5 71.4%
Need New LOA 4 9 5 4 1.2 55.6%
Need New LOA 5 3 1 2 0.5 33.3%
Need New LOA 6 2 2 0 Inf 100.0%
Need New LOA 7 3 3 0 Inf 100.0%
Need New LOA 8 1 0 1 0.0 0.0%
Need New LOA 9 2 2 0 Inf 100.0%
Need New LOA 14 1 1 0 Inf 100.0%
Need New LOA NA 4 2 2 1.0 50.0%
Opt Out 1 1 1 0 Inf 100.0%
Opt Out 2 2 2 0 Inf 100.0%
Opt Out 3 1 1 0 Inf 100.0%
Opt Out 4 2 2 0 Inf 100.0%
Opt Out 5 1 1 0 Inf 100.0%
Opt Out 6 2 1 1 1.0 50.0%
Opt Out NA 13 13 0 Inf 100.0%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract inthe previous year.

  1. Most of the findings above hold true when looking at the binned version of volume.
  2. The only difference is that the threshold might be 6 instead of 7+.
Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 4 1 3 0.3 25.0%
Need New LOA 1 12 6 6 1.0 50.0%
Need New LOA 2-3 20 9 11 0.8 45.0%
Need New LOA 4-6 14 8 6 1.3 57.1%
Need New LOA 7+ 7 6 1 6.0 85.7%
Need New LOA NA 4 2 2 1.0 50.0%
Opt Out 1 1 1 0 Inf 100.0%
Opt Out 2-3 3 3 0 Inf 100.0%
Opt Out 4-6 5 4 1 4.0 80.0%
Opt Out NA 13 13 0 Inf 100.0%
Renewals Previous Yr and Current Yr Interaction Volume

Only looking at NNLOAs in the below table.

  1. The table becomes a bit sparse due to all of the combinations.
  2. Overall, if you are in the 4-6 or 7+ the previous year, dropping down below 4-6 interaction volumes is correlated with lower renewal rates.
  3. Low engagement in the previous year tends to be correlated with low engagement in the subsequent year.
  • Even when you bump a school from 1 to 2-3, the rate does not jump much
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_Bins II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
0 0 1 0 1 0.0 0.0%
0 1 2 0 2 0.0 0.0%
0 2-3 1 1 0 Inf 100.0%
1 0 2 1 1 1.0 50.0%
1 1 2 1 1 1.0 50.0%
1 2-3 6 3 3 1.0 50.0%
1 4-6 2 1 1 1.0 50.0%
2-3 0 5 1 4 0.2 20.0%
2-3 1 5 2 3 0.7 40.0%
2-3 2-3 7 4 3 1.3 57.1%
2-3 4-6 2 1 1 1.0 50.0%
2-3 7+ 1 1 0 Inf 100.0%
4-6 0 2 0 2 0.0 0.0%
4-6 1 2 1 1 1.0 50.0%
4-6 2-3 4 2 2 1.0 50.0%
4-6 4-6 5 4 1 4.0 80.0%
4-6 7+ 1 1 0 Inf 100.0%
7+ 2-3 3 2 1 2.0 66.7%
7+ 4-6 2 2 0 Inf 100.0%
7+ 7+ 2 2 0 Inf 100.0%
NA 0 1 1 0 Inf 100.0%
NA 2-3 2 0 2 0.0 0.0%
NA 4-6 1 1 0 Inf 100.0%
Trend of Impact Interactions YoY, Binned

Note: (-1 = Negative Trend, 0 = Neutral, 1 = Positive Trend)

  1. Without knowing what the previous year volume was, you would still advocate for increased volume given the below.
  2. Stability YoY performs well below the average. We know from the above that this is
  3. Decreasing consumption YoY is correlated with higher renewal rates.
Renewal Ratio by Previous Year Impact Interaction
DecisionType Volume_Trend N ATL Dropped Ratio Perc_Renewed
Need New LOA -1 29 15 14 1.1 51.7%
Need New LOA 0 9 3 6 0.5 33.3%
Need New LOA 1 23 14 9 1.6 60.9%
Opt Out -1 5 4 1 4.0 80.0%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 15 15 0 Inf 100.0%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. The Estimate for Impact Interaction Volume is Positive -> increased consumption means increased likelihood to renew.
  2. This is statistically significant.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -0.34 0.374 -0.90 0.365
Impact Interaction Volume 0.36 0.129 2.79 0.005
Penetration by Event Grouping
  1. Predominantly, we see that BAF consumption involves Events, SL Led Impact, and Expert Calls.
  2. Very little in Service, Experience, and PLWs/Onsites.
  3. Large difference in Opt Outs and NNLOA for Research Interviews
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 50.8% 3.3% 34.4% 23.0% 6.6% 0.0% 49.2%
Opt Out 59.1% 4.5% 50.0% 22.7% 9.1% 0.0% 54.5%
Renewal by Event Consumption
  1. Nearly a 18% bump and the N relatively large in both bins.
  2. NNLOA with an Event are 9% above the average for NNLOA.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 30 13 17 0.8 43.3%
Need New LOA 1 31 19 12 1.6 61.3%
Opt Out 0 9 9 0 Inf 100.0%
Opt Out 1 13 12 1 12.0 92.3%
Renewal by Service Consumption
  1. Too few service consumption data points, but those who did use it renewed.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 59 30 29 1 50.8%
Need New LOA 1 2 2 0 Inf 100.0%
Opt Out 0 21 20 1 20 95.2%
Opt Out 1 1 1 0 Inf 100.0%
Renewal by SL Consumption
  1. Smaller bump for SL Led consumption - 7% - as compared to Events.
  2. Much closer to the average NNLOA renewal rate.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 40 20 20 1.0 50.0%
Need New LOA 1 21 12 9 1.3 57.1%
Opt Out 0 11 11 0 Inf 100.0%
Opt Out 1 11 10 1 10.0 90.9%
Renewal by Research Interview Consumption
  1. Relatively high lift above average NNLOA renewal rate - 13%
  2. Difference between consuming and not consuming is roughly 16%
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 47 23 24 1.0 48.9%
Need New LOA 1 14 9 5 1.8 64.3%
Opt Out 0 17 16 1 16.0 94.1%
Opt Out 1 5 5 0 Inf 100.0%
Renewal by PLW OR Onsite Consumption
  1. Onsite and/or PLW positively associated with renewal, but N is very low.
  2. Close to a 24% bump above the average.
  3. The average for those who don’t use the service is still relatively close to the average.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 57 29 28 1 50.9%
Need New LOA 1 4 3 1 3 75.0%
Opt Out 0 20 19 1 19 95.0%
Opt Out 1 2 2 0 Inf 100.0%
Renewal by Experience Consumption
  1. No data.
Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 61 32 29 1.1 52.5%
Opt Out 0 22 21 1 21.0 95.5%
Renewal by Expert Call Consumption
  1. Possibly the best discriminator.
  2. Those who consume this impact interaction are nearly 19% above the average.
  3. Compared to those who did not consume this impact interaction, the renewal rate is nearly double.
  4. N in both groups is rather large.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 31 11 20 0.6 35.5%
Need New LOA 1 30 21 9 2.3 70.0%
Opt Out 0 10 9 1 9.0 90.0%
Opt Out 1 12 12 0 Inf 100.0%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: Top 2 are Events and Expert Call, with Expert Call being the standout.
  2. Expert Calls and Research Interviews are highly correlated, as are Events and SL Led Impact Interactions. Suggesting tight pairing within these sets of impact interactions.
  3. Research Interviews and SL Led Impact Interactions are negatively correlated, meaning having one means you are less likely to have the other.

##### Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only. I also took out ‘Service’ which is not statistically significant but whose large estimate was distracting.

  1. Expert Calls is the only statistically significant result. It is positively associated with Renewal.
  2. If you play around with inclusion/exclusion, getting rid of ExpertCalls bumps Research Interviews up, not SL Led or Events.
  3. PLW and Onsite shows a negative relationship to Renewals when you take into account other impact interactions. I would not read much into this given the small N and other evidence above.
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) -0.98 0.449 -2.18 0.03
II_Events_Volume 0.19 0.346 0.54 0.59
II_SLLed_Volume 0.33 0.395 0.85 0.40
II_ResearchInterview_Volume 0.39 0.480 0.81 0.42
II_PLW_Onsite_Volume -0.20 1.616 -0.13 0.90
II_ExpertCall_Volume 0.86 0.356 2.41 0.02