Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

SAF Analysis

Key Takeaways

Renewal by Decision Type
  1. NNLOA is hovering just above 50%: on part with other flagship programs (AAF, BAF, GRP)
  2. Opt Out performance and volume is helping overall renewal rate, as opt outs are ~31% of total decisions.
  3. Keep the 1.0 Ratio for NNLOA in mind throughout the analysis.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 49 25 24 1.0 51.0%
Opt Out 22 20 2 10.0 90.9%
Total 71 45 26 1.7 63.4%
Renewal Rate by Account Segment
  1. Some consistency in performance across segments:
  • Selective, Large P&P, and International are all performing at or close to 70%.
  1. Regionals are performing poorly
  • Regional Private N is too low
    • Regional Public is the second largest constituency and performing below the average.
Renewal Ratio by Account Segment
AccountSegment N ATL Dropped Ratio Perc_Renewed
International 6 4 2 2.0 66.7%
Large Public & Private 30 21 9 2.3 70.0%
Regional Private 5 1 4 0.2 20.0%
Regional Public 17 10 7 1.4 58.8%
Selective 13 9 4 2.2 69.2%
Renewal by Decision Type & Segment
  1. [Large P&P]
  • Performance on NNLOA is fairly good.
    • Performance on Opt Outs is solid.
  1. [Regional Public]
  • The size of the opt out pool is disguising the poor performance of NNLOA
  1. [Regional Private] Only the Opt Out Renewed.
  2. [Selective]
  • Similar to Regional Public, the opt outs are performing very well, while NNLOA is less performant.
Renewal Ratio by Account Segment
AccountSegment DecisionType N ATL Dropped Ratio Perc_Renewed
International Need New LOA 6 4 2 2.0 66.7%
Large Public & Private Need New LOA 22 14 8 1.8 63.6%
Large Public & Private Opt Out 8 7 1 7.0 87.5%
Regional Private Need New LOA 4 0 4 0.0 0.0%
Regional Private Opt Out 1 1 0 Inf 100.0%
Regional Public Need New LOA 9 3 6 0.5 33.3%
Regional Public Opt Out 8 7 1 7.0 87.5%
Selective Need New LOA 8 4 4 1.0 50.0%
Selective Opt Out 5 5 0 Inf 100.0%
GRP 3-4
  1. [Large P&P] The Opt Out performance for GRP 3-4 looks pretty good, while it is less so for non-GRP 3-4. First time we are seeing this, but temper conclusions as the N is pretty small.
Renewal Ratio by Navigate Bundling
AccountSegment DecisionType GRP3-4 N ATL Dropped Ratio Perc_Renewed
International Need New LOA 0 6 4 2 2.0 66.7%
Large Public & Private Need New LOA 0 21 13 8 1.6 61.9%
Large Public & Private Need New LOA 1 1 1 0 Inf 100.0%
Large Public & Private Opt Out 0 3 2 1 2.0 66.7%
Large Public & Private Opt Out 1 5 5 0 Inf 100.0%
Regional Private Need New LOA 0 4 0 4 0.0 0.0%
Regional Private Opt Out 0 1 1 0 Inf 100.0%
Regional Public Need New LOA 0 9 3 6 0.5 33.3%
Regional Public Opt Out 0 6 5 1 5.0 83.3%
Regional Public Opt Out 1 2 2 0 Inf 100.0%
Selective Need New LOA 0 8 4 4 1.0 50.0%
Selective Opt Out 0 4 4 0 Inf 100.0%
Selective Opt Out 1 1 1 0 Inf 100.0%
Impact Interaction Volume Histogram
  1. Overall medium-low engagement: with a sizable number of partners in 0, 1, 2 categories.
  2. Peak around 4, with a second peak near 7.
  3. Thick right tail suggests some heavier users.

##### Impact Interaction Volume Histogram Split by Decision

  1. Surprising number of opt outs at 3.
  2. Some opt outs in the tail, but the 7 peak looks like it is composed mostly of NNLOAs.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. NNLOA performance varies a lot due to small N.
  2. NNLOA performance between 4 and 7 seems to follow the expected pattern of increasing renewal.
  3. Surprisingly, NNLOA performance for 8+ is pretty poor, but N is lower here.
  4. NNLOA - 4 Remains the threshold for increasing renewal.
  5. Opt Out performance is invariant to Impact interaction volume.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 4 1 3 0.3 25.0%
Need New LOA 1 4 0 4 0.0 0.0%
Need New LOA 2 7 3 4 0.8 42.9%
Need New LOA 3 5 2 3 0.7 40.0%
Need New LOA 4 9 6 3 2.0 66.7%
Need New LOA 5 4 3 1 3.0 75.0%
Need New LOA 6 4 3 1 3.0 75.0%
Need New LOA 7 7 6 1 6.0 85.7%
Need New LOA 8 2 1 1 1.0 50.0%
Need New LOA 9 2 0 2 0.0 0.0%
Need New LOA 12 1 0 1 0.0 0.0%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 3 3 0 Inf 100.0%
Opt Out 2 2 1 1 1.0 50.0%
Opt Out 3 6 6 0 Inf 100.0%
Opt Out 4 3 3 0 Inf 100.0%
Opt Out 5 2 2 0 Inf 100.0%
Opt Out 7 2 2 0 Inf 100.0%
Opt Out 8 1 0 1 0.0 0.0%
Opt Out 10 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume, (Binned)
  1. [NNLOA] Expected monotonic increase.
Renewal Ratio for Impact Interaction Volume
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2 0 2 0 0.0%
Need New LOA 1 1 0 1 0 0.0%
Need New LOA 2-3 6 4 2 2 66.7%
Need New LOA 4-6 8 6 2 3 75.0%
Need New LOA 7+ 5 4 1 4 80.0%
Opt Out 1 1 1 0 Inf 100.0%
Opt Out 2-3 3 3 0 Inf 100.0%
Opt Out 4-6 3 3 0 Inf 100.0%
Opt Out 7+ 1 0 1 0 0.0%
Renewal by Previous Yr Impact Interaction Volume & Decision
  1. [NNLOA] 5+ Impact Interactions in the Previous Year seems to indicate positive renewal in the subsequent year. Some contradicting evidence as 2 Impact Interactions also looks good.
  2. [Opt Out] Invariance.
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 6 3 3 1.0 50.0%
Need New LOA 1 8 2 6 0.3 25.0%
Need New LOA 2 6 4 2 2.0 66.7%
Need New LOA 3 7 3 4 0.8 42.9%
Need New LOA 4 6 2 4 0.5 33.3%
Need New LOA 5 6 4 2 2.0 66.7%
Need New LOA 6 6 4 2 2.0 66.7%
Need New LOA 7 1 1 0 Inf 100.0%
Need New LOA NA 3 2 1 2.0 66.7%
Opt Out 0 1 1 0 Inf 100.0%
Opt Out 2 2 1 1 1.0 50.0%
Opt Out 4 1 1 0 Inf 100.0%
Opt Out 5 2 2 0 Inf 100.0%
Opt Out 8 1 1 0 Inf 100.0%
Opt Out NA 15 14 1 14.0 93.3%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract in the previous year.

  1. Binning does not help to distinguish healthy vs. not. 2-3 and 4-6 are relatively similar.
Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 6 3 3 1.0 50.0%
Need New LOA 1 8 2 6 0.3 25.0%
Need New LOA 2-3 13 7 6 1.2 53.8%
Need New LOA 4-6 18 10 8 1.2 55.6%
Need New LOA 7+ 1 1 0 Inf 100.0%
Need New LOA NA 3 2 1 2.0 66.7%
Opt Out 0 1 1 0 Inf 100.0%
Opt Out 2-3 2 1 1 1.0 50.0%
Opt Out 4-6 3 3 0 Inf 100.0%
Opt Out 7+ 1 1 0 Inf 100.0%
Opt Out NA 15 14 1 14.0 93.3%
Renewals Previous Yr and Current Yr Interaction Volume

Only looking at NNLOAs in the below table.

  1. Increasing utilization in the final year is a good indicator of renewal. If you look at 0, 1, and 2-3, getting these partners to increase utilization is better than maintaining or dropping in utilization.
  2. If already a heavy user, 4+, then some mixed results, but Maintenance or Increase should still be the goal.
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_Bins II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
0 1 2 0 2 0.0 0.0%
0 4-6 2 2 0 Inf 100.0%
0 7+ 2 1 1 1.0 50.0%
1 0 2 0 2 0.0 0.0%
1 2-3 2 1 1 1.0 50.0%
1 4-6 2 0 2 0.0 0.0%
1 7+ 2 1 1 1.0 50.0%
2-3 0 1 1 0 Inf 100.0%
2-3 1 1 0 1 0.0 0.0%
2-3 2-3 4 1 3 0.3 25.0%
2-3 4-6 5 3 2 1.5 60.0%
2-3 7+ 2 2 0 Inf 100.0%
4-6 0 1 0 1 0.0 0.0%
4-6 1 1 0 1 0.0 0.0%
4-6 2-3 4 2 2 1.0 50.0%
4-6 4-6 6 5 1 5.0 83.3%
4-6 7+ 6 3 3 1.0 50.0%
7+ 4-6 1 1 0 Inf 100.0%
NA 2-3 2 1 1 1.0 50.0%
NA 4-6 1 1 0 Inf 100.0%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. Increasing impact interactions does not appear to significantly increase the probability of renewal.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -0.59 0.550 -1.07 0.285
Impact Interaction Volume 0.15 0.112 1.34 0.181
Penetration by Event Grouping
  1. A couple of differences across the categories, with the rest of the categories being roughly equivalent:
  • NNLOAs higher in Service penetration.
    • Opt Outs higher in SL Led penetration - by a nearly 20% margin.
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 63.3% 24.5% 53.1% 16.3% 10.2% 2.0% 59.2%
Opt Out 59.1% 18.2% 72.7% 13.6% 9.1% 0.0% 40.9%
Renewal by Event Consumption
  1. [NNLOA] Nearly no impact from attending an event vs. not.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 18 9 9 1.0 50.0%
Need New LOA 1 31 16 15 1.1 51.6%
Opt Out 0 9 8 1 8.0 88.9%
Opt Out 1 13 12 1 12.0 92.3%
Renewal by Service Consumption
  1. [NNLOA] Renewal nearly 16% above the average and a 20% difference in consumption vs. not.
  2. [Opt Out] N too low to react to negative relationship between service consumption and renewal.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 37 17 20 0.8 45.9%
Need New LOA 1 12 8 4 2.0 66.7%
Opt Out 0 18 17 1 17.0 94.4%
Opt Out 1 4 3 1 3.0 75.0%
Renewal by SL Consumption
  1. [NNLOA] Decent lift for having an SL Led Interaction - 6.7% above NNLOA average, 14% bump above not having one.
  2. [Opt Out] There seems to be a bit of benefit here, a 10% lift above not having had this interaction.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 23 10 13 0.8 43.5%
Need New LOA 1 26 15 11 1.4 57.7%
Opt Out 0 6 5 1 5.0 83.3%
Opt Out 1 16 15 1 15.0 93.8%
Renewal by Research Interview Consumption
  1. [NNLOA] Relatively decent lift above average NNLOA renewal rate - 11.5%
  2. [NNLOA] Difference between consuming and not consuming is roughly 14%
  3. [Opt Out] N too low.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 41 20 21 1.0 48.8%
Need New LOA 1 8 5 3 1.7 62.5%
Opt Out 0 19 17 2 8.5 89.5%
Opt Out 1 3 3 0 Inf 100.0%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Onsite and/or PLW positively associated with renewal! Small N, though.
  2. [NNLOA] Close to a 9% bump above the average and a 10% bump above not having an onsite.
  3. [Opt Out] N is too low to react to negative relationship between renewal and onsite consumption.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 44 22 22 1.0 50.0%
Need New LOA 1 5 3 2 1.5 60.0%
Opt Out 0 20 19 1 19.0 95.0%
Opt Out 1 2 1 1 1.0 50.0%
Renewal by Experience Consumption
  1. One instance of consumption - insufficient data.
Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 48 25 23 1.1 52.1%
Need New LOA 1 1 0 1 0.0 0.0%
Opt Out 0 22 20 2 10.0 90.9%
Renewal by Expert Call Consumption
  1. [NNLOA] No difference between engagement and not. Perfectly at average.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 20 10 10 1.0 50.0%
Need New LOA 1 29 15 14 1.1 51.7%
Opt Out 0 13 13 0 Inf 100.0%
Opt Out 1 9 7 2 3.5 77.8%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: SL Led stands out, with Research Interview coming in second.
  2. Negative correlation with Renewal: Onsites (low N, so ignore), Expert Calls, and Events (rare)
  3. Negative Halo Effect: SL Led is negatively related to Service, PLW/Onsite, and Expert Calls.

##### Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only.

  1. SL Led is closes to significance.
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) -0.83 0.555 -1.49 0.14
II_Events_Volume 0.11 0.198 0.57 0.57
II_SLLed_Volume 0.46 0.311 1.47 0.14
II_ResearchInterview_Volume 0.52 0.883 0.58 0.56
II_PLW_Onsite_Volume -0.05 0.837 -0.06 0.95
II_ExpertCall_Volume 0.16 0.250 0.64 0.52
II_Service_Volume 0.20 0.532 0.37 0.71