Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

ITF Analysis

Key Takeaways

Renewal by Decision Type
  1. NNLOA is hovering just above 40%: the lowest of all programs looked at thus far (AAF, AF, BAF, GRP)
  2. Opt Out performance and volume is helping overall renewal rate, as opt outs are ~29% of total decisions.
  3. Keep the 0.7 Ratio for NNLOA in mind throughout the analysis.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 47 19 28 0.7 40.4%
Opt Out 19 17 2 8.5 89.5%
Total 66 36 30 1.2 54.5%
Renewal Rate by Account Segment
  1. Some consistency in performance across segments:
  • Regional Public, Selective, and Regional Private all performing poorly.
  1. Large P&P has the highest performance and the nearly half of partnerships.
Renewal Ratio by Account Segment
AccountSegment N ATL Dropped Ratio Perc_Renewed
International 2 1 1 1.0 50.0%
Large Public & Private 32 23 9 2.6 71.9%
Regional Private 8 3 5 0.6 37.5%
Regional Public 11 5 6 0.8 45.5%
Selective 13 4 9 0.4 30.8%
Renewal by Decision Type & Segment
  1. [Large P&P]
  • Performance on NNLOA is roughly what we have seen in other flagship forums
    • Performance on Opt Outs is superb - perfect score.
  1. [Regionals & Selective]
  • NNLOA is a particular problem for Regional Public and Selective
    • Selective NNLOA is particularly poor.
    • Opt Out Performance appears to be OK, even with low N.
Renewal Ratio by Account Segment
AccountSegment DecisionType N ATL Dropped Ratio Perc_Renewed
International Need New LOA 2 1 1 1.0 50.0%
Large Public & Private Need New LOA 21 12 9 1.3 57.1%
Large Public & Private Opt Out 11 11 0 Inf 100.0%
Regional Private Need New LOA 8 3 5 0.6 37.5%
Regional Public Need New LOA 7 2 5 0.4 28.6%
Regional Public Opt Out 4 3 1 3.0 75.0%
Selective Need New LOA 9 1 8 0.1 11.1%
Selective Opt Out 4 3 1 3.0 75.0%
GRP 3-4
  1. The performance for GRP 3-4 does not look very different across Large Public & Private Decision Types.
  2. Perhaps a small boost for Selective and Regional Public on Opt Outs - super low N.
Renewal Ratio by Navigate Bundling
AccountSegment DecisionType GRP3-4 N ATL Dropped Ratio Perc_Renewed
International Need New LOA 0 2 1 1 1.0 50.0%
Large Public & Private Need New LOA 0 20 11 9 1.2 55.0%
Large Public & Private Need New LOA 1 1 1 0 Inf 100.0%
Large Public & Private Opt Out 0 4 4 0 Inf 100.0%
Large Public & Private Opt Out 1 7 7 0 Inf 100.0%
Regional Private Need New LOA 0 8 3 5 0.6 37.5%
Regional Public Need New LOA 0 7 2 5 0.4 28.6%
Regional Public Opt Out 0 2 1 1 1.0 50.0%
Regional Public Opt Out 1 2 2 0 Inf 100.0%
Selective Need New LOA 0 9 1 8 0.1 11.1%
Selective Opt Out 0 3 2 1 2.0 66.7%
Selective Opt Out 1 1 1 0 Inf 100.0%
Impact Interaction Volume Histogram
  1. Overall low engagement: with a lot of partners in 0, 1, 2 categories.
  2. Thick right tail suggests some heavier users.

##### Impact Interaction Volume Histogram Split by Decision

  1. Opt Outs spread out.
  2. Relatively uniform distribution of Opt Outs and NNLOA in the tail.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. NNLOA performance varies a lot due to small N.
  2. NNLOA after 3 seems to be the threshold.
  3. Utilization at 0, 1, 2 for NNLOA is deadly and 25 of the 66 partnerships fall in these buckets.
  4. Opt Out performance is invariant to Impact interaction volume.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 10 1 9 0.1 10.0%
Need New LOA 1 7 2 5 0.4 28.6%
Need New LOA 2 8 3 5 0.6 37.5%
Need New LOA 3 8 6 2 3.0 75.0%
Need New LOA 4 5 2 3 0.7 40.0%
Need New LOA 5 3 2 1 2.0 66.7%
Need New LOA 6 2 0 2 0.0 0.0%
Need New LOA 7 2 1 1 1.0 50.0%
Need New LOA 9 2 2 0 Inf 100.0%
Opt Out 0 3 3 0 Inf 100.0%
Opt Out 1 4 4 0 Inf 100.0%
Opt Out 2 4 2 2 1.0 50.0%
Opt Out 3 1 1 0 Inf 100.0%
Opt Out 4 2 2 0 Inf 100.0%
Opt Out 6 2 2 0 Inf 100.0%
Opt Out 7 1 1 0 Inf 100.0%
Opt Out 8 2 2 0 Inf 100.0%
Renewal by Impact Interaction Volume, (Binned)
  1. [NNLOA] the II Volume Bins look off because of 4-6!
Renewal Ratio for Impact Interaction Volume
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 1 2 0.5 33.3%
Need New LOA 1 2 1 1 1.0 50.0%
Need New LOA 2-3 8 6 2 3.0 75.0%
Need New LOA 4-6 6 2 4 0.5 33.3%
Need New LOA 7+ 2 2 0 Inf 100.0%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 3 3 0 Inf 100.0%
Opt Out 2-3 1 1 0 Inf 100.0%
Opt Out 4-6 3 3 0 Inf 100.0%
Opt Out 7+ 2 2 0 Inf 100.0%
Renewal by Impact Interaction Volume, (RE-Binned)
  1. [NNLOA] If we go with 3+, those at 3 and 6+ smooth out the dip.
Renewal Ratio for Large Public & Private by Service Consumption Bins
DecisionType II_Volume_BinsV2 N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 1 2 0.5 33.3%
Need New LOA 1 2 1 1 1.0 50.0%
Need New LOA 2 3 1 2 0.5 33.3%
Need New LOA 3+ 13 9 4 2.2 69.2%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 3 3 0 Inf 100.0%
Opt Out 2 1 1 0 Inf 100.0%
Opt Out 3+ 5 5 0 Inf 100.0%
Renewal by Previous Yr Impact Interaction Volume & Decision
  1. [NNLOA] It also looks like prior year engagement below 3 is a good indicator of poor performance.
  2. [Opt Out] Invariance.
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 5 1 4 0.2 20.0%
Need New LOA 1 5 2 3 0.7 40.0%
Need New LOA 2 8 0 8 0.0 0.0%
Need New LOA 3 6 5 1 5.0 83.3%
Need New LOA 4 4 2 2 1.0 50.0%
Need New LOA 5 4 2 2 1.0 50.0%
Need New LOA 6 5 2 3 0.7 40.0%
Need New LOA 7 3 2 1 2.0 66.7%
Need New LOA 8 1 0 1 0.0 0.0%
Need New LOA 9 3 2 1 2.0 66.7%
Need New LOA NA 3 1 2 0.5 33.3%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 2 2 0 Inf 100.0%
Opt Out 2 1 0 1 0.0 0.0%
Opt Out 3 2 1 1 1.0 50.0%
Opt Out 7 2 2 0 Inf 100.0%
Opt Out 9 1 1 0 Inf 100.0%
Opt Out NA 9 9 0 Inf 100.0%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract in the previous year.

  1. Re-binning shows that 3+ in the prior year is a decent jump above the NNLOA average.
Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_BinsV2 N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 5 1 4 0.2 20.0%
Need New LOA 1 5 2 3 0.7 40.0%
Need New LOA 2 8 0 8 0.0 0.0%
Need New LOA 3+ 26 15 11 1.4 57.7%
Need New LOA NA 3 1 2 0.5 33.3%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 2 2 0 Inf 100.0%
Opt Out 2 1 0 1 0.0 0.0%
Opt Out 3+ 5 4 1 4.0 80.0%
Opt Out NA 9 9 0 Inf 100.0%
Renewals Previous Yr and Current Yr Interaction Volume

Only looking at NNLOAs in the below table.

  1. Maintenance looks like it is key. 16 fall in the 3+,3+ bucket and renew at a high rate.
  2. Dropping to 0 is undesirable.
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_BinsV2 II_Volume_BinsV2 N ATL Dropped Ratio Perc_Renewed
0 0 2 0 2 0.0 0.0%
0 1 3 1 2 0.5 33.3%
1 1 2 1 1 1.0 50.0%
1 3+ 3 1 2 0.5 33.3%
2 0 5 0 5 0.0 0.0%
2 2 1 0 1 0.0 0.0%
2 3+ 2 0 2 0.0 0.0%
3+ 0 2 0 2 0.0 0.0%
3+ 1 2 0 2 0.0 0.0%
3+ 2 6 3 3 1.0 50.0%
3+ 3+ 16 12 4 3.0 75.0%
NA 0 1 1 0 Inf 100.0%
NA 2 1 0 1 0.0 0.0%
NA 3+ 1 0 1 0.0 0.0%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. Significant. More impact interactions does translate to a higher percentage renewal rate.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -1.24 0.509 -2.44 0.015
Impact Interaction Volume 0.31 0.145 2.14 0.032
Penetration by Event Grouping
  1. A couple of differences across the categories, with the rest of the categories being roughly equivalent:
  • NNLOAs higher in Events penetration.
    • Opt Outs higher in Service penetration.
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 51.1% 0.0% 40.4% 25.5% 12.8% 0.0% 46.8%
Opt Out 42.1% 10.5% 42.1% 26.3% 10.5% 0.0% 47.4%
Renewal by Event Consumption
  1. [NNLOA] Get them to an event. Nearly double the renewal rate!
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 23 6 17 0.4 26.1%
Need New LOA 1 24 13 11 1.2 54.2%
Opt Out 0 11 9 2 4.5 81.8%
Opt Out 1 8 8 0 Inf 100.0%
Renewal by Service Consumption
  1. N to low to make meaning here.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 47 19 28 0.7 40.4%
Opt Out 0 17 15 2 7.5 88.2%
Opt Out 1 2 2 0 Inf 100.0%
Renewal by SL Consumption
  1. [NNLOA] Decent lift for having an SL Led Interaction - 12% above NNLOA average, 20% bump above not having one.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 28 9 19 0.5 32.1%
Need New LOA 1 19 10 9 1.1 52.6%
Opt Out 0 11 10 1 10.0 90.9%
Opt Out 1 8 7 1 7.0 87.5%
Renewal by Research Interview Consumption
  1. [NNLOA] Relatively high lift above average NNLOA renewal rate - 18%
  2. [NNLOA] Difference between consuming and not consuming is roughly 24%
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 35 12 23 0.5 34.3%
Need New LOA 1 12 7 5 1.4 58.3%
Opt Out 0 14 12 2 6.0 85.7%
Opt Out 1 5 5 0 Inf 100.0%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Onsite and/or PLW positively associated with renewal! Small N, though.
  2. [NNLOA] Close to a 10% bump above the average, low relative to other utilization types.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 41 16 25 0.6 39.0%
Need New LOA 1 6 3 3 1.0 50.0%
Opt Out 0 17 15 2 7.5 88.2%
Opt Out 1 2 2 0 Inf 100.0%
Renewal by Experience Consumption
  1. No data.
Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 47 19 28 0.7 40.4%
Opt Out 0 19 17 2 8.5 89.5%
Renewal by Expert Call Consumption
  1. [NNLOA] No difference between engagement and not. Perfectly at average.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 25 10 15 0.7 40.0%
Need New LOA 1 22 9 13 0.7 40.9%
Opt Out 0 10 10 0 Inf 100.0%
Opt Out 1 9 7 2 3.5 77.8%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: Events stand out, with Research Interview and SL Led close second.
  2. Negative correlation with Renewal: Expert Center Calls
  3. Halo Effect: Events highly correlated with the other categories.
  4. SL Led and Events are highly, highly correlated.

##### Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only.

  1. Research Interviews and Events are significant.
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) -1.29 0.569 -2.27 0.02
II_Events_Volume 1.57 0.595 2.63 0.01
II_SLLed_Volume -0.24 0.373 -0.64 0.52
II_ResearchInterview_Volume 0.88 0.524 1.67 0.09
II_PLW_Onsite_Volume 0.27 1.190 0.23 0.82
II_ExpertCall_Volume -1.06 0.694 -1.53 0.13
II_Service_Volume NA NA NA NA