Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

FF Analysis

Key Takeaways

Renewal by Decision Type
  1. NNLOA is just below 60%: this is higher than most other forums.
  2. Opt Out performance is lower than what we’ve seen in other forums, which tend to be above 90%
  3. Opt Out volume is also higher than what we see in other forums, with nearly 40% of decisions being opt outs.
  4. Keep the 1.5 Ratio for NNLOA in mind throughout the analysis.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 32 19 13 1.5 59.4%
Opt Out 21 17 4 4.2 81.0%
Total 53 36 17 2.1 67.9%
Renewal Rate by Account Segment
  1. Large Public and Private and Selective
  • Most of the partnerships live in these two segments
    • A little separation between the segments, nearly 9%.
  1. Regional Public and International are low N
  • Regional Public is International performing the same and below the above two segments.
Renewal Ratio by Account Segment
AccountSegment N ATL Dropped Ratio Perc_Renewed
International 2 1 1 1.0 50.0%
Large Public & Private 29 22 7 3.1 75.9%
Regional Private 1 0 1 0.0 0.0%
Regional Public 6 3 3 1.0 50.0%
Selective 15 10 5 2.0 66.7%
Renewal by Decision Type & Segment
  1. [Large P&P]
  • Performance on NNLOA is just below the average, but close.
    • Performance on Opt Outs is perfect.
  1. [Selective]
  • NNLOA performance is one of the best we’ve seen across all segments and forums.
    • Opt Out performance is seriously lagging, both compared to Large P&P and compared to the NNLOA for Selectives.
Renewal Ratio by Account Segment
AccountSegment DecisionType N ATL Dropped Ratio Perc_Renewed
International Need New LOA 1 1 0 Inf 100.0%
International Opt Out 1 0 1 0.0 0.0%
Large Public & Private Need New LOA 17 10 7 1.4 58.8%
Large Public & Private Opt Out 12 12 0 Inf 100.0%
Regional Private Need New LOA 1 0 1 0.0 0.0%
Regional Public Need New LOA 3 1 2 0.5 33.3%
Regional Public Opt Out 3 2 1 2.0 66.7%
Selective Need New LOA 10 7 3 2.3 70.0%
Selective Opt Out 5 3 2 1.5 60.0%
GRP 3-4
  1. [Large P&P] The Opt Out performance for GRP 3-4 is the same as opt out performance for non GRPs
  2. No NNLOAs, all Opt Outs, and all renewed.
Renewal Ratio by Navigate Bundling
AccountSegment DecisionType GRP3-4 N ATL Dropped Ratio Perc_Renewed
International Need New LOA 0 1 1 0 Inf 100.0%
International Opt Out 0 1 0 1 0.0 0.0%
Large Public & Private Need New LOA 0 16 9 7 1.3 56.2%
Large Public & Private Need New LOA 1 1 1 0 Inf 100.0%
Large Public & Private Opt Out 0 6 6 0 Inf 100.0%
Large Public & Private Opt Out 1 6 6 0 Inf 100.0%
Regional Private Need New LOA 0 1 0 1 0.0 0.0%
Regional Public Need New LOA 0 3 1 2 0.5 33.3%
Regional Public Opt Out 0 1 0 1 0.0 0.0%
Regional Public Opt Out 1 2 2 0 Inf 100.0%
Selective Need New LOA 0 10 7 3 2.3 70.0%
Selective Opt Out 0 4 2 2 1.0 50.0%
Selective Opt Out 1 1 1 0 Inf 100.0%
Impact Interaction Volume Histogram
  1. Overall medium-low engagement: with a sizable number of partners in 0, 1, 2-3 categories.
  2. Peak around 3.
  3. Some heavier utilizers, but more sparsely distributed than other LOBs.

##### Impact Interaction Volume Histogram Split by Decision

  1. Heavy Opt Out distribution near 2 & 3.
  2. Heaviest users are NNLOAs.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. [NNLOA] performance varies a lot due to small N.
  2. [NNLOA] performance at or beyond 5 appears solid - 9/11 renewed.
  3. [Opt Out] Some minor evidence suggesting that 4+ impact interactions is a threshold.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 1 2 0.5 33.3%
Need New LOA 1 5 2 3 0.7 40.0%
Need New LOA 2 4 2 2 1.0 50.0%
Need New LOA 3 5 3 2 1.5 60.0%
Need New LOA 4 4 2 2 1.0 50.0%
Need New LOA 5 2 2 0 Inf 100.0%
Need New LOA 6 1 0 1 0.0 0.0%
Need New LOA 7 2 2 0 Inf 100.0%
Need New LOA 9 2 1 1 1.0 50.0%
Need New LOA 12 3 3 0 Inf 100.0%
Need New LOA 13 1 1 0 Inf 100.0%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 2 1 1 1.0 50.0%
Opt Out 2 4 3 1 3.0 75.0%
Opt Out 3 7 6 1 6.0 85.7%
Opt Out 4 1 0 1 0.0 0.0%
Opt Out 5 2 2 0 Inf 100.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 7 2 2 0 Inf 100.0%
Renewal by Impact Interaction Volume, (Binned)
  1. [NNLOA] Expected monotonic increase, with each bin showing a higher Ratio.
Renewal Ratio for Impact Interaction Volume
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 1 2 0.5 33.3%
Need New LOA 1 5 2 3 0.7 40.0%
Need New LOA 2-3 9 5 4 1.2 55.6%
Need New LOA 4-6 7 4 3 1.3 57.1%
Need New LOA 7+ 8 7 1 7.0 87.5%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 2 1 1 1.0 50.0%
Opt Out 2-3 11 9 2 4.5 81.8%
Opt Out 4-6 4 3 1 3.0 75.0%
Opt Out 7+ 2 2 0 Inf 100.0%
Renewal by Previous Yr Impact Interaction Volume & Decision
  1. [NNLOA] No clear pattern. 0 or 1 in the pervious year does not perform well.
  2. [Opt Out] Most of the drops are in the 3 or 4 bucket.
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2 0 2 0.0 0.0%
Need New LOA 1 4 1 3 0.3 25.0%
Need New LOA 2 5 2 3 0.7 40.0%
Need New LOA 3 3 3 0 Inf 100.0%
Need New LOA 4 3 2 1 2.0 66.7%
Need New LOA 5 5 5 0 Inf 100.0%
Need New LOA 6 1 0 1 0.0 0.0%
Need New LOA 7 1 0 1 0.0 0.0%
Need New LOA 8 1 0 1 0.0 0.0%
Need New LOA 9 1 1 0 Inf 100.0%
Need New LOA 10 1 1 0 Inf 100.0%
Need New LOA NA 5 4 1 4.0 80.0%
Opt Out 0 1 1 0 Inf 100.0%
Opt Out 2 1 1 0 Inf 100.0%
Opt Out 3 3 2 1 2.0 66.7%
Opt Out 4 2 0 2 0.0 0.0%
Opt Out 5 1 1 0 Inf 100.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 8 1 1 0 Inf 100.0%
Opt Out 16 1 1 0 Inf 100.0%
Opt Out NA 10 9 1 9.0 90.0%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract in the previous year.

  1. Binning does not help to distinguish healthy vs. not. Perhaps 2+ interactions for NNLOA is a threshold, but there is not enough data to support this.
Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2 0 2 0.0 0.0%
Need New LOA 1 4 1 3 0.3 25.0%
Need New LOA 2-3 8 5 3 1.7 62.5%
Need New LOA 4-6 9 7 2 3.5 77.8%
Need New LOA 7+ 4 2 2 1.0 50.0%
Need New LOA NA 5 4 1 4.0 80.0%
Opt Out 0 1 1 0 Inf 100.0%
Opt Out 2-3 4 3 1 3.0 75.0%
Opt Out 4-6 4 2 2 1.0 50.0%
Opt Out 7+ 2 2 0 Inf 100.0%
Opt Out NA 10 9 1 9.0 90.0%
Renewals Previous Yr and Current Yr Interaction Volume

Only looking at NNLOAs in the below table.

  1. Increasing utilization in the final year is a good indicator of renewal. If you look at 0, 1, and 2-3, getting these partners to increase utilization is better than maintaining or dropping in utilization.
  2. If already a heavy user, 4+, then some mixed results, but Maintenance or Increase should still be the goal.
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_Bins II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
0 2-3 2 0 2 0 0.0%
1 1 2 0 2 0 0.0%
1 4-6 2 1 1 1 50.0%
2-3 0 1 1 0 Inf 100.0%
2-3 1 1 0 1 0 0.0%
2-3 2-3 3 2 1 2 66.7%
2-3 7+ 3 2 1 2 66.7%
4-6 0 1 0 1 0 0.0%
4-6 1 1 1 0 Inf 100.0%
4-6 2-3 3 2 1 2 66.7%
4-6 4-6 2 2 0 Inf 100.0%
4-6 7+ 2 2 0 Inf 100.0%
7+ 0 1 0 1 0 0.0%
7+ 4-6 1 0 1 0 0.0%
7+ 7+ 2 2 0 Inf 100.0%
NA 1 1 1 0 Inf 100.0%
NA 2-3 1 1 0 Inf 100.0%
NA 4-6 2 1 1 1 50.0%
NA 7+ 1 1 0 Inf 100.0%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. Significant. Increasing impact interactions significantly increases the probability of renewal.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -0.59 0.597 -0.99 0.321
Impact Interaction Volume 0.25 0.133 1.86 0.063
Penetration by Event Grouping
  1. Some interesting differences between Opt Outs and NNLOA:
  • NNLOAs higher in PLWs and Onsites, Experiences, and Expert Calls
    • Opt Outs higher in Events & Research Interviews
  1. Similarities: Services and SL Led are closely
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 56.2% 9.4% 46.9% 31.2% 18.8% 18.8% 56.2%
Opt Out 61.9% 9.5% 47.6% 52.4% 4.8% 9.5% 19.0%
Renewal by Event Consumption
  1. [NNLOA] Small bump of 7% above the average, while not consuming this Impact Interaction is a 9% drop below NNLOA.
  2. [Opt Outs] Similar to NNLOA insofar as consumption is a small bump above the Opt Out average, while not consuming is a drop below the average.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 14 7 7 1.0 50.0%
Need New LOA 1 18 12 6 2.0 66.7%
Opt Out 0 8 6 2 3.0 75.0%
Opt Out 1 13 11 2 5.5 84.6%
Renewal by Service Consumption

N too low, but minor evidence that there is positive benefit to this consumption.

Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 29 16 13 1.2 55.2%
Need New LOA 1 3 3 0 Inf 100.0%
Opt Out 0 19 15 4 3.8 78.9%
Opt Out 1 2 2 0 Inf 100.0%
Renewal by SL Consumption
  1. [NNLOA] Decent lift for having an SL Led Interaction - 7% above NNLOA average, 14% bump above not having one.
  2. [Opt Out] Reverse relationship, having an SL Led interaction decreases likelihood to renew.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 17 9 8 1.1 52.9%
Need New LOA 1 15 10 5 2.0 66.7%
Opt Out 0 11 10 1 10.0 90.9%
Opt Out 1 10 7 3 2.3 70.0%
Renewal by Research Interview Consumption
  1. [NNLOA] Relatively decent lift above average NNLOA renewal rate - 11.5%
  2. [NNLOA] Difference between consuming and not consuming is roughly 15.5%
  3. [Opt Out] Decent lift above the Opt Out average as well, 10%.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 22 12 10 1.2 54.5%
Need New LOA 1 10 7 3 2.3 70.0%
Opt Out 0 10 7 3 2.3 70.0%
Opt Out 1 11 10 1 10.0 90.9%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Onsite and/or PLW positively associated with renewal! Small N, though.
  2. [NNLOA] Close to a 7% bump above the average and a 9% bump above not having an onsite.
  3. [Opt Out] N is too low.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 26 15 11 1.4 57.7%
Need New LOA 1 6 4 2 2.0 66.7%
Opt Out 0 20 16 4 4.0 80.0%
Opt Out 1 1 1 0 Inf 100.0%
Renewal by Experience Consumption
  1. [NNLOA] Very similar to Onsites. Close to a 7% bump above the average and a 9% bump above not having an onsite.
  2. [Opt Out] N is too low.
Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 26 15 11 1.4 57.7%
Need New LOA 1 6 4 2 2.0 66.7%
Opt Out 0 19 15 4 3.8 78.9%
Opt Out 1 2 2 0 Inf 100.0%
Renewal by Expert Call Consumption
  1. [NNLOA] Large lift from consuming an expert call: 13% above the average and ~30% above not consuming an expert call
  2. [Opt Out] Too little data to support the negative relationship.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 14 6 8 0.8 42.9%
Need New LOA 1 18 13 5 2.6 72.2%
Opt Out 0 17 15 2 7.5 88.2%
Opt Out 1 4 2 2 1.0 50.0%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: Events, Research Interview, and Service touches are positively correlated.
  2. Negative correlation with Renewal: None
  3. Positive Halo Effect: Events are correlated with Onsites, Service and Expert Calls.

##### Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only.

  1. Events are significant.
  2. Research interviews are the next closest.
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) -0.63 0.683 -0.93 0.35
II_Events_Volume 0.68 0.363 1.88 0.06
II_SLLed_Volume 0.20 0.536 0.37 0.71
II_ResearchInterview_Volume 1.22 0.929 1.32 0.19
II_PLW_Onsite_Volume -0.99 1.300 -0.76 0.45
II_ExpertCall_Volume 0.01 0.332 0.04 0.97
II_Experience_Volume -1.22 1.303 -0.93 0.35