Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

AF Analysis

Renewal by Decision Type
  1. NNLOA is hovering just above 55%: placed closed to BAF and AAF
  2. Opt Out performance and volume is helping overall renewal rate, as opt outs are ~32% of total decisions.
  3. Keep the 1.3 Ratio for NNLOA in mind throughout the analysis.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 68 38 30 1.3 55.9%
Opt Out 32 30 2 15.0 93.8%
Total 100 68 32 2.1 68.0%
Renewal Rate by Account Segment
  1. Some consistency in performance across segments:
  • Large Public & Private and Selective occupy high rates and most of the partnership
    • Regional Public and Regional Private perform similarly.
Renewal Ratio by Account Segment
AccountSegment N ATL Dropped Ratio Perc_Renewed
International 2 2 0 Inf 100.0%
Large Public & Private 48 36 12 3.0 75.0%
Regional Private 8 4 4 1.0 50.0%
Regional Public 15 7 8 0.9 46.7%
Selective 27 19 8 2.4 70.4%
Renewal by Decision Type & Segment
  1. [Large P&P & Selectives]
  • Performance on NNLOA is relatively high
    • Performance on Opt Outs is superb - perfect score.
  1. [Regionals]
  • NNLOA is a particular problem for Regional Public.
    • Opt Out Performance appears to be OK, even with low N.
Renewal Ratio by Account Segment
AccountSegment DecisionType N ATL Dropped Ratio Perc_Renewed
International Need New LOA 1 1 0 Inf 100.0%
International Opt Out 1 1 0 Inf 100.0%
Large Public & Private Need New LOA 32 20 12 1.7 62.5%
Large Public & Private Opt Out 16 16 0 Inf 100.0%
Regional Private Need New LOA 8 4 4 1.0 50.0%
Regional Public Need New LOA 9 2 7 0.3 22.2%
Regional Public Opt Out 6 5 1 5.0 83.3%
Selective Need New LOA 18 11 7 1.6 61.1%
Selective Opt Out 9 8 1 8.0 88.9%
GRP 3-4
  1. The performance for GRP 3-4 does not look very different across Large Public & Private Decision Types.
Renewal Ratio by Navigate Bundling
AccountSegment DecisionType GRP3-4 N ATL Dropped Ratio Perc_Renewed
International Need New LOA 0 1 1 0 Inf 100.0%
International Opt Out 0 1 1 0 Inf 100.0%
Large Public & Private Need New LOA 0 31 19 12 1.6 61.3%
Large Public & Private Need New LOA 1 1 1 0 Inf 100.0%
Large Public & Private Opt Out 0 10 10 0 Inf 100.0%
Large Public & Private Opt Out 1 6 6 0 Inf 100.0%
Regional Private Need New LOA 0 8 4 4 1.0 50.0%
Regional Public Need New LOA 0 9 2 7 0.3 22.2%
Regional Public Opt Out 0 4 3 1 3.0 75.0%
Regional Public Opt Out 1 2 2 0 Inf 100.0%
Selective Need New LOA 0 18 11 7 1.6 61.1%
Selective Opt Out 0 8 7 1 7.0 87.5%
Selective Opt Out 1 1 1 0 Inf 100.0%
Impact Interaction Volume Histogram
  1. High impact interaction consumption with a peak at 7!
  2. Density between 5 and 9.
  3. Minimal low utilizers.

##### Impact Interaction Volume Histogram Split by Decision

  1. Opt Outs spread out.
  2. Relatively uniform distribution of Opt Outs and NNLOA in the tail.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. NNLOA performance varies a lot due to a lot of buckets!
  2. NNLOA at 4 seems to be the threshold.
  3. Utilization <= 3 for NNLOA is deadly and 17 of the 68 partnerships fall in these buckets.
  4. Opt Out performance is invariant to Impact interaction volume.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 4 1 3 0.3 25.0%
Need New LOA 1 3 1 2 0.5 33.3%
Need New LOA 2 5 1 4 0.2 20.0%
Need New LOA 3 5 2 3 0.7 40.0%
Need New LOA 4 3 2 1 2.0 66.7%
Need New LOA 5 8 4 4 1.0 50.0%
Need New LOA 6 7 5 2 2.5 71.4%
Need New LOA 7 13 7 6 1.2 53.8%
Need New LOA 8 6 5 1 5.0 83.3%
Need New LOA 9 8 5 3 1.7 62.5%
Need New LOA 10 4 3 1 3.0 75.0%
Need New LOA 11 1 1 0 Inf 100.0%
Need New LOA 12 1 1 0 Inf 100.0%
Opt Out 0 1 0 1 0.0 0.0%
Opt Out 2 2 2 0 Inf 100.0%
Opt Out 3 3 3 0 Inf 100.0%
Opt Out 4 5 5 0 Inf 100.0%
Opt Out 5 3 3 0 Inf 100.0%
Opt Out 6 4 4 0 Inf 100.0%
Opt Out 7 4 4 0 Inf 100.0%
Opt Out 8 7 6 1 6.0 85.7%
Opt Out 9 2 2 0 Inf 100.0%
Opt Out 11 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume, (Binned)
  1. [NNLOA] 4+ still looks like the threshold with improving renewal at 7+
Renewal Ratio for Impact Interaction Volume
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 4 1 3 0.3 25.0%
Need New LOA 1 3 1 2 0.5 33.3%
Need New LOA 2-3 10 3 7 0.4 30.0%
Need New LOA 4-6 18 11 7 1.6 61.1%
Need New LOA 7+ 33 22 11 2.0 66.7%
Opt Out 0 1 0 1 0.0 0.0%
Opt Out 2-3 5 5 0 Inf 100.0%
Opt Out 4-6 12 12 0 Inf 100.0%
Opt Out 7+ 14 13 1 13.0 92.9%
Renewal by Previous Yr Impact Interaction Volume & Decision
  1. [NNLOA] 5+ in the previous year looks pretty good.
  2. [NNLOA] A few hiccups at higher interactions. Unsure what is driving this.
  3. [Opt Out] Invariance.
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 1 1 0 Inf 100.0%
Need New LOA 1 2 1 1 1.0 50.0%
Need New LOA 2 6 3 3 1.0 50.0%
Need New LOA 3 6 1 5 0.2 16.7%
Need New LOA 4 6 2 4 0.5 33.3%
Need New LOA 5 7 5 2 2.5 71.4%
Need New LOA 6 8 5 3 1.7 62.5%
Need New LOA 7 7 5 2 2.5 71.4%
Need New LOA 8 5 2 3 0.7 40.0%
Need New LOA 9 4 3 1 3.0 75.0%
Need New LOA 10 4 2 2 1.0 50.0%
Need New LOA 12 1 0 1 0.0 0.0%
Need New LOA 13 1 0 1 0.0 0.0%
Need New LOA 14 1 1 0 Inf 100.0%
Need New LOA NA 9 7 2 3.5 77.8%
Opt Out 0 1 1 0 Inf 100.0%
Opt Out 1 1 1 0 Inf 100.0%
Opt Out 2 1 1 0 Inf 100.0%
Opt Out 3 2 2 0 Inf 100.0%
Opt Out 4 2 2 0 Inf 100.0%
Opt Out 5 1 1 0 Inf 100.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 7 3 3 0 Inf 100.0%
Opt Out 8 1 1 0 Inf 100.0%
Opt Out 9 1 1 0 Inf 100.0%
Opt Out 10 1 1 0 Inf 100.0%
Opt Out 13 1 1 0 Inf 100.0%
Opt Out 14 1 1 0 Inf 100.0%
Opt Out NA 15 13 2 6.5 86.7%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract in the previous year.

  1. Previous Year interaction is not as good of a discriminator as 4-6 is close to 7+.
Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 1 1 0 Inf 100.0%
Need New LOA 1 2 1 1 1.0 50.0%
Need New LOA 2-3 12 4 8 0.5 33.3%
Need New LOA 4-6 21 12 9 1.3 57.1%
Need New LOA 7+ 23 13 10 1.3 56.5%
Need New LOA NA 9 7 2 3.5 77.8%
Opt Out 0 1 1 0 Inf 100.0%
Opt Out 1 1 1 0 Inf 100.0%
Opt Out 2-3 3 3 0 Inf 100.0%
Opt Out 4-6 4 4 0 Inf 100.0%
Opt Out 7+ 8 8 0 Inf 100.0%
Opt Out NA 15 13 2 6.5 86.7%
Renewals Previous Yr and Current Yr Interaction Volume

Only looking at NNLOAs in the below table.

  1. Oddly, 7+ to 7+ is not super performant.
  2. Instead, 4-6 to 7+, so increased utilization from a relatively high utilization in the previous year, it was performs well.
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_Bins II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
0 0 1 1 0 Inf 100.0%
1 7+ 2 1 1 1.0 50.0%
2-3 0 1 0 1 0.0 0.0%
2-3 1 1 0 1 0.0 0.0%
2-3 2-3 2 1 1 1.0 50.0%
2-3 4-6 4 1 3 0.3 25.0%
2-3 7+ 4 2 2 1.0 50.0%
4-6 0 2 0 2 0.0 0.0%
4-6 1 1 0 1 0.0 0.0%
4-6 2-3 3 0 3 0.0 0.0%
4-6 4-6 4 2 2 1.0 50.0%
4-6 7+ 11 10 1 10.0 90.9%
7+ 2-3 2 0 2 0.0 0.0%
7+ 4-6 7 6 1 6.0 85.7%
7+ 7+ 14 7 7 1.0 50.0%
NA 1 1 1 0 Inf 100.0%
NA 2-3 3 2 1 2.0 66.7%
NA 4-6 3 2 1 2.0 66.7%
NA 7+ 2 2 0 Inf 100.0%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. Significant. More impact interactions does translate to a higher percentage renewal rate.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -1.14 0.590 -1.94 0.053
Impact Interaction Volume 0.24 0.093 2.58 0.010
Penetration by Event Grouping
  1. A decent amount of variation between Opt Outs and NNLOAs:
  • Similar on: Events, Research Interviews, Experience
    • Opt Outs higher in Service penetration, SL and Expert Center Calls.
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 77.9% 57.4% 44.1% 36.8% 48.5% 7.4% 55.9%
Opt Out 78.1% 68.8% 62.5% 34.4% 34.4% 9.4% 62.5%
Renewal by Event Consumption
  1. [NNLOA] Get them to an event. Nearly 50% increase in renewal!
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 15 6 9 0.7 40.0%
Need New LOA 1 53 32 21 1.5 60.4%
Opt Out 0 7 6 1 6.0 85.7%
Opt Out 1 25 24 1 24.0 96.0%
Renewal by Service Consumption
  1. [NNLOA] Small lift above the NNLOA average (~4%) and decent lift (~8%) above not consuming this type of interaction.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 29 15 14 1.1 51.7%
Need New LOA 1 39 23 16 1.4 59.0%
Opt Out 0 10 8 2 4.0 80.0%
Opt Out 1 22 22 0 Inf 100.0%
Renewal by SL Consumption
  1. [NNLOA] Slight negative relationship between consumption and renewal. Odd.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 38 22 16 1.4 57.9%
Need New LOA 1 30 16 14 1.1 53.3%
Opt Out 0 12 10 2 5.0 83.3%
Opt Out 1 20 20 0 Inf 100.0%
Renewal by Research Interview Consumption
  1. [NNLOA] Relatively high lift above average NNLOA renewal rate - 21%
  2. [NNLOA] Difference between consuming and not consuming is roughly 32%
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 43 19 24 0.8 44.2%
Need New LOA 1 25 19 6 3.2 76.0%
Opt Out 0 21 20 1 20.0 95.2%
Opt Out 1 11 10 1 10.0 90.9%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Onsite and/or PLW positively associated with renewal! High N!
  2. [NNLOA] Close to a 6% bump above the average and 9% bump above not having one.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 35 18 17 1.1 51.4%
Need New LOA 1 33 20 13 1.5 60.6%
Opt Out 0 21 19 2 9.5 90.5%
Opt Out 1 11 11 0 Inf 100.0%
Renewal by Experience Consumption
  1. [NNLOA] Small N, but great results!
Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 63 33 30 1.1 52.4%
Need New LOA 1 5 5 0 Inf 100.0%
Opt Out 0 29 28 1 28.0 96.6%
Opt Out 1 3 2 1 2.0 66.7%
Renewal by Expert Call Consumption
  1. [NNLOA] Similar to PLW/Onsite - nice lift above NNLOA average and 10% lift above not consuming this category.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 30 15 15 1.0 50.0%
Need New LOA 1 38 23 15 1.5 60.5%
Opt Out 0 12 11 1 11.0 91.7%
Opt Out 1 20 19 1 19.0 95.0%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: Events, Research Interview, and Service.
  2. Nothing negatively correlated with renewal…although SL Led is close to 0.
  3. Similar to other Forums, Events and Expert Center are negatively correlated.
  4. Oddly, Research Interview and Events are correlated (positively)
  5. Events and SL Led less correlated here than in other forums.

##### Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only.

  1. Research Interviews and Onsites are close to significant. Thinking that consumption across categories might be highly correlated!
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) -0.97 0.544 -1.79 0.07
II_Events_Volume 0.21 0.209 1.01 0.31
II_ResearchInterview_Volume 0.76 0.481 1.58 0.11
II_PLW_Onsite_Volume 0.44 0.301 1.48 0.14
II_ExpertCall_Volume 0.14 0.254 0.53 0.59
II_Service_Volume 0.08 0.353 0.24 0.81