Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

ISEF Analysis

Key Takeaways

Renewal by Decision Type
  1. [NNLOA] Higher than what we’ve seen in 4 Year flagships.
  2. [Opt Out] On par with NNLOA. Very different from 4 Year or DLF performance.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 21 14 7 2 66.7%
Opt Out 12 8 4 2 66.7%
Total 33 22 11 2 66.7%
Impact Interaction Volume Histogram
  1. Small volume of low utilizers at 3 or below.
  2. Two peaks: one at 4 and another around 9. Fits in with two sets of users - low and heavy.
  3. High surge of users at the limit of heavy users.

##### Impact Interaction Volume Histogram Split by Decision

  1. More of the Opt Outs are towards the lower end, due to higher presence in 4 bucket.
  2. High utilization at the extreme are all NNLOA.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. [NNLOA] With the N so small across many bins, only minor evidence of positive impact in utilization, at 8+.
  • Makes you wonder whether there is such as thing as ‘over-utilization’.
  1. A couple of high utilization opt outs dropped, but otherwise all renewals with heavier utilization.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 1 1 0 1 0 0.0%
Need New LOA 2 1 0 1 0 0.0%
Need New LOA 3 1 1 0 Inf 100.0%
Need New LOA 4 2 1 1 1 50.0%
Need New LOA 5 2 1 1 1 50.0%
Need New LOA 6 2 1 1 1 50.0%
Need New LOA 7 2 1 1 1 50.0%
Need New LOA 8 2 2 0 Inf 100.0%
Need New LOA 9 4 3 1 3 75.0%
Need New LOA 12 4 4 0 Inf 100.0%
Opt Out 3 1 1 0 Inf 100.0%
Opt Out 4 3 2 1 2 66.7%
Opt Out 5 1 1 0 Inf 100.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 7 1 1 0 Inf 100.0%
Opt Out 8 1 1 0 Inf 100.0%
Opt Out 9 1 1 0 Inf 100.0%
Opt Out 10 2 0 2 0 0.0%
Opt Out 11 1 0 1 0 0.0%
Renewal by Impact Interaction Volume, (Binned)
  1. [NNLOA] 7+ seems to be a threshold beyond which the % renewed remains relatively stable.
  2. [Opt Out] Higher utilization looks a little bit worse, but a few drops is the cause and therefore no reason to react.
Renewal Ratio for Impact Interaction Volume
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 1 1 0 1 0 0.0%
Need New LOA 2-3 2 1 1 1 50.0%
Need New LOA 4-6 6 3 3 1 50.0%
Need New LOA 7+ 12 10 2 5 83.3%
Opt Out 2-3 1 1 0 Inf 100.0%
Opt Out 4-6 5 4 1 4 80.0%
Opt Out 7+ 6 3 3 1 50.0%
Renewal by Previous Yr Impact Interaction Volume & Decision
  1. [NNLOA] Higher utilization, 4+, does seem to be associate with renewal.
  2. [Opt Out] Most are in the NA bucket, so no pattern.
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 2 3 1 2 0.5 33.3%
Need New LOA 3 2 0 2 0.0 0.0%
Need New LOA 4 6 5 1 5.0 83.3%
Need New LOA 5 4 3 1 3.0 75.0%
Need New LOA 6 2 1 1 1.0 50.0%
Need New LOA 7 1 1 0 Inf 100.0%
Need New LOA 8 2 2 0 Inf 100.0%
Need New LOA 9 1 1 0 Inf 100.0%
Opt Out 6 2 1 1 1.0 50.0%
Opt Out 7 1 0 1 0.0 0.0%
Opt Out NA 9 7 2 3.5 77.8%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract in the previous year.

  1. For NNLOA, the 4+ range in the previous year looks solid.
Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 2-3 5 1 4 0.2 20.0%
Need New LOA 4-6 12 9 3 3.0 75.0%
Need New LOA 7+ 4 4 0 Inf 100.0%
Opt Out 4-6 2 1 1 1.0 50.0%
Opt Out 7+ 1 0 1 0.0 0.0%
Opt Out NA 9 7 2 3.5 77.8%
Renewals Previous Yr and Current Yr Interaction Volume

Only looking at NNLOAs in the below table.

  1. Maintaining utilization might be the key.
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_Bins II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
2-3 1 1 0 1 0 0.0%
2-3 2-3 1 0 1 0 0.0%
2-3 4-6 1 0 1 0 0.0%
2-3 7+ 2 1 1 1 50.0%
4-6 2-3 1 1 0 Inf 100.0%
4-6 4-6 4 2 2 1 50.0%
4-6 7+ 7 6 1 6 85.7%
7+ 4-6 1 1 0 Inf 100.0%
7+ 7+ 3 3 0 Inf 100.0%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. Increasing impact interactions IS significantly associated with the probability of renewal.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -1.97 1.344 -1.47 0.142
Impact Interaction Volume 0.41 0.203 2.01 0.044
Penetration by Event Grouping
  1. NNLOAs get more SL Led, Onsites, and Expert Calls.
  2. Parity on Research Interviews.
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 90.5% 4.8% 23.8% 76.2% 52.4% 0.0% 76.2%
Opt Out 100.0% 8.3% 8.3% 75.0% 25.0% 0.0% 33.3%
Renewal by Event Consumption
  1. [NNLOA] Marginal lift above average, 18% dip if not attended.
  2. [Opt Out] All attended, no pattern.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2 1 1 1.0 50.0%
Need New LOA 1 19 13 6 2.2 68.4%
Opt Out 1 12 8 4 2.0 66.7%
Renewal by Service Consumption

N too small. Looks impactful.

Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 20 13 7 1.9 65.0%
Need New LOA 1 1 1 0 Inf 100.0%
Opt Out 0 11 7 4 1.8 63.6%
Opt Out 1 1 1 0 Inf 100.0%
Renewal by SL Consumption
  1. [NNLOA] Negative relationship, but medium effect: 6% below the average.Marginal benefit if not consumed - ignore.
  2. [Opt Out] Too little data.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 16 11 5 2.2 68.8%
Need New LOA 1 5 3 2 1.5 60.0%
Opt Out 0 11 8 3 2.7 72.7%
Opt Out 1 1 0 1 0.0 0.0%
Renewal by Research Interview Consumption
  1. [NNLOA] Small positive relationship with renewal: 1% lift above averate and 7% drop if not consumed.
  2. [Opt Out] Huge positive relationship. 10% bump and giant decrease (small n) if not consumed.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 5 3 2 1.5 60.0%
Need New LOA 1 16 11 5 2.2 68.8%
Opt Out 0 3 1 2 0.5 33.3%
Opt Out 1 9 7 2 3.5 77.8%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Small negative relationship: 4% drop from average and 3% lift above average if not consumed.
  2. [Opt Out] Parity if consumed or not.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 10 7 3 2.3 70.0%
Need New LOA 1 11 7 4 1.8 63.6%
Opt Out 0 9 6 3 2.0 66.7%
Opt Out 1 3 2 1 2.0 66.7%
Renewal by Experience Consumption

No Data.

Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 21 14 7 2 66.7%
Opt Out 0 12 8 4 2 66.7%
Renewal by Expert Call Consumption
  1. [NNLOA] 8% lift above the average and 35% difference between consumption vs. not.
  2. [Opt Out] Negative relationship driven by a small N. Nearly a 41% drop if consumed.%.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 5 2 3 0.7 40.0%
Need New LOA 1 16 12 4 3.0 75.0%
Opt Out 0 8 7 1 7.0 87.5%
Opt Out 1 4 1 3 0.3 25.0%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: Research Interviews and Events appear towards the top.
  2. Negative correlation with Renewal: SL Led and Onsites
  3. Mixed messages. All the correlations are positive for a couple and negative for 3.

##### Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only.

  1. [NNLOA Only] Events are borderline.
  2. [With Opt Outs] No significant results.
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) -3.35 2.448 -1.37 0.17
II_Events_Volume 0.60 0.375 1.59 0.11
II_SLLed_Volume -1.81 1.805 -1.00 0.32
II_PLW_Onsite_Volume 0.60 1.010 0.59 0.55
II_ExpertCall_Volume 1.32 1.218 1.09 0.28
II_ResearchInterview_Volume 0.84 0.899 0.94 0.35