Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

DLF Analysis

Key Takeaways

Renewal by Decision Type
  1. [NNLOA] Higher than what we’ve seen in 4 Year flagships.
  2. [Opt Out] Performance is on par with what we’ve seen in some of the better 4 Year forums. It is also a much higher proportion of total decisions, ~60%.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 39 25 14 1.8 64.1%
Opt Out 59 54 5 10.8 91.5%
Total 98 79 19 4.2 80.6%
Impact Interaction Volume Histogram
  1. Medium volume of low utilizers at 2 or below.
  2. Two peaks: one at 2 and another around 6. Fits in with two sets of users - low and heavy.
  3. Thick right tail suggests some heavier users.

##### Impact Interaction Volume Histogram Split by Decision

  1. More of the Opt Outs are towards the heavier usage tail.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. NNLOA performance definitely suggests higher rates as utilization goes up, with a hiccup at 5.
  • Makes you wonder whether there is such as thing as ‘over-utilization’.
  1. A couple of high utilization opt outs dropped, but otherwise all renewals with heavier utilization.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 4 1 3 0.3 25.0%
Need New LOA 1 2 0 2 0.0 0.0%
Need New LOA 2 4 3 1 3.0 75.0%
Need New LOA 3 7 5 2 2.5 71.4%
Need New LOA 4 3 2 1 2.0 66.7%
Need New LOA 5 3 1 2 0.5 33.3%
Need New LOA 6 6 6 0 Inf 100.0%
Need New LOA 7 2 2 0 Inf 100.0%
Need New LOA 8 4 3 1 3.0 75.0%
Need New LOA 9 2 1 1 1.0 50.0%
Need New LOA 12 1 0 1 0.0 0.0%
Need New LOA 13 1 1 0 Inf 100.0%
Opt Out 0 3 3 0 Inf 100.0%
Opt Out 1 8 7 1 7.0 87.5%
Opt Out 2 8 8 0 Inf 100.0%
Opt Out 3 3 3 0 Inf 100.0%
Opt Out 4 6 5 1 5.0 83.3%
Opt Out 5 6 5 1 5.0 83.3%
Opt Out 6 6 6 0 Inf 100.0%
Opt Out 7 3 3 0 Inf 100.0%
Opt Out 8 3 3 0 Inf 100.0%
Opt Out 9 6 5 1 5.0 83.3%
Opt Out 10 1 1 0 Inf 100.0%
Opt Out 11 3 2 1 2.0 66.7%
Opt Out 12 1 1 0 Inf 100.0%
Opt Out 13 1 1 0 Inf 100.0%
Opt Out 14 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume, (Binned)
  1. [NNLOA] 2+ seems to be a threshold beyond which the % renewed remains relatively stable.
  2. [Opt Out] Higher utilization looks a little bit worse, but a few drops is the cause and therefore no reason to react.
Renewal Ratio for Impact Interaction Volume
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 4 1 3 0.3 25.0%
Need New LOA 1 2 0 2 0.0 0.0%
Need New LOA 2-3 11 8 3 2.7 72.7%
Need New LOA 4-6 12 9 3 3.0 75.0%
Need New LOA 7+ 10 7 3 2.3 70.0%
Opt Out 0 3 3 0 Inf 100.0%
Opt Out 1 8 7 1 7.0 87.5%
Opt Out 2-3 11 11 0 Inf 100.0%
Opt Out 4-6 18 16 2 8.0 88.9%
Opt Out 7+ 19 17 2 8.5 89.5%
Renewal by Previous Yr Impact Interaction Volume & Decision
  1. [NNLOA] Higher utilization, 7-9, does not seem to associate strongly with renewal.
  2. [Opt Out] Mostly Invariant.
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 1 4 1 3 0.3 25.0%
Need New LOA 2 4 3 1 3.0 75.0%
Need New LOA 3 6 4 2 2.0 66.7%
Need New LOA 4 2 2 0 Inf 100.0%
Need New LOA 5 6 4 2 2.0 66.7%
Need New LOA 6 6 4 2 2.0 66.7%
Need New LOA 7 2 1 1 1.0 50.0%
Need New LOA 8 2 1 1 1.0 50.0%
Need New LOA 9 1 0 1 0.0 0.0%
Need New LOA 10 1 1 0 Inf 100.0%
Need New LOA NA 5 4 1 4.0 80.0%
Opt Out 1 2 2 0 Inf 100.0%
Opt Out 2 2 2 0 Inf 100.0%
Opt Out 3 1 1 0 Inf 100.0%
Opt Out 4 1 1 0 Inf 100.0%
Opt Out 5 2 2 0 Inf 100.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 7 5 5 0 Inf 100.0%
Opt Out 8 3 2 1 2.0 66.7%
Opt Out 9 3 3 0 Inf 100.0%
Opt Out 10 1 0 1 0.0 0.0%
Opt Out 11 3 3 0 Inf 100.0%
Opt Out 12 1 1 0 Inf 100.0%
Opt Out 18 1 1 0 Inf 100.0%
Opt Out 19 1 1 0 Inf 100.0%
Opt Out NA 32 29 3 9.7 90.6%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract in the previous year.

  1. For NNLOA, the 2-6 range in the previous year looks solid, with 7+ utilization looking a bit weaker.
Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 1 4 1 3 0.3 25.0%
Need New LOA 2-3 10 7 3 2.3 70.0%
Need New LOA 4-6 14 10 4 2.5 71.4%
Need New LOA 7+ 6 3 3 1.0 50.0%
Need New LOA NA 5 4 1 4.0 80.0%
Opt Out 1 2 2 0 Inf 100.0%
Opt Out 2-3 3 3 0 Inf 100.0%
Opt Out 4-6 4 4 0 Inf 100.0%
Opt Out 7+ 18 16 2 8.0 88.9%
Opt Out NA 32 29 3 9.7 90.6%
Renewals Previous Yr and Current Yr Interaction Volume

Only looking at NNLOAs in the below table.

  1. Increasing utilization in the final year is a good indicator of renewal, if you were previously utilizing >2.
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_Bins II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
1 0 1 0 1 0.0 0.0%
1 2-3 1 0 1 0.0 0.0%
1 4-6 1 0 1 0.0 0.0%
1 7+ 1 1 0 Inf 100.0%
2-3 0 1 0 1 0.0 0.0%
2-3 1 1 0 1 0.0 0.0%
2-3 2-3 1 1 0 Inf 100.0%
2-3 4-6 6 5 1 5.0 83.3%
2-3 7+ 1 1 0 Inf 100.0%
4-6 0 2 1 1 1.0 50.0%
4-6 2-3 3 2 1 2.0 66.7%
4-6 4-6 2 2 0 Inf 100.0%
4-6 7+ 7 5 2 2.5 71.4%
7+ 1 1 0 1 0.0 0.0%
7+ 2-3 4 3 1 3.0 75.0%
7+ 7+ 1 0 1 0.0 0.0%
NA 2-3 2 2 0 Inf 100.0%
NA 4-6 3 2 1 2.0 66.7%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. Increasing impact interactions is not significantly associated with the probability of renewal.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -0.11 0.602 -0.18 0.861
Impact Interaction Volume 0.15 0.117 1.31 0.190
Penetration by Event Grouping
  1. Opt Out utilization is higher on everything except Expert Calls.:
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 53.8% 0.0% 30.8% 30.8% 17.9% 0.0% 59.0%
Opt Out 71.2% 0.0% 39.0% 39.0% 23.7% 0.0% 47.5%
Renewal by Event Consumption
  1. [NNLOA] 7% bump from attending an event (vs. the average), with a nearly 16% lift vs. not attending.
  2. [Opt Out] Small positive relationship.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 18 10 8 1.2 55.6%
Need New LOA 1 21 15 6 2.5 71.4%
Opt Out 0 17 15 2 7.5 88.2%
Opt Out 1 42 39 3 13.0 92.9%
Renewal by Service Consumption

No data here.

Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 39 25 14 1.8 64.1%
Opt Out 0 59 54 5 10.8 91.5%
Renewal by SL Consumption
  1. [NNLOA] Negative relationship, but medium effect: 6% below the average if consumed and 8% below not having consumed this type.
  2. [Opt Out] Positive relationship with a 4% bump above the average.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 27 18 9 2.0 66.7%
Need New LOA 1 12 7 5 1.4 58.3%
Opt Out 0 36 32 4 8.0 88.9%
Opt Out 1 23 22 1 22.0 95.7%
Renewal by Research Interview Consumption
  1. [NNLOA] Negative relationship with renewal (rare for research interviews). If consumed, nearly 14% drop from the average and a 20% increase if not consumed.
  2. [Opt Out] Also negative relationship. About equidistant to average - opt out or NNLOA.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 27 19 8 2.4 70.4%
Need New LOA 1 12 6 6 1.0 50.0%
Opt Out 0 36 34 2 17.0 94.4%
Opt Out 1 23 20 3 6.7 87.0%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Medium negative relationship: 7% drop from average and 2% lift above average if not consumed.
  2. [Opt Out] Sizable negative relationship: 13% below the average if consumed and 17% difference if not consumed.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 32 21 11 1.9 65.6%
Need New LOA 1 7 4 3 1.3 57.1%
Opt Out 0 45 43 2 21.5 95.6%
Opt Out 1 14 11 3 3.7 78.6%
Renewal by Experience Consumption

No Data.

Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 39 25 14 1.8 64.1%
Opt Out 0 59 54 5 10.8 91.5%
Renewal by Expert Call Consumption
  1. [NNLOA] 5 above the average and 13% difference between consuming and not.
  2. [Opt Out] Positive relationship driven with a very small lift - 1%.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 16 9 7 1.3 56.2%
Need New LOA 1 23 16 7 2.3 69.6%
Opt Out 0 31 28 3 9.3 90.3%
Opt Out 1 28 26 2 13.0 92.9%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: Expert Calls and Events appear towards the top.
  2. Negative correlation with Renewal: Research Interviews and Onsites.
  3. Positive Halo Effect: Events are key to consumption of other impact interactions.

##### Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only.

  1. [NNLOA Only] Nothing is ‘significant’.
  2. [With Opt Outs] Events get close, but not quite there.
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) 1.09 0.434 2.50 0.01
II_Events_Volume 0.34 0.228 1.50 0.13
II_SLLed_Volume 0.10 0.568 0.18 0.86
II_PLW_Onsite_Volume -0.72 0.576 -1.24 0.21
II_ExpertCall_Volume 0.25 0.295 0.84 0.40
II_ResearchInterview_Volume -0.22 0.271 -0.80 0.42