Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

CCEF Analysis

Key Takeaways

Renewal by Decision Type
  1. [NNLOA] Similar performance to 4 Year flagships.
  2. [Opt Out] Performance is less than what we’ve seen in other forums, albeit still solid. Opt Outs close to ~33%.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 41 23 18 1.3 56.1%
Opt Out 19 16 3 5.3 84.2%
Total 60 39 21 1.9 65.0%
Impact Interaction Volume Histogram
  1. Low volume of low utilizers - a good thing.
  2. Peak around 4, with a second peak near 7.
  3. Thick right tail suggests some heavier users.

##### Impact Interaction Volume Histogram Split by Decision

  1. Most of the opt outs and NNLOAs fall within the 4-7 range.
  2. Some opt outs in the tail, but the 7 peak looks like it is composed mostly of NNLOAs.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. NNLOA performance varies a lot due to small N.
  2. NNLOA performance around 5+ appears to be solid. The performance 4 and below is significantly worse.
  3. Opt Out performance is mostly invariant to Impact interaction volume.
  4. A couple of high utilization opt outs dropped.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 1 2 0.5 33.3%
Need New LOA 1 1 0 1 0.0 0.0%
Need New LOA 2 2 0 2 0.0 0.0%
Need New LOA 3 3 3 0 Inf 100.0%
Need New LOA 4 7 3 4 0.8 42.9%
Need New LOA 5 5 3 2 1.5 60.0%
Need New LOA 6 4 1 3 0.3 25.0%
Need New LOA 7 9 5 4 1.2 55.6%
Need New LOA 8 2 2 0 Inf 100.0%
Need New LOA 9 3 3 0 Inf 100.0%
Need New LOA 12 1 1 0 Inf 100.0%
Need New LOA 23 1 1 0 Inf 100.0%
Opt Out 2 1 1 0 Inf 100.0%
Opt Out 3 3 3 0 Inf 100.0%
Opt Out 4 4 4 0 Inf 100.0%
Opt Out 5 3 3 0 Inf 100.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 8 3 2 1 2.0 66.7%
Opt Out 9 1 1 0 Inf 100.0%
Opt Out 10 1 1 0 Inf 100.0%
Opt Out 11 1 0 1 0.0 0.0%
Opt Out 17 1 0 1 0.0 0.0%
Renewal by Impact Interaction Volume, (Binned)
  1. [NNLOA] Because the N at 4 is large and performance is week, we see a dip in the 4-6 bucket.
  2. [Opt Out] The few volume utilizers who dropped are causing the dip in Opt Out Renewals in 7+.
Renewal Ratio for Impact Interaction Volume
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 1 2 0.5 33.3%
Need New LOA 1 1 0 1 0.0 0.0%
Need New LOA 2-3 5 3 2 1.5 60.0%
Need New LOA 4-6 16 7 9 0.8 43.8%
Need New LOA 7+ 16 12 4 3.0 75.0%
Opt Out 2-3 4 4 0 Inf 100.0%
Opt Out 4-6 8 8 0 Inf 100.0%
Opt Out 7+ 7 4 3 1.3 57.1%
Renewal by Previous Yr Impact Interaction Volume & Decision
  1. [NNLOA] Decent amount of variation with no clear pattern.
  2. [Opt Out] Mostly Invariance. The high utilizers above were high utilizers in the prio years as well.
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 0 3 0.0 0.0%
Need New LOA 1 3 2 1 2.0 66.7%
Need New LOA 2 1 1 0 Inf 100.0%
Need New LOA 3 7 5 2 2.5 71.4%
Need New LOA 4 6 1 5 0.2 16.7%
Need New LOA 5 3 0 3 0.0 0.0%
Need New LOA 6 3 3 0 Inf 100.0%
Need New LOA 8 2 2 0 Inf 100.0%
Need New LOA 9 2 2 0 Inf 100.0%
Need New LOA 10 3 1 2 0.5 33.3%
Need New LOA 11 1 0 1 0.0 0.0%
Need New LOA 12 2 1 1 1.0 50.0%
Need New LOA 13 1 1 0 Inf 100.0%
Need New LOA 14 1 1 0 Inf 100.0%
Need New LOA 16 1 1 0 Inf 100.0%
Need New LOA 17 1 1 0 Inf 100.0%
Need New LOA NA 1 1 0 Inf 100.0%
Opt Out 1 2 2 0 Inf 100.0%
Opt Out 3 2 2 0 Inf 100.0%
Opt Out 5 1 1 0 Inf 100.0%
Opt Out 7 1 0 1 0.0 0.0%
Opt Out 9 3 3 0 Inf 100.0%
Opt Out 12 1 1 0 Inf 100.0%
Opt Out 19 1 0 1 0.0 0.0%
Opt Out NA 8 7 1 7.0 87.5%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract in the previous year.

  1. Binning does not help to distinguish healthy vs. not.
Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 0 3 0.0 0.0%
Need New LOA 1 3 2 1 2.0 66.7%
Need New LOA 2-3 8 6 2 3.0 75.0%
Need New LOA 4-6 12 4 8 0.5 33.3%
Need New LOA 7+ 14 10 4 2.5 71.4%
Need New LOA NA 1 1 0 Inf 100.0%
Opt Out 1 2 2 0 Inf 100.0%
Opt Out 2-3 2 2 0 Inf 100.0%
Opt Out 4-6 1 1 0 Inf 100.0%
Opt Out 7+ 6 4 2 2.0 66.7%
Opt Out NA 8 7 1 7.0 87.5%
Renewals Previous Yr and Current Yr Interaction Volume

Only looking at NNLOAs in the below table.

  1. Increasing utilization in the final year is a good indicator of renewal. 2-3 to 4-6 and 7+.
  2. If already a heavy user, 4+ and you drop in utilization, this appears to be a negative indicator.
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_Bins II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
0 0 1 0 1 0 0.0%
0 4-6 2 0 2 0 0.0%
1 4-6 3 2 1 2 66.7%
2-3 2-3 3 2 1 2 66.7%
2-3 4-6 1 1 0 Inf 100.0%
2-3 7+ 4 3 1 3 75.0%
4-6 0 1 0 1 0 0.0%
4-6 1 1 0 1 0 0.0%
4-6 2-3 1 0 1 0 0.0%
4-6 4-6 3 0 3 0 0.0%
4-6 7+ 6 4 2 2 66.7%
7+ 0 1 1 0 Inf 100.0%
7+ 2-3 1 1 0 Inf 100.0%
7+ 4-6 6 3 3 1 50.0%
7+ 7+ 6 5 1 5 83.3%
NA 4-6 1 1 0 Inf 100.0%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. Increasing impact interactions is significantly, positively associated with the probability of renewal.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -1.21 0.781 -1.55 0.120
Impact Interaction Volume 0.27 0.138 1.99 0.046
Penetration by Event Grouping
  1. A couple of differences across the categories, with the rest of the categories being roughly equivalent:
  • NNLOAs higher in PLW/Onsites.
    • Opt Outs higher in SL Led penetration - by a nearly 14% margin - and Expert Center Calls
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 56.1% 36.6% 75.6% 2.4% 46.3% 0.0% 70.7%
Opt Out 57.9% 36.8% 89.5% 0.0% 31.6% 0.0% 84.2%
Renewal by Event Consumption
  1. [NNLOA] 9% bump from attending an event (vs. the average), with a nearly 21% lift vs. not attending.
  2. [Opt Out] Negative relationship driven by heavy utilizers and no drops by low utilizers.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 18 8 10 0.8 44.4%
Need New LOA 1 23 15 8 1.9 65.2%
Opt Out 0 8 8 0 Inf 100.0%
Opt Out 1 11 8 3 2.7 72.7%
Renewal by Service Consumption
  1. [NNLOA] Renewal nearly 10% above the average and a 16% difference in consumption vs. not.
  2. [Opt Out] Negative relationship driven by heavy utilizers and no drops by low utilizers
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 26 13 13 1.0 50.0%
Need New LOA 1 15 10 5 2.0 66.7%
Opt Out 0 12 11 1 11.0 91.7%
Opt Out 1 7 5 2 2.5 71.4%
Renewal by SL Consumption
  1. [NNLOA] Negative relationship, but marginal. A couple % below the average if consumed and 4% below not having consumed this type.
  2. [Opt Out] Negative relationship driven by heavy utilizers and no drops by low utilizers, plus almost all partners have consumed this interaction.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 10 6 4 1.5 60.0%
Need New LOA 1 31 17 14 1.2 54.8%
Opt Out 0 2 2 0 Inf 100.0%
Opt Out 1 17 14 3 4.7 82.4%
Renewal by Research Interview Consumption
  1. [NNLOA] Too little data.
  2. [Opt Out] No data.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 40 22 18 1.2 55.0%
Need New LOA 1 1 1 0 Inf 100.0%
Opt Out 0 19 16 3 5.3 84.2%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Onsite and/or PLW positively associated with renewal! 7% above the average and 13% difference between not consuming.
  2. [Opt Out] N is low but this is the one interaction type where renewal rates look close for Opt Outs.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 22 11 11 1.0 50.0%
Need New LOA 1 19 12 7 1.7 63.2%
Opt Out 0 13 11 2 5.5 84.6%
Opt Out 1 6 5 1 5.0 83.3%
Renewal by Experience Consumption

No Data.

Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 41 23 18 1.3 56.1%
Opt Out 0 19 16 3 5.3 84.2%
Renewal by Expert Call Consumption
  1. [NNLOA] Largest lift here with nearly 9% above the average and 30% difference between consuming and not.
  2. [Opt Out] Negative relationship driven by heavy utilizers and no drops by low utilizers.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 12 4 8 0.5 33.3%
Need New LOA 1 29 19 10 1.9 65.5%
Opt Out 0 3 3 0 Inf 100.0%
Opt Out 1 16 13 3 4.3 81.2%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: Expert Calls and Onsites appear towards the top.
  2. Negative correlation with Renewal: sL Led Events.
  3. Negative Halo Effect: SL Led is negatively related to Service and Expert Calls, while positively related to Events and Onsites.

##### Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only.

  1. Onsites and Expert Calls are the closest, but nothing is ‘significant’.
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) -0.81 0.796 -1.01 0.31
II_Events_Volume 0.42 0.402 1.05 0.29
II_SLLed_Volume -0.39 0.326 -1.21 0.23
II_PLW_Onsite_Volume 1.03 0.711 1.45 0.15
II_ExpertCall_Volume 0.40 0.300 1.33 0.18
II_Service_Volume 0.50 0.715 0.70 0.49