Exec Summary

  1. The amount of impact interactions remain important, statistically.

  2. The threshold for impact interactions appears to be 4+.

  3. No particular impact interaction stands out, as there is a lot of correlation amongst several of them: PLW / Onsites had the largest impact.

Methodological

  1. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  2. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

  3. We include data from FY21 onwards.

  4. We look at unit decision rates, not $ Renewed.

ITS Analysis

THIS ENTIRE DATA SET LOOKS AT ITS DECISIONS BEYOND FY22 AND WITHOUT AND CRPs

Renewal Rate by Type, Excluding CRP and <=FY22
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 33 24 9 2.7 72.7%
Opt Out 28 26 2 13.0 92.9%
Total 204 154 50 3.1 75.5%
Renewal Rate by Account Segment
  1. Most of the decisions in LPP, which performs the best.
  2. A bit of a reversal for Selective, without CRPs these go down and Regional Public went up.
Renewal Ratio by Account Segment
AccountSegment N ATL Dropped Ratio Perc_Renewed
Canada 1 1 0 Inf 100.0%
Large Public & Private 37 32 5 6.4 86.5%
Regional Private 3 2 1 2.0 66.7%
Regional Public 10 8 2 4.0 80.0%
Selective 10 7 3 2.3 70.0%
Renewal by Decision Type & Segment
Renewal Ratio by Account Segment
AccountSegment DecisionType N ATL Dropped Ratio Perc_Renewed
Canada Need New LOA 1 1 0 Inf 100.0%
Large Public & Private Need New LOA 16 13 3 4.3 81.2%
Large Public & Private Opt Out 21 19 2 9.5 90.5%
Regional Private Need New LOA 3 2 1 2.0 66.7%
Regional Public Need New LOA 5 3 2 1.5 60.0%
Regional Public Opt Out 5 5 0 Inf 100.0%
Selective Need New LOA 8 5 3 1.7 62.5%
Selective Opt Out 2 2 0 Inf 100.0%
Impact Interaction Volume Histogram
  1. Lots of ITS partners w/o interactions!
  2. Solid, yet declining density between 1 and 7.
  3. Long tail, some stronger users.

##### Impact Interaction Volume Histogram Split by Decision

1.The two distributions are not equivalent - looks like more non-utilizers in teh opt out category.

Impact Interaction Volume Histogram Split by Year
  1. Roughly equal number of non-utilizers across the years.
  2. Some high utilizers in 24 and 25, none really in 23.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. NNLOA 0 still renews, albeit a little lower than the average.
  2. NNLOA at 4 seems to be the threshold.
  3. Weak evidence that 3+ for Opt Outs could be beneficial.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 2 1 2.0 66.7%
Need New LOA 1 5 2 3 0.7 40.0%
Need New LOA 2 2 1 1 1.0 50.0%
Need New LOA 3 5 3 2 1.5 60.0%
Need New LOA 4 6 5 1 5.0 83.3%
Need New LOA 5 3 2 1 2.0 66.7%
Need New LOA 6 3 3 0 Inf 100.0%
Need New LOA 7 3 3 0 Inf 100.0%
Need New LOA 9 2 2 0 Inf 100.0%
Need New LOA 10 1 1 0 Inf 100.0%
Opt Out 0 10 9 1 9.0 90.0%
Opt Out 1 5 5 0 Inf 100.0%
Opt Out 2 4 3 1 3.0 75.0%
Opt Out 3 2 2 0 Inf 100.0%
Opt Out 5 2 2 0 Inf 100.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 7 2 2 0 Inf 100.0%
Opt Out 9 1 1 0 Inf 100.0%
Opt Out 10 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume, Just Fy24 & FY25
  1. still 4+ for NNLOA.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 2 1 2.0 66.7%
Need New LOA 1 3 1 2 0.5 33.3%
Need New LOA 3 3 1 2 0.5 33.3%
Need New LOA 4 5 4 1 4.0 80.0%
Need New LOA 5 3 2 1 2.0 66.7%
Need New LOA 6 3 3 0 Inf 100.0%
Need New LOA 7 3 3 0 Inf 100.0%
Need New LOA 9 2 2 0 Inf 100.0%
Need New LOA 10 1 1 0 Inf 100.0%
Opt Out 0 6 6 0 Inf 100.0%
Opt Out 1 5 5 0 Inf 100.0%
Opt Out 2 4 3 1 3.0 75.0%
Opt Out 3 2 2 0 Inf 100.0%
Opt Out 5 1 1 0 Inf 100.0%
Opt Out 6 1 1 0 Inf 100.0%
Opt Out 7 2 2 0 Inf 100.0%
Opt Out 9 1 1 0 Inf 100.0%
Opt Out 10 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume, (Binned)
  1. [NNLOA] 4+ still looks like the threshold with improving renewal at 7+
Renewal Ratio for Impact Interaction Volume
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 2 1 2.0 66.7%
Need New LOA 1 5 2 3 0.7 40.0%
Need New LOA 2-3 7 4 3 1.3 57.1%
Need New LOA 4-6 12 10 2 5.0 83.3%
Need New LOA 7+ 6 6 0 Inf 100.0%
Opt Out 0 10 9 1 9.0 90.0%
Opt Out 1 5 5 0 Inf 100.0%
Opt Out 2-3 6 5 1 5.0 83.3%
Opt Out 4-6 3 3 0 Inf 100.0%
Opt Out 7+ 4 4 0 Inf 100.0%
Renewal by Impact Interaction Volume, (Binned), w AskEAB
  1. [NNLOA] 4+ still looks like the threshold with improving renewal at 7+
Renewal Ratio for Impact Interaction Volume w AskEAB
DecisionType II_Volume_Bins_AskEAB N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2 1 1 1.0 50.0%
Need New LOA 1 3 1 2 0.5 33.3%
Need New LOA 2-3 7 3 4 0.8 42.9%
Need New LOA 4-6 7 6 1 6.0 85.7%
Need New LOA 7+ 14 13 1 13.0 92.9%
Opt Out 0 9 9 0 Inf 100.0%
Opt Out 1 1 1 0 Inf 100.0%
Opt Out 2-3 5 3 2 1.5 60.0%
Opt Out 4-6 5 5 0 Inf 100.0%
Opt Out 7+ 8 8 0 Inf 100.0%
Renewal by Previous Yr Impact Interaction Volume & Decision
Renewal Ratio by Previous Year Impact Interaction
DecisionType PreviousYr_II_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2 2 0 Inf 100.0%
Need New LOA 1 2 1 1 1.0 50.0%
Need New LOA 2 4 3 1 3.0 75.0%
Need New LOA 3 5 3 2 1.5 60.0%
Need New LOA 4 3 1 2 0.5 33.3%
Need New LOA 5 3 2 1 2.0 66.7%
Need New LOA 6 5 5 0 Inf 100.0%
Need New LOA 10 1 0 1 0.0 0.0%
Need New LOA NA 8 7 1 7.0 87.5%
Opt Out 0 5 5 0 Inf 100.0%
Opt Out NA 23 21 2 10.5 91.3%
Renewal by Previous Yr Impact Interaction Volume Binned & Decision

Note: If II_PrevYr_Volume_Bins = NA, then there was not an active contract in the previous year.

Renewal Ratio by Previous Year Impact Interaction
DecisionType II_PrevYr_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2 2 0 Inf 100.0%
Need New LOA 1 2 1 1 1.0 50.0%
Need New LOA 2-3 9 6 3 2.0 66.7%
Need New LOA 4-6 11 8 3 2.7 72.7%
Need New LOA 7+ 1 0 1 0.0 0.0%
Need New LOA NA 8 7 1 7.0 87.5%
Opt Out 0 5 5 0 Inf 100.0%
Opt Out NA 23 21 2 10.5 91.3%
Renewals Previous Yr and Current Yr Interaction Volume
Renewal Ratio by Previous and Current Year Impact Interaction
II_PrevYr_Volume_Bins II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
0 0 1 1 0 Inf 100.0%
0 1 1 1 0 Inf 100.0%
1 0 1 0 1 0.0 0.0%
1 7+ 1 1 0 Inf 100.0%
2-3 2-3 4 1 3 0.3 25.0%
2-3 4-6 4 4 0 Inf 100.0%
2-3 7+ 1 1 0 Inf 100.0%
4-6 1 2 0 2 0.0 0.0%
4-6 2-3 2 2 0 Inf 100.0%
4-6 4-6 5 4 1 4.0 80.0%
4-6 7+ 2 2 0 Inf 100.0%
7+ 4-6 1 0 1 0.0 0.0%
NA 0 1 1 0 Inf 100.0%
NA 1 2 1 1 1.0 50.0%
NA 2-3 1 1 0 Inf 100.0%
NA 4-6 2 2 0 Inf 100.0%
NA 7+ 2 2 0 Inf 100.0%
Simple Regression for Impact Interaction Volume

Note: Looking at NNLOAs, only.

  1. Significant. More impact interactions does translate to a higher percentage renewal rate.
Simple Regression Model with Impact Interaction Volume
term estimate std.error statistic p.value
Intercept -0.47 0.723 -0.65 0.516
Impact Interaction Volume 0.44 0.210 2.10 0.036
Penetration by Event Grouping
  1. More heavily distributed towards events.
  2. The rest of the categories, which are consumed, are penetrated about the same amount (expert calls a little lower)
  3. Opt Outs and NNLOA are not distributed similarly across the interactions, because opt outs consume much less.
Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 72.7% 0.0% 39.4% 39.4% 42.4% 0.0% 24.2%
Opt Out 39.3% 0.0% 25.0% 14.3% 17.9% 0.0% 28.6%
Renewal by Event Consumption
  1. [NNLOA] No real bump from attending.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 9 7 2 3.5 77.8%
Need New LOA 1 24 17 7 2.4 70.8%
Opt Out 0 17 16 1 16.0 94.1%
Opt Out 1 11 10 1 10.0 90.9%
Renewal by Service Consumption
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 33 24 9 2.7 72.7%
Opt Out 0 28 26 2 13.0 92.9%
Renewal by SL Consumption
  1. [NNLOA] Small increase of 4% above the mean.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 20 14 6 2.3 70.0%
Need New LOA 1 13 10 3 3.3 76.9%
Opt Out 0 21 19 2 9.5 90.5%
Opt Out 1 7 7 0 Inf 100.0%
Renewal by Research Interview Consumption
  1. [NNLOA] Very large lift, nearly 20% if consumed vs. not. Just over a third have consumed this. Large lift above the mean, ~12%.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 20 13 7 1.9 65.0%
Need New LOA 1 13 11 2 5.5 84.6%
Opt Out 0 24 23 1 23.0 95.8%
Opt Out 1 4 3 1 3.0 75.0%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Another very large increase, 22%, if consumed. 13% lift above the mean.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 19 12 7 1.7 63.2%
Need New LOA 1 14 12 2 6.0 85.7%
Opt Out 0 23 21 2 10.5 91.3%
Opt Out 1 5 5 0 Inf 100.0%
Renewal by Experience Consumption
  1. [NNLOA] No Data.
Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 33 24 9 2.7 72.7%
Opt Out 0 28 26 2 13.0 92.9%
Renewal by Expert Call Consumption
  1. [NNLOA] Largest lift yet, with 15% above the mean and nearly 20% difference. Much smaller % have consumed this.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 25 17 8 2.1 68.0%
Need New LOA 1 8 7 1 7.0 87.5%
Opt Out 0 20 18 2 9.0 90.0%
Opt Out 1 8 8 0 Inf 100.0%
Renewal by AskEAB Consumption
  1. [NNLOA] Largest lift yet, with 15% above the mean and nearly 20% difference. Much smaller % have consumed this.
Renewal Ratio by AskEAB Consumption
DecisionType II_AskEAB_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 11 5 6 0.8 45.5%
Need New LOA 1 22 19 3 6.3 86.4%
Opt Out 0 12 11 1 11.0 91.7%
Opt Out 1 16 15 1 15.0 93.8%
Correlation betwen Event Groupings, and Renewal (for fun)
  1. Correlation with Renewal: Events and PLW/Onsites
  2. Nothing really negatively correlated with renewal rate.
  3. A few interactions are correlated with each other: Events, PLW/Onsite, SL Led.
  4. Expert call with a few negative correlations.

Multivariate Regression (still kind of simple)

Note: Looking at NNLOA, only.

  1. Too much correlation between everything, so nothing stands out.
  2. Onsite effect is likely the largest.
Regression Estimates for a Model Fitted on Event Grouping Volumes
term estimate std.error statistic p.value
(Intercept) -0.87 0.815 -1.07 0.29
II_Events_Volume 0.34 0.290 1.16 0.24
II_ResearchInterview_Volume 1.11 0.896 1.24 0.22
II_PLW_Onsite_Volume 1.97 1.363 1.45 0.15
II_ExpertCall_Volume 0.09 0.714 0.12 0.90
II_SLLed_Volume -0.82 1.254 -0.66 0.51
II_AskEAB_Volume 0.32 0.448 0.72 0.47