EMF Analysis

The purpose of this analysis is to understand the variable effect of impact interactions on the likelihood of a partner to renew their partnership. Simply, which service levers matter more for partnership renewal.

Executive Summary

Methodological Considerations

  1. The data set consists of decisions from FY19 through March of FY22.

  2. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  3. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

  4. Lastly, we are looking at unit renewal rates instead of $ renewal rates. This allows us to treat impact interactions equitably across partnerships.

Renewal by Decision Type

The below table should not come as a surprise. Opt Outs tend to perform very well while NNLOA can be quite mixed.

  1. Remember the Numbers: 41% for NNLOA, 91% for Opt Outs.
  2. NNLOA performs lower than what we’ve seen for AAF & BAF.
  3. Opt Out performance is in range of what we would expect.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 112 46 66 0.7 41.1%
Opt Out 43 38 5 7.6 88.4%
Total 155 84 71 1.2 54.2%

Renewals by Decision Type and Year

Since we are combining multiple years of data, we will frequently take a look at whether there are differences across the years.

  1. NNLOA - Increase from from FY19, which was very low.
  2. Opt Outs - All of the drops are in a single year FY21.
Renewal Ratio by Decision Type
DecisionType FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 2019 15 3 12 0.2 20.0%
Need New LOA 2020 43 16 27 0.6 37.2%
Need New LOA 2021 44 22 22 1.0 50.0%
Need New LOA 2022 10 5 5 1.0 50.0%
Opt Out 2019 12 12 0 Inf 100.0%
Opt Out 2020 14 14 0 Inf 100.0%
Opt Out 2021 12 7 5 1.4 58.3%
Opt Out 2022 5 5 0 Inf 100.0%
Total NA 155 84 71 1.2 54.2%
Renewal Rate by Account Segment
  1. About ~50% of decision are in Large P&P, which, with Selectives, have the highest performance.
  2. Regionals perform remarkably similar. Given that there are Opt Outs included in the below, this performance is really low.
Renewal Ratio by Account Segment
AccountSegment N ATL Dropped Ratio Perc_Renewed
Canada 5 2 3 0.7 40.0%
Large Public & Private 76 45 31 1.5 59.2%
Regional Private 15 7 8 0.9 46.7%
Regional Public 34 15 19 0.8 44.1%
Selective 25 15 10 1.5 60.0%
Renewal Rate by Account Segment & Fiscal Year
  1. Large P&P has seen increased renewal rates across the years.
  2. The other Segments (excluding Canada) have been performing worse across the year - although Regional Publics have been steady since FY20
Renewal Ratio by Account Segment
AccountSegment FiscalYear N ATL Dropped Ratio Perc_Renewed
Canada 2019 2 1 1 1.0 50.0%
Canada 2020 2 0 2 0.0 0.0%
Canada 2022 1 1 0 Inf 100.0%
Large Public & Private 2019 12 5 7 0.7 41.7%
Large Public & Private 2020 27 16 11 1.5 59.3%
Large Public & Private 2021 30 19 11 1.7 63.3%
Large Public & Private 2022 7 5 2 2.5 71.4%
Regional Private 2019 3 2 1 2.0 66.7%
Regional Private 2020 8 4 4 1.0 50.0%
Regional Private 2021 4 1 3 0.3 25.0%
Regional Public 2019 5 3 2 1.5 60.0%
Regional Public 2020 14 6 8 0.8 42.9%
Regional Public 2021 10 4 6 0.7 40.0%
Regional Public 2022 5 2 3 0.7 40.0%
Selective 2019 5 4 1 4.0 80.0%
Selective 2020 6 4 2 2.0 66.7%
Selective 2021 12 5 7 0.7 41.7%
Selective 2022 2 2 0 Inf 100.0%
Renewal by Decision Type & Segment
  1. NNLOA on Regional Publics is driving the average down, since there is a large enough N and the performance is an outlier.
  2. Although, it is not as though Selectives and Regional Private NNLOA rates are much higher.
Renewal Ratio by Account Segment
AccountSegment DecisionType N ATL Dropped Ratio Perc_Renewed
Canada Need New LOA 4 1 3 0.3 25.0%
Canada Opt Out 1 1 0 Inf 100.0%
Large Public & Private Need New LOA 55 27 28 1.0 49.1%
Large Public & Private Opt Out 21 18 3 6.0 85.7%
Regional Private Need New LOA 14 6 8 0.8 42.9%
Regional Private Opt Out 1 1 0 Inf 100.0%
Regional Public Need New LOA 25 7 18 0.4 28.0%
Regional Public Opt Out 9 8 1 8.0 88.9%
Selective Need New LOA 14 5 9 0.6 35.7%
Selective Opt Out 11 10 1 10.0 90.9%
CRP Inclusion

No CRPs! Took them out.

Renewal Ratio by CRP Inclusion
GRP3-4 N ATL Dropped Ratio Perc_Renewed
0 155 84 71 1.2 54.2%
Impact Interaction Volume Histogram
  1. Central tendency increased to about 4, about where we see most of the other forums.
  2. The tail is between 6-10 and there are enough data points there to see if these units renew better.
  3. A lot of the mass is between 2-5. Makes me wonder if performance would differ based on WHAT was consumed.

Impact Interaction Volume Histogram Split by Decision
  1. 0-2 utilization looks to perform pretty low.
  2. That 3-5 region contains a lot of drops as well.
  3. Even the 6+ region has some clusters of drops.

Impact Interaction Volume Histogram Split by Pool Date FY
  1. FY22 units have lower utilization than FY20/21, which you see comprises much of the tail.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. [NNLOA] some of the worst performance is in the lower end of the utilization spectrum. We also have some not great performance at
  2. [Opt Outs] Utilization doesn’t seem to matter.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 8 3 5 0.6 37.5%
Need New LOA 1 13 2 11 0.2 15.4%
Need New LOA 2 21 7 14 0.5 33.3%
Need New LOA 3 22 11 11 1.0 50.0%
Need New LOA 4 11 7 4 1.8 63.6%
Need New LOA 5 19 8 11 0.7 42.1%
Need New LOA 6 3 0 3 0.0 0.0%
Need New LOA 7 4 3 1 3.0 75.0%
Need New LOA 8 5 1 4 0.2 20.0%
Need New LOA 9 2 2 0 Inf 100.0%
Need New LOA 10 4 2 2 1.0 50.0%
Opt Out 0 1 1 0 Inf 100.0%
Opt Out 1 3 2 1 2.0 66.7%
Opt Out 2 3 3 0 Inf 100.0%
Opt Out 3 4 3 1 3.0 75.0%
Opt Out 4 10 9 1 9.0 90.0%
Opt Out 5 5 4 1 4.0 80.0%
Opt Out 6 5 4 1 4.0 80.0%
Opt Out 7 6 6 0 Inf 100.0%
Opt Out 8 2 2 0 Inf 100.0%
Opt Out 9 3 3 0 Inf 100.0%
Opt Out 10 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume and Decision Type, (Binned)

Adjusted the binning to better capture the high volume present in BAF.

  1. [NNLOA] Low bar is 2+.
  2. [NNLOA] 7+ does offer some benefit, but 4-6 is not crazy lift above 2-3.
Renewal Ratio by Impact Interaction Volume (Binned)
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 8 3 5 0.6 37.5%
Need New LOA 1 13 2 11 0.2 15.4%
Need New LOA 2-3 43 18 25 0.7 41.9%
Need New LOA 4-6 33 15 18 0.8 45.5%
Need New LOA 7+ 15 8 7 1.1 53.3%
Opt Out 0 1 1 0 Inf 100.0%
Opt Out 1 3 2 1 2.0 66.7%
Opt Out 2-3 7 6 1 6.0 85.7%
Opt Out 4-6 20 17 3 5.7 85.0%
Opt Out 7+ 12 12 0 Inf 100.0%

Renewal by Impact Interaction Volume and Account Segment, (Binned), Only NNLOA

  1. Large Public & Private performs differently from the other segments. Here, we see that increased utilization is clearly tied to renewal.
  2. For the other Segments, the bar might be lower, knowing that increasing consumption doesn’t necessarily lead to higher renewal rates.
Renewal Ratio by Impact Interaction Volume (Binned)
AccountSegment II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Canada 0 3 1 2 0.5 33.3%
Canada 1 1 0 1 0.0 0.0%
Large Public & Private 0 3 1 2 0.5 33.3%
Large Public & Private 1 5 1 4 0.2 20.0%
Large Public & Private 2-3 26 11 15 0.7 42.3%
Large Public & Private 4-6 14 9 5 1.8 64.3%
Large Public & Private 7+ 7 5 2 2.5 71.4%
Regional Private 1 2 0 2 0.0 0.0%
Regional Private 2-3 5 4 1 4.0 80.0%
Regional Private 4-6 6 1 5 0.2 16.7%
Regional Private 7+ 1 1 0 Inf 100.0%
Regional Public 0 2 1 1 1.0 50.0%
Regional Public 1 3 0 3 0.0 0.0%
Regional Public 2-3 7 2 5 0.4 28.6%
Regional Public 4-6 8 2 6 0.3 25.0%
Regional Public 7+ 5 2 3 0.7 40.0%
Selective 1 2 1 1 1.0 50.0%
Selective 2-3 5 1 4 0.2 20.0%
Selective 4-6 5 3 2 1.5 60.0%
Selective 7+ 2 0 2 0.0 0.0%

Simple Regression for Impact Interaction Volume

  1. The Estimate for Impact Interaction Volume is Positive -> increased consumption means increased likelihood to renew.
  2. This is statistically significant.
  3. When you build a more complicated model, impact interaction is no longer statistically significant. This means Large P&P are really driving the results here.

All Decisions

  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 0.57 0.30 – 1.04 0.073
II Credited Volume 1.21 1.06 – 1.39 0.007
Observations 155
R2 Tjur 0.049

All NNLOA

  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 0.47 0.23 – 0.91 0.029
II Credited Volume 1.12 0.96 – 1.31 0.161
Observations 112
R2 Tjur 0.018

All NNLOA - Except Large P&P

  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 0.47 0.17 – 1.17 0.114
II Credited Volume 1.02 0.82 – 1.25 0.861
Observations 57
R2 Tjur 0.001

NNLOA - Large P&P

  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 0.35 0.11 – 1.01 0.064
II Credited Volume 1.34 1.03 – 1.83 0.044
Observations 55
R2 Tjur 0.084
Penetration by Event Grouping
  1. Expert Calls lead on penetration. Then, Events, Service, SL Led, and Research Interview pen is all roughly similar.
  2. Events and SL Led are more heavily penetrated for Opt Outs.

Reminder: 1. Experience = VIP Days & Collaboratives

  1. Service = Benchmarking, Audits, Secret Shopper, Customer Research Projects, Market Insights, and Survey Walkthroughs

  2. Expert Call = Expert Center, Benchmarking Walkthrough, Tool’s Walkthrough

  3. SL Led = Research SL Impact, Evergreen Action, Exec Update, National Meeting Pre-wire

  4. Research Interview = Research Initial & Scoping

  5. Events = National Meeting, COVID Working Group

  6. PLW/Onsite = Onsite, PLW

Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 39.3% 40.2% 45.5% 35.7% 17.0% 3.6% 54.5%
Opt Out 62.8% 37.2% 72.1% 46.5% 18.6% 9.3% 67.4%
Renewal by Event Consumption
  1. [NNLOA] 9% lift for attending events. Decent utilization.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 68 24 44 0.5 35.3%
Need New LOA 1 44 22 22 1.0 50.0%
Opt Out 0 16 15 1 15.0 93.8%
Opt Out 1 27 23 4 5.8 85.2%
Renewal by Event Consumption Volume
  1. Seeing benefit at 1, with 2-3 showing marked improvement in renewal rate.

Renewal by Event, OT

  1. NNLOA: FY21 really driving the results here. FY22 is also solid, just a very low N.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2019 12 2 10 0.2 16.7%
Need New LOA 0 2020 32 13 19 0.7 40.6%
Need New LOA 0 2021 16 6 10 0.6 37.5%
Need New LOA 0 2022 8 3 5 0.6 37.5%
Need New LOA 1 2019 3 1 2 0.5 33.3%
Need New LOA 1 2020 11 3 8 0.4 27.3%
Need New LOA 1 2021 28 16 12 1.3 57.1%
Need New LOA 1 2022 2 2 0 Inf 100.0%
Opt Out 0 2019 2 2 0 Inf 100.0%
Opt Out 0 2020 8 8 0 Inf 100.0%
Opt Out 0 2021 3 2 1 2.0 66.7%
Opt Out 0 2022 3 3 0 Inf 100.0%
Opt Out 1 2019 10 10 0 Inf 100.0%
Opt Out 1 2020 6 6 0 Inf 100.0%
Opt Out 1 2021 9 5 4 1.2 55.6%
Opt Out 1 2022 2 2 0 Inf 100.0%
Renewal by Service Consumption
  1. [NNLOA] Marginal lift above the average - 3%. N is decently sized, unlikely to be statistically significant.
  2. [Opt Out] Even split, basically.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 67 26 41 0.6 38.8%
Need New LOA 1 45 20 25 0.8 44.4%
Opt Out 0 27 24 3 8.0 88.9%
Opt Out 1 16 14 2 7.0 87.5%
Renewal by Service Consumption Volume
  1. Seeing that even those who consume multiple do not show improvement on renewal rate.

Renewal by Service Consumption, OT

  1. NNLOA: FY20 and FY21 have the most consumption and nearly no lift, hence the overall result.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2019 11 1 10 0.1 9.1%
Need New LOA 0 2020 22 7 15 0.5 31.8%
Need New LOA 0 2021 28 14 14 1.0 50.0%
Need New LOA 0 2022 6 4 2 2.0 66.7%
Need New LOA 1 2019 4 2 2 1.0 50.0%
Need New LOA 1 2020 21 9 12 0.8 42.9%
Need New LOA 1 2021 16 8 8 1.0 50.0%
Need New LOA 1 2022 4 1 3 0.3 25.0%
Opt Out 0 2019 6 6 0 Inf 100.0%
Opt Out 0 2020 10 10 0 Inf 100.0%
Opt Out 0 2021 7 4 3 1.3 57.1%
Opt Out 0 2022 4 4 0 Inf 100.0%
Opt Out 1 2019 6 6 0 Inf 100.0%
Opt Out 1 2020 4 4 0 Inf 100.0%
Opt Out 1 2021 5 3 2 1.5 60.0%
Opt Out 1 2022 1 1 0 Inf 100.0%
Renewal by SL Consumption
  1. [NNLOA] Marginal lift above the mean - 2%. Decent sized N.
  2. [Opt Out] More lift here, than we’ve seen elsewhere.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 61 24 37 0.6 39.3%
Need New LOA 1 51 22 29 0.8 43.1%
Opt Out 0 12 10 2 5.0 83.3%
Opt Out 1 31 28 3 9.3 90.3%

SL Consumption Volume & Renewal

  1. Perhaps 2+ is a bit stronger performance, although not by much.

Renewal by SL Consumption, OT
  1. NNLOA: FY21 was an outstanding positive year.
  2. NNLOA: FY20/22 does not show the same benefit.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2019 10 3 7 0.4 30.0%
Need New LOA 0 2020 21 8 13 0.6 38.1%
Need New LOA 0 2021 23 9 14 0.6 39.1%
Need New LOA 0 2022 7 4 3 1.3 57.1%
Need New LOA 1 2019 5 0 5 0.0 0.0%
Need New LOA 1 2020 22 8 14 0.6 36.4%
Need New LOA 1 2021 21 13 8 1.6 61.9%
Need New LOA 1 2022 3 1 2 0.5 33.3%
Opt Out 0 2019 1 1 0 Inf 100.0%
Opt Out 0 2020 4 4 0 Inf 100.0%
Opt Out 0 2021 4 2 2 1.0 50.0%
Opt Out 0 2022 3 3 0 Inf 100.0%
Opt Out 1 2019 11 11 0 Inf 100.0%
Opt Out 1 2020 10 10 0 Inf 100.0%
Opt Out 1 2021 8 5 3 1.7 62.5%
Opt Out 1 2022 2 2 0 Inf 100.0%
Renewal by Research Interview Consumption
  1. [NNLOA] Essentially no benefit here.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 72 30 42 0.7 41.7%
Need New LOA 1 40 16 24 0.7 40.0%
Opt Out 0 23 18 5 3.6 78.3%
Opt Out 1 20 20 0 Inf 100.0%
Research Interview Volume & Renewal
  1. More does not mean better results.

Renewal By Research Interview Consumption, OT
  1. NNLOA: FY21 is a lot of data working towards the negative.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2019 8 1 7 0.1 12.5%
Need New LOA 0 2020 18 5 13 0.4 27.8%
Need New LOA 0 2021 36 19 17 1.1 52.8%
Need New LOA 0 2022 10 5 5 1.0 50.0%
Need New LOA 1 2019 7 2 5 0.4 28.6%
Need New LOA 1 2020 25 11 14 0.8 44.0%
Need New LOA 1 2021 8 3 5 0.6 37.5%
Opt Out 0 2019 4 4 0 Inf 100.0%
Opt Out 0 2020 5 5 0 Inf 100.0%
Opt Out 0 2021 10 5 5 1.0 50.0%
Opt Out 0 2022 4 4 0 Inf 100.0%
Opt Out 1 2019 8 8 0 Inf 100.0%
Opt Out 1 2020 9 9 0 Inf 100.0%
Opt Out 1 2021 2 2 0 Inf 100.0%
Opt Out 1 2022 1 1 0 Inf 100.0%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Largest lift we’ve seen yet. 22% bump.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 93 34 59 0.6 36.6%
Need New LOA 1 19 12 7 1.7 63.2%
Opt Out 0 35 31 4 7.8 88.6%
Opt Out 1 8 7 1 7.0 87.5%
PLW / Onsite Volume & Renewal
  1. Only takes 1.

Renewal by PLW/Onsite OT
  1. NNLOA: Always at least as good, mostly significantly better.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2019 12 1 11 0.1 8.3%
Need New LOA 0 2020 36 13 23 0.6 36.1%
Need New LOA 0 2021 37 16 21 0.8 43.2%
Need New LOA 0 2022 8 4 4 1.0 50.0%
Need New LOA 1 2019 3 2 1 2.0 66.7%
Need New LOA 1 2020 7 3 4 0.8 42.9%
Need New LOA 1 2021 7 6 1 6.0 85.7%
Need New LOA 1 2022 2 1 1 1.0 50.0%
Opt Out 0 2019 10 10 0 Inf 100.0%
Opt Out 0 2020 12 12 0 Inf 100.0%
Opt Out 0 2021 9 5 4 1.2 55.6%
Opt Out 0 2022 4 4 0 Inf 100.0%
Opt Out 1 2019 2 2 0 Inf 100.0%
Opt Out 1 2020 2 2 0 Inf 100.0%
Opt Out 1 2021 3 2 1 2.0 66.7%
Opt Out 1 2022 1 1 0 Inf 100.0%
Renewal by Experience Consumption

Basically no utilization.

Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 108 44 64 0.7 40.7%
Need New LOA 1 4 2 2 1.0 50.0%
Opt Out 0 39 34 5 6.8 87.2%
Opt Out 1 4 4 0 Inf 100.0%
Experience Volume & Renewal

Basically no utilization.

Renewal by Experience Consumption OT

Basically no utilization.

Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2019 14 2 12 0.2 14.3%
Need New LOA 0 2020 40 15 25 0.6 37.5%
Need New LOA 0 2021 44 22 22 1.0 50.0%
Need New LOA 0 2022 10 5 5 1.0 50.0%
Need New LOA 1 2019 1 1 0 Inf 100.0%
Need New LOA 1 2020 3 1 2 0.5 33.3%
Opt Out 0 2019 10 10 0 Inf 100.0%
Opt Out 0 2020 13 13 0 Inf 100.0%
Opt Out 0 2021 11 6 5 1.2 54.5%
Opt Out 0 2022 5 5 0 Inf 100.0%
Opt Out 1 2019 2 2 0 Inf 100.0%
Opt Out 1 2020 1 1 0 Inf 100.0%
Opt Out 1 2021 1 1 0 Inf 100.0%
Renewal by Expert Call Consumption
  1. [NNLOA] Small negative relationship, with a decent sized N.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 51 23 28 0.8 45.1%
Need New LOA 1 61 23 38 0.6 37.7%
Opt Out 0 14 13 1 13.0 92.9%
Opt Out 1 29 25 4 6.2 86.2%
Expert Call Volume & Renewal
  1. A lot of volume here.
  2. Seeing drops even on the very high utilizers.

Renewal by Expert Call Consumption OT
  1. NNLOA: Consistently underperforming non-consumers since FY20 (included).
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2019 11 1 10 0.1 9.1%
Need New LOA 0 2020 18 9 9 1.0 50.0%
Need New LOA 0 2021 18 10 8 1.2 55.6%
Need New LOA 0 2022 4 3 1 3.0 75.0%
Need New LOA 1 2019 4 2 2 1.0 50.0%
Need New LOA 1 2020 25 7 18 0.4 28.0%
Need New LOA 1 2021 26 12 14 0.9 46.2%
Need New LOA 1 2022 6 2 4 0.5 33.3%
Opt Out 0 2019 4 4 0 Inf 100.0%
Opt Out 0 2020 4 4 0 Inf 100.0%
Opt Out 0 2021 5 4 1 4.0 80.0%
Opt Out 0 2022 1 1 0 Inf 100.0%
Opt Out 1 2019 8 8 0 Inf 100.0%
Opt Out 1 2020 10 10 0 Inf 100.0%
Opt Out 1 2021 7 3 4 0.8 42.9%
Opt Out 1 2022 4 4 0 Inf 100.0%

Finer Grain Look at What Was Consumed

Event Category and Event Purpose

  1. Benchmarking survey looks pretty good compared to the average.
  2. Strategy Gap Analysis is mixed, although the other audits look better.
  3. Tool’s walkthrough?
Renewal Ratio by Event Category and Event Purpose
Event_Category EventPurpose N ATL Dropped Ratio Perc_Renewed
Benchmark 2019 EMF Benchmarking Survey 22 14 8 1.8 63.6%
Benchmark Benchmark 9 6 3 2.0 66.7%
Benchmarking Walkthrough Benchmarking Walkthrough 3 1 2 0.5 33.3%
Collaborative Enrollment Strategy Gap Analysis 1 1 0 Inf 100.0%
Collaborative MREC: Rural Strategy Collaborative 6 4 2 2.0 66.7%
COVID Working Group Session EMF COVID-19 Working Group - Mitigating the Enrollment Impacts of COVID-19 10 8 2 4.0 80.0%
Diagnostic or Audit Enrollment Strategy Gap Analysis 29 12 17 0.7 41.4%
Diagnostic or Audit Need Cliff Analysis 5 4 1 4.0 80.0%
Diagnostic or Audit Orientation Web Audit (EMF) 6 4 2 2.0 66.7%
Evergreen Action Evergreen Action 11 6 5 1.2 54.5%
Executive Update Executive Update 2 0 2 0.0 0.0%
Expert Center Expert Center 48 27 22 1.2 56.2%
Expert Center Expert Center Call 30 16 14 1.1 53.3%
Market Research NA 4 2 2 1.0 50.0%
National Meeting 2018-2019 Enrollment Management Forum National Meetings, 7 4 3 1.3 57.1%
National Meeting 2018-2019 Enrollment Management Forum National Meetings, Chicago, IL 11 8 3 2.7 72.7%
National Meeting 2018-2019 Enrollment Management Forum National Meetings, Los Angeles, CA 2 2 0 Inf 100.0%
National Meeting 2018-2019 Enrollment Management Forum National Meetings, Washington, DC 8 3 5 0.6 37.5%
National Meeting 2019-2020 Enrollment Management Forum National Meeting, 15 10 5 2.0 66.7%
National Meeting 2019-2020 Enrollment Management Forum National Meeting, Washington, DC 7 4 3 1.3 57.1%
National Meeting EMF Executive Pop-ups, 3 1 2 0.5 33.3%
National Meeting EMF Working Session: Fall Admissions Planning, 15 7 8 0.9 46.7%
National Meeting Recruiting Underrepresented Students: An EAB Conversation Series, 7 6 1 6.0 85.7%
National Meeting Serving the Students of the Pandemic: An Exclusive Session for Selective Institutions, 1 1 0 Inf 100.0%
National Meeting The Future of Undergrad Enrollment for Flagship and Selective Institutions, 1 1 0 Inf 100.0%
National Meeting The Future of Undergrad Enrollment for Private Institutions, 2 2 0 Inf 100.0%
National Meeting The Future of Undergrad Enrollment for Public Institutions, 1 1 0 Inf 100.0%
National Meeting Virtual Working Session EMF: Mitigating the Enrollment Impacts of COVID19, 10 8 2 4.0 80.0%
National Meeting Pre-Wire National Meeting Pre-Wire 1 1 0 Inf 100.0%
Onsite Onsite | EMF | 01/07/2019 1 0 1 0.0 0.0%
Onsite Onsite | EMF | 02/08/2019 1 1 0 Inf 100.0%
Onsite Onsite | EMF | 03/29/2019 1 1 0 Inf 100.0%
Onsite Onsite | EMF | 05/09/2019 1 1 0 Inf 100.0%
Onsite Onsite | EMF | 06/13/2019 1 0 1 0.0 0.0%
Onsite Onsite | EMF | 08/27/2019 1 0 1 0.0 0.0%
Onsite Onsite | EMF | Presentation Date: 04/09/2019 1 1 0 Inf 100.0%
Onsite Onsite | EMF | Presentation Date: 04/30/2019 1 0 1 0.0 0.0%
Onsite Onsite | EMF | Presentation Date: 07/23/2019 1 1 0 Inf 100.0%
Onsite Onsite | EMF | Presentation date: 7/11/18 1 0 1 0.0 0.0%
Onsite Onsite | EMF l Presentation date: 9/21/18 1 1 0 Inf 100.0%
Onsite Onsite l EMF l Presentation date: 11/28/18 1 1 0 Inf 100.0%
Onsite Onsite l EMF l Presentation date: 7/31/2018 1 1 0 Inf 100.0%
Onsite Onsite: student Enrollment Expereince - EMF | Presentation Date: 1/29/2020 1 1 0 Inf 100.0%
Onsite Partner Intensive | EMF | Supporting Students Financially| Date of Presentation: 10/19/21 1 1 0 Inf 100.0%
Onsite Planning Call 10/3 1 0 1 0.0 0.0%
Research Initial 2019 EMF Benchmarking Survey 1 1 0 Inf 100.0%
Research Initial Alternative Pricing Strategies 2 2 0 Inf 100.0%
Research Initial EMF Org of the Future 31 18 13 1.4 58.1%
Research Initial EMF Predictive Analytics 2018 7 2 5 0.4 28.6%
Research Initial EMF Selectives Working Group 4 2 2 1.0 50.0%
Research Initial Enrollment Strategy Gap Analysis 1 1 0 Inf 100.0%
Research Initial Managing Campus Crises (Student Activism and Sexual Misconduct) 1 0 1 0.0 0.0%
Research Initial MREC: Rural Strategy Collaborative 2 1 1 1.0 50.0%
Research Initial MREC: Top Challenges and Priorities in Enrollment Marketing & Communications 3 2 1 2.0 66.7%
Research Initial Price Sensitivity 6 3 3 1.0 50.0%
Research Initial Recruitment-Marketing Audience Strategy 2 2 0 Inf 100.0%
Research Initial Student Debt Crisis 14 10 4 2.5 71.4%
Research SL Impact Research SL Impact - Blueprint for Growth/SoF Presentation 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Board Meeting on State of the Sector 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Call 4 2 2 1.0 50.0%
Research SL Impact Research SL Impact - Call - St. Bonaventure - EMF - Ann Lehman (AVP for AA) 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Campus Visit 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Career Website Audit 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Charting a Path to Persistence Discussion 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Check In 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Check In, Enrollment Management 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Custom Analysis 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Custom EMF Working Session Attendance 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Enrollment Growth Self Assessment Discussion 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Enrollment Strategy Gap Analysis Walkthrough 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Enrollment Success Self-Assessment 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Executive Cabinet Presentation 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Expert Call 4 3 1 3.0 75.0%
Research SL Impact Research SL Impact - In-Person Meeting with Enrollment Manager, Karen Stachyra 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - In-Person Meeting with Jennifer Greene 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - In Person Meeting 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - in person meeting + dinner 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - In Person Meeting with SCSU EMF Team 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - in person SL visit at EMF meeting 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - In Person Visit-Research SL Impact-Ann Kalayil 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - In Person Visit-Research SL Impact-Carrie Jo West 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - In Person Visit-Research SL Impact-David Eisler 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - In Person Visit-Research SL Impact-Heather Harker 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - In Person Visit-Research SL Impact-Penny Elkins 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - INT KSC EMF NM March 12-13 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - INT UCM EMF PA 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Leadership Call 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Making the Academic Market Smart and Res Hall Discussion 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Meeting 7 5 2 2.5 71.4%
Research SL Impact Research SL Impact - Meeting - Baylor - EMF - Jennifer Carron (VPEM) 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - National Meeting Prewire 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Partner Intensive Participant: Growth Strategy 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Partner Overview 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Plymouth State Market Research tracking 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - PSU EM @ EMF Feb. 3-4 2020 NM 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Purdue NW, EMF - Joy Colwell, Interim VC of Enrollment - Check-in 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Quarterly Call, Enrollment Management 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Second Exec Roundtable Topic 2 2 0 Inf 100.0%
Research SL Impact Research SL Impact - Service Planning Call 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Sewanee IPM 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - SF MSSU EMF Call 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - SF MSSU virtual meeting 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - SF Seton Hall EMF Presentation 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - SGSU In-Person Meeting 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Student of the Future Debrief Session 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Test Optional Policy Call 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - UCM Pan-EAB calls 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - UGA EMF Call 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Unmet Need Cliff and Career Services 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - VIP Day Meeting 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Virtual Commencement and Virtual Advising 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Working Group Second Program 5 5 0 Inf 100.0%
Research SL Impact Research SL Impact Visit 31 18 14 1.3 58.1%
Tool’s Walkthrough Tool’s Walk Through 16 6 10 0.6 37.5%
Tool’s Walkthrough Tool’s Walk Through Call 2 1 1 1.0 50.0%
VIP Day VIP Day 1 1 0 Inf 100.0%
VIP Day West Virginia University Virtual Impact Workshops 1 1 0 Inf 100.0%
Webinar PLW | EMF | 04/25/2019 1 1 0 Inf 100.0%
Webinar PLW: Customized Presentation - EMF | Date of Presentation: 12/2/2019 1 1 0 Inf 100.0%
Webinar PLW: Paving the Path to Transfer - EMF 1 0 1 0.0 0.0%
Webinar Virtual Meeting | EMF | Customized Presentation | Presentation Date: 10/15/20 1 1 0 Inf 100.0%
Webinar Virtual Meeting | EMF | Enrollment State of the Union 1 1 0 Inf 100.0%
Webinar Virtual Meeting | EMF | Pricing & Aid 1 0 1 0.0 0.0%
Webinar Virtual Meeting | EMF | Pricing and Aid 1 1 0 Inf 100.0%
Webinar Virtual Meeting | EMF | State of the Union| Date of Presentation: 4/22/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | EMF | The Enrollment State of the Union 3 3 0 Inf 100.0%
Webinar Virtual Meeting | EMF | The New Science of Enrollment| Date of Presentation 3/18/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting: South Dakota State: Serving Students of the Pandemic: 1 0 1 0.0 0.0%
NA NA 9 4 5 0.8 44.4%
Correlation betwen Event Groupings, and Renewal
  1. Events and SL Led most correlated with renewal.
  2. Expert Calls have mild positive associations with the other impact interactions.

Multivariate Regression (still kind of simple)

Note: Looking at all decision types.

  1. Events and SL Led are statistically significant at the .1 level.
  2. If you take out the Large P&P and only like NNLOA: events and service pop.
  3. If you only look at Large P&P - all decisions - SL Led and maybe PLW/Onsite
  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 0.44 0.16 – 1.17 0.109
II Events Volume 1.45 0.81 – 2.73 0.226
II SLLed Volume 1.67 0.98 – 3.02 0.070
II ResearchInterview
Volume 		1.13 0.61 – 2.17 0.710
II PLW Onsite Volume 2.75 0.86 – 10.84 0.108
II ExpertCall Volume 1.05 0.67 – 1.73 0.848
II Service Volume 1.60 0.70 – 4.15 0.289
Observations 76
R2 Tjur 0.127