Impact Interaction Analyses - HESF
Josh Wilson
5/11/2022
HESF Analysis
The purpose of this analysis is to understand the variable effect of impact interactions on the likelihood of a partner to renew their partnership. Simply, which service levers matter more for partnership renewal.
Executive Summary
Methodological Considerations
The data set consists of decisions from FY19 through March of FY22.
Similar to previous analyses, we are primarily looking at the year leading up to a decision.
We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.
Lastly, we are looking at unit renewal rates instead of $ renewal rates. This allows us to treat impact interactions equitably across partnerships.
Renewal by Decision Type
The below table should not come as a surprise. Opt Outs tend to perform very well while NNLOA can be quite mixed.
- Remember the Numbers: 76% for NNLOA, 91% for Opt Outs.
- NNLOA perform exceedingly well as compared to other forums. Opt Outs perform at the same level.
- 5 Drops is not a lot and understanding differences in impact interactions could be difficult.
| DecisionType | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|
| Need New LOA | 21 | 16 | 5 | 3.2 | 76.2% |
| Opt Out | 23 | 21 | 2 | 10.5 | 91.3% |
| Total | 44 | 37 | 7 | 5.3 | 84.1% |
Renewals by Decision Type and Year
Since we are combining multiple years of data, we will frequently take a look at whether there are differences across the years.
- NNLOA - Solid increase in renewal rate FY22; only two years of data
- Opt Outs increased in renewal rate across the time span, with FY21/22 being perfect.
| DecisionType | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 2021 | 14 | 10 | 4 | 2.5 | 71.4% |
| Need New LOA | 2022 | 7 | 6 | 1 | 6.0 | 85.7% |
| Opt Out | 2019 | 2 | 1 | 1 | 1.0 | 50.0% |
| Opt Out | 2020 | 9 | 8 | 1 | 8.0 | 88.9% |
| Opt Out | 2021 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 2022 | 7 | 7 | 0 | Inf | 100.0% |
| Total | NA | 44 | 37 | 7 | 5.3 | 84.1% |
Renewal Rate by Account Segment
- Most in the Large P&P, with very few drops.
- Most of the drops come from Selectives.
- Regional Publics have the second highest N, and the second highest drops.
| AccountSegment | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|
| 4 Year Other | 1 | 1 | 0 | Inf | 100.0% |
| Canada | 2 | 2 | 0 | Inf | 100.0% |
| Large Public & Private | 23 | 22 | 1 | 22.0 | 95.7% |
| Regional Private | 1 | 1 | 0 | Inf | 100.0% |
| Regional Public | 9 | 7 | 2 | 3.5 | 77.8% |
| Selective | 8 | 4 | 4 | 1.0 | 50.0% |
Renewal Rate by Account Segment & Fiscal Year
- Selectives have a drop a year, at least, albeit with small N each year.
| AccountSegment | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| 4 Year Other | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Canada | 2022 | 2 | 2 | 0 | Inf | 100.0% |
| Large Public & Private | 2020 | 4 | 4 | 0 | Inf | 100.0% |
| Large Public & Private | 2021 | 10 | 9 | 1 | 9.0 | 90.0% |
| Large Public & Private | 2022 | 9 | 9 | 0 | Inf | 100.0% |
| Regional Private | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Regional Public | 2019 | 2 | 1 | 1 | 1.0 | 50.0% |
| Regional Public | 2020 | 2 | 2 | 0 | Inf | 100.0% |
| Regional Public | 2021 | 5 | 4 | 1 | 4.0 | 80.0% |
| Selective | 2020 | 3 | 2 | 1 | 2.0 | 66.7% |
| Selective | 2021 | 3 | 1 | 2 | 0.5 | 33.3% |
| Selective | 2022 | 2 | 1 | 1 | 1.0 | 50.0% |
Renewal by Decision Type & Segment
- Selective NNLOA is really the only weak point.
- Everywhere else we are seeing renewal rates at very high levels.
| AccountSegment | DecisionType | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| 4 Year Other | Opt Out | 1 | 1 | 0 | Inf | 100.0% |
| Canada | Need New LOA | 1 | 1 | 0 | Inf | 100.0% |
| Canada | Opt Out | 1 | 1 | 0 | Inf | 100.0% |
| Large Public & Private | Need New LOA | 11 | 10 | 1 | 10.0 | 90.9% |
| Large Public & Private | Opt Out | 12 | 12 | 0 | Inf | 100.0% |
| Regional Private | Need New LOA | 1 | 1 | 0 | Inf | 100.0% |
| Regional Public | Need New LOA | 4 | 3 | 1 | 3.0 | 75.0% |
| Regional Public | Opt Out | 5 | 4 | 1 | 4.0 | 80.0% |
| Selective | Need New LOA | 4 | 1 | 3 | 0.3 | 25.0% |
| Selective | Opt Out | 4 | 3 | 1 | 3.0 | 75.0% |
CRP Inclusion
No CRPs! Took them out.
| GRP3-4 | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|
| 0 | 44 | 37 | 7 | 5.3 | 84.1% |
Impact Interaction Volume Histogram
- Central tendency increased to about 5.6, one of the highest we’ve seen.
- There is tail with heavy a few utilizers.
- A couple of peaks at 3 and 5.
Impact Interaction Volume Histogram Split by Decision
- Not many drops. They are scattered across utilization amounts.
- This likely means it will be tough to say whether utilization is having a strong positive impact.
Impact Interaction Volume Histogram Split by Pool Date FY
- FY22 units have lower utilization than FY21, which you see comprises much of the tail.
Renewal by Interaction Volume and Decision Type, Unbinned
- [NNLOA] With a few of the drops in the both the lower and higher utilization bucket, 4+ might be a good bet.
| DecisionType | II_Credited_Volume | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 1 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 3 | 3 | 1 | 2 | 0.5 | 33.3% |
| Need New LOA | 4 | 4 | 4 | 0 | Inf | 100.0% |
| Need New LOA | 5 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 6 | 5 | 3 | 2 | 1.5 | 60.0% |
| Need New LOA | 8 | 1 | 0 | 1 | 0.0 | 0.0% |
| Need New LOA | 9 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 10 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 13 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 14 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 16 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 1 | 4 | 3 | 1 | 3.0 | 75.0% |
| Opt Out | 2 | 2 | 1 | 1 | 1.0 | 50.0% |
| Opt Out | 3 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 5 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 6 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 7 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 8 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 17 | 1 | 1 | 0 | Inf | 100.0% |
Renewal by Impact Interaction Volume and Decision Type, (Binned)
Adjusted the binning to better capture the high volume present in BAF.
- [NNLOA] Increasing renewal rate from 2-3 onwards.
- [NNLOA] 4+ is again confirmed.
- [Opt Out] Could argue for any engagement.
| DecisionType | II_Volume_Bins | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 1 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 2-3 | 3 | 1 | 2 | 0.5 | 33.3% |
| Need New LOA | 4-6 | 10 | 8 | 2 | 4.0 | 80.0% |
| Need New LOA | 7+ | 7 | 6 | 1 | 6.0 | 85.7% |
| Opt Out | 0 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 1 | 4 | 3 | 1 | 3.0 | 75.0% |
| Opt Out | 2-3 | 6 | 5 | 1 | 5.0 | 83.3% |
| Opt Out | 4-6 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 7+ | 6 | 6 | 0 | Inf | 100.0% |
Renewal by Impact Interaction Volume and Account Segment, (Binned)
- Parsed to finely.
- Maybe Selectives need more interaction - that’s not necessarily data backed.
| AccountSegment | II_Volume_Bins | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| 4 Year Other | 2-3 | 1 | 1 | 0 | Inf | 100.0% |
| Canada | 1 | 1 | 1 | 0 | Inf | 100.0% |
| Canada | 2-3 | 1 | 1 | 0 | Inf | 100.0% |
| Large Public & Private | 0 | 1 | 1 | 0 | Inf | 100.0% |
| Large Public & Private | 1 | 2 | 2 | 0 | Inf | 100.0% |
| Large Public & Private | 2-3 | 5 | 4 | 1 | 4.0 | 80.0% |
| Large Public & Private | 4-6 | 8 | 8 | 0 | Inf | 100.0% |
| Large Public & Private | 7+ | 7 | 7 | 0 | Inf | 100.0% |
| Regional Private | 4-6 | 1 | 1 | 0 | Inf | 100.0% |
| Regional Public | 1 | 1 | 0 | 1 | 0.0 | 0.0% |
| Regional Public | 4-6 | 3 | 3 | 0 | Inf | 100.0% |
| Regional Public | 7+ | 5 | 4 | 1 | 4.0 | 80.0% |
| Selective | 0 | 1 | 1 | 0 | Inf | 100.0% |
| Selective | 1 | 1 | 1 | 0 | Inf | 100.0% |
| Selective | 2-3 | 2 | 0 | 2 | 0.0 | 0.0% |
| Selective | 4-6 | 3 | 1 | 2 | 0.5 | 33.3% |
| Selective | 7+ | 1 | 1 | 0 | Inf | 100.0% |
Large P&P Renewal by Impact Interaction Volume (Binned) & Decision Type
- Too finely parsed.
| DecisionType | II_Volume_Bins | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 2-3 | 2 | 1 | 1 | 1 | 50.0% |
| Need New LOA | 4-6 | 6 | 6 | 0 | Inf | 100.0% |
| Need New LOA | 7+ | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 0 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 2-3 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 4-6 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 7+ | 4 | 4 | 0 | Inf | 100.0% |
#####Simple Regression for Impact Interaction Volume
- The Estimate for Impact Interaction Volume is Positive -> increased consumption means increased likelihood to renew.
- This is NOT statistically significant.
| Target_Binary | |||
|---|---|---|---|
| Predictors | Odds Ratios | CI | p |
| (Intercept) | 2.80 | 0.73 – 11.85 | 0.139 |
| II Credited Volume | 1.14 | 0.91 – 1.53 | 0.315 |
| Observations | 44 | ||
| R2 Tjur | 0.022 | ||
Penetration by Event Grouping
- Expert Calls, Events, and SL Led are the highest penetration, which is similar to AAF and BAF.
- Events and Expert Calls are more heavily penetrated for NNLOA vs. Opt Outs.
Reminder: 1. Experience = VIP Days & Collaboratives
Service = Benchmarking, Audits, Secret Shopper, Customer Research Projects, Market Insights, and Survey Walkthroughs
Expert Call = Expert Center, Benchmarking Walkthrough, Tool’s Walkthrough
SL Led = Research SL Impact, Evergreen Action, Exec Update, National Meeting Pre-wire
Research Interview = Research Initial & Scoping
Events = National Meeting, COVID Working Group
PLW/Onsite = Onsite, PLW
| DecisionType | Perc_Events | Perc_Service | Perc_SLLed | Perc_ResearchInt | Perc_PLW_Onsite | Perc_Experience | Perc_ExpertCall |
|---|---|---|---|---|---|---|---|
| Need New LOA | 81.0% | 9.5% | 57.1% | 19.0% | 28.6% | 0.0% | 71.4% |
| Opt Out | 56.5% | 0.0% | 52.2% | 21.7% | 26.1% | 4.3% | 56.5% |
Renewal by Event Consumption
- [NNLOA] Almost all of the NNLOA units attended events, which also means the drops were allocated here. Inversion is meaningless.
- [Opt Out] Inversion here as well, but the split on attendance is more even.
| DecisionType | II_Events_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 4 | 4 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 17 | 12 | 5 | 2.4 | 70.6% |
| Opt Out | 0 | 10 | 10 | 0 | Inf | 100.0% |
| Opt Out | 1 | 13 | 11 | 2 | 5.5 | 84.6% |
Renewal by Event Consumption Volume
- Crazy high event volume vs. what we’ve seen in other forums
- Wonder if this program is a catchall or something.
Renewal by Event, OT
- NNLOA: Only FY21/22 utilization and the drops are also allocated at these timepoints -> makes events look worse!
- Opt Out: Drops were all pre-FY21
| DecisionType | II_Events_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2022 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2021 | 12 | 8 | 4 | 2 | 66.7% |
| Need New LOA | 1 | 2022 | 5 | 4 | 1 | 4 | 80.0% |
| Opt Out | 0 | 2020 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2021 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2019 | 2 | 1 | 1 | 1 | 50.0% |
| Opt Out | 1 | 2020 | 6 | 5 | 1 | 5 | 83.3% |
| Opt Out | 1 | 2021 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2022 | 2 | 2 | 0 | Inf | 100.0% |
Renewal by Service Consumption
- [NNLOA] Low N. 1 Drop out of 2 utilizations.
- [Opt Out] No consumption here.
| DecisionType | II_Service_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 19 | 15 | 4 | 3.8 | 78.9% |
| Need New LOA | 1 | 2 | 1 | 1 | 1.0 | 50.0% |
| Opt Out | 0 | 23 | 21 | 2 | 10.5 | 91.3% |
Renewal by Service Consumption Volume
- Wild that the one unit which renewed consumed 6 of these.
Renewal by Service Consumption, OT
- NNLOA: All recent consumption.
| DecisionType | II_Service_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 13 | 9 | 4 | 2.2 | 69.2% |
| Need New LOA | 0 | 2022 | 6 | 6 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2022 | 1 | 0 | 1 | 0.0 | 0.0% |
| Opt Out | 0 | 2019 | 2 | 1 | 1 | 1.0 | 50.0% |
| Opt Out | 0 | 2020 | 9 | 8 | 1 | 8.0 | 88.9% |
| Opt Out | 0 | 2021 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 7 | 7 | 0 | Inf | 100.0% |
Renewal by SL Consumption
- [NNLOA] 6% bump from SL Led Interactions.
- [Opt Out] Even split.
| DecisionType | II_SLLed_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 9 | 6 | 3 | 2 | 66.7% |
| Need New LOA | 1 | 12 | 10 | 2 | 5 | 83.3% |
| Opt Out | 0 | 11 | 10 | 1 | 10 | 90.9% |
| Opt Out | 1 | 12 | 11 | 1 | 11 | 91.7% |
SL Consumption Volume & Renewal
- Nicely constrained between 1-3 interactions. More than 1 looks strong.
Renewal by SL Consumption, OT
- NNLOA: Consistently better than not consuming.
- NNLOA: Only FY21/22 consumption.
| DecisionType | II_SLLed_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 6 | 4 | 2 | 2 | 66.7% |
| Need New LOA | 0 | 2022 | 3 | 2 | 1 | 2 | 66.7% |
| Need New LOA | 1 | 2021 | 8 | 6 | 2 | 3 | 75.0% |
| Need New LOA | 1 | 2022 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2019 | 1 | 0 | 1 | 0 | 0.0% |
| Opt Out | 0 | 2020 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2021 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2019 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2020 | 5 | 4 | 1 | 4 | 80.0% |
| Opt Out | 1 | 2021 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2022 | 3 | 3 | 0 | Inf | 100.0% |
Renewal by Research Interview Consumption
- [NNLOA/Opt Out] Low N, but captures only those who renew.
| DecisionType | II_ResearchInterview_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 17 | 12 | 5 | 2.4 | 70.6% |
| Need New LOA | 1 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 0 | 18 | 16 | 2 | 8.0 | 88.9% |
| Opt Out | 1 | 5 | 5 | 0 | Inf | 100.0% |
Research Interview Volume & Renewal
- Any consumption is positive.
Renewal By Research Interview Consumption, OT
- NNLOA/Opt Out: Consistent across the past few years, in the amount delivered, and of course, in the renewal rate, which is perfect.
| DecisionType | II_ResearchInterview_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 11 | 7 | 4 | 1.8 | 63.6% |
| Need New LOA | 0 | 2022 | 6 | 5 | 1 | 5.0 | 83.3% |
| Need New LOA | 1 | 2021 | 3 | 3 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2019 | 2 | 1 | 1 | 1.0 | 50.0% |
| Opt Out | 0 | 2020 | 7 | 6 | 1 | 6.0 | 85.7% |
| Opt Out | 0 | 2021 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2020 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2022 | 2 | 2 | 0 | Inf | 100.0% |
Renewal by PLW OR Onsite Consumption
- [NNLOA] Small lift of 7% above the average.
| DecisionType | II_PLW_Onsite_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 15 | 11 | 4 | 2.8 | 73.3% |
| Need New LOA | 1 | 6 | 5 | 1 | 5.0 | 83.3% |
| Opt Out | 0 | 17 | 15 | 2 | 7.5 | 88.2% |
| Opt Out | 1 | 6 | 6 | 0 | Inf | 100.0% |
PLW / Onsite Volume & Renewal
- A few institutions got 2+.
- 1 is where the effect starts.
Renewal by PLW/Onsite OT
- NNLOA: Always better to consume this than not. Not many in FY22.
- Opt Outs: Perfect. Nothing more to say.
| DecisionType | II_PLW_Onsite_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 9 | 6 | 3 | 2 | 66.7% |
| Need New LOA | 0 | 2022 | 6 | 5 | 1 | 5 | 83.3% |
| Need New LOA | 1 | 2021 | 5 | 4 | 1 | 4 | 80.0% |
| Need New LOA | 1 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2019 | 2 | 1 | 1 | 1 | 50.0% |
| Opt Out | 0 | 2020 | 7 | 6 | 1 | 6 | 85.7% |
| Opt Out | 0 | 2021 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2020 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2021 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2022 | 2 | 2 | 0 | Inf | 100.0% |
Renewal by Experience Consumption
Basically no utilization.
| DecisionType | II_Experience_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 21 | 16 | 5 | 3.2 | 76.2% |
| Opt Out | 0 | 22 | 20 | 2 | 10.0 | 90.9% |
| Opt Out | 1 | 1 | 1 | 0 | Inf | 100.0% |
Experience Volume & Renewal
Basically no utilization.
Renewal by Experience Consumption OT
Basically no utilization.
| DecisionType | II_Experience_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 14 | 10 | 4 | 2.5 | 71.4% |
| Need New LOA | 0 | 2022 | 7 | 6 | 1 | 6.0 | 85.7% |
| Opt Out | 0 | 2019 | 2 | 1 | 1 | 1.0 | 50.0% |
| Opt Out | 0 | 2020 | 9 | 8 | 1 | 8.0 | 88.9% |
| Opt Out | 0 | 2021 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 7 | 7 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
Renewal by Expert Call Consumption
- [NNLOA] Decent utilization with a small lift - 4%. Not consuming is 10% drop.
- [Opt Out] Perfect.
| DecisionType | II_ExpertCall_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 6 | 4 | 2 | 2 | 66.7% |
| Need New LOA | 1 | 15 | 12 | 3 | 4 | 80.0% |
| Opt Out | 0 | 10 | 8 | 2 | 4 | 80.0% |
| Opt Out | 1 | 13 | 13 | 0 | Inf | 100.0% |
Expert Call Volume & Renewal
- A lot of volume here, too.
- Drops are a bit scattered on utilization. Effect, if it exists, starts at 1.
Renewal by Expert Call Consumption OT
- NNLOA: No clear pattern. Only two years and it flips from positive to negative.
| DecisionType | II_ExpertCall_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 4 | 2 | 2 | 1 | 50.0% |
| Need New LOA | 0 | 2022 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2021 | 10 | 8 | 2 | 4 | 80.0% |
| Need New LOA | 1 | 2022 | 5 | 4 | 1 | 4 | 80.0% |
| Opt Out | 0 | 2019 | 1 | 0 | 1 | 0 | 0.0% |
| Opt Out | 0 | 2020 | 4 | 3 | 1 | 3 | 75.0% |
| Opt Out | 0 | 2021 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2019 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2020 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2021 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2022 | 5 | 5 | 0 | Inf | 100.0% |
Finer Grain Look at What Was Consumed
Event Category and Event Purpose
- Presidential Lab looks compelling from a ‘garnered interest’ perspective.
- Same with Hallmarks of antiracist institution.
- Tool’s walkthrough?
| Event_Category | EventPurpose | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| COVID Working Group Session | HESF COVID-19 Working Group - CoS Critical Conversation | 2 | 0 | 2 | 0.0 | 0.0% |
| COVID Working Group Session | The Chief of Staff’s Role in Government Relations and Strategic Partnerships: A Working Session to E, | 2 | 2 | 0 | Inf | 100.0% |
| Executive Update | Executive Update | 1 | 1 | 0 | Inf | 100.0% |
| Expert Center | Expert Center | 23 | 20 | 3 | 6.7 | 87.0% |
| Expert Center | Expert Center Call | 5 | 5 | 0 | Inf | 100.0% |
| Market Research | NA | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | 2019 Strategy Forum National Meeting, Washington, DC | 4 | 3 | 1 | 3.0 | 75.0% |
| National Meeting | 2019 Student of the Future, Washington, DC | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | 2020 Chief of Staff Leadership Virtual Series, | 3 | 1 | 2 | 0.5 | 33.3% |
| National Meeting | 2020 HESF Series, | 7 | 5 | 2 | 2.5 | 71.4% |
| National Meeting | 2020 HESF Series, San Jose, CA | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | 2021 Presidential Experience Lab: The Future of Work Post-Pandemic and Implications for Higher Ed, | 8 | 7 | 1 | 7.0 | 87.5% |
| National Meeting | Cultivate an International Partnership Portfolio, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Rising Higher Education Leaders Fellowship, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | EAB Presidents Advisory Council Inaugural Meetings, | 3 | 3 | 0 | Inf | 100.0% |
| National Meeting | EAB Presidents Advisory Council, | 3 | 3 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Rising Higher Education Leaders Fellowship, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Empower Faculty and Unit Leaders to Make Smarter Partnership Decisions, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | Exploring the Changing Landscape of College Athletics, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Future Presidents Intensive: Optional Session, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | HESF GRP Executive Pop-Up, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | HESF Working Sessions, | 5 | 3 | 2 | 1.5 | 60.0% |
| National Meeting | Identify New Partnership Opportunities and Expansions, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | Imperatives for the Post-Vaccine University, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Looking Back at the Fall and Looking Ahead to the Future, | 8 | 7 | 1 | 7.0 | 87.5% |
| National Meeting | New Presidents Intensive Summer Cohort, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | New Presidents Intensive, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | OPMs at a Crossroad: Transformation, Consolidation, and Scrutiny in Online Education, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Presidents Zappos Tour 2018, | 2 | 1 | 1 | 1.0 | 50.0% |
| National Meeting | The Future of Work: Strategic Imperatives for Higher Ed in the Digital Economy, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Virtual Executive Strategy Session for University Leaders: Emerging from 2020 in a Position of Stren, | 2 | 1 | 1 | 1.0 | 50.0% |
| National Meeting | Virtual Working Session HESF: Responding to COVID19: Chiefs of Staff Critical Conversation, | 2 | 0 | 2 | 0.0 | 0.0% |
| National Meeting | Virtual Working Session HESF: Responding to COVID19: Organizational Resiliency, | 3 | 1 | 2 | 0.5 | 33.3% |
| Onsite | Onsite | HESF | 10/18/2019 | 1 | 1 | 0 | Inf | 100.0% |
| Onsite | NA | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | 2021 Chief Diversity Officer | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | Board Onboarding Toolkit | 2 | 2 | 0 | Inf | 100.0% |
| Research Initial | Carnegie Re-Classification | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | Critical Conversation: Coordinating Cross-Campus Response to Flashpoints | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | Hallmarks of an Antiracist Institution | 5 | 5 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - 360 Degree Student Equity Audit | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Call | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - DEIJ call | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - DEIJ Walkthrough | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Expert Call | 2 | 2 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Expert Session with David Attis | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Future Visioning Session | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - GRP/HESF SL Impact Interaction | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - HESF Call with Ira Blake, President, UH-Clear Lake | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - HESF Planning Session | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Higher Education Strategy Forum Welcome Discussion | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Impact Call | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - In Person Visit | 1 | 0 | 1 | 0.0 | 0.0% |
| Research SL Impact | Research SL Impact - In Person Visit-Research SL Impact-Alan Finn | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - In Person Visit-Research SL Impact-Annette Trimbee | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - In Person Visit-Research SL Impact-Kris Devine | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - LRP Discussion | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - LRP Leadership Discussion | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Meeting - CMU - BAF - Daryl Weinert | 1 | 0 | 1 | 0.0 | 0.0% |
| Research SL Impact | Research SL Impact - Office Hours Podcast | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Partner Intensive Participant: Supporting Students Financially | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - Second Exec Roundtable Topic | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - SF NIU HESF Fellowship | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact - SF SH HESF Fellowship | 1 | 1 | 0 | Inf | 100.0% |
| Research SL Impact | Research SL Impact Visit | 5 | 4 | 1 | 4.0 | 80.0% |
| Scoping | 2021 Student Belonging Scoping | 1 | 1 | 0 | Inf | 100.0% |
| Survey Walkthrough | Academic Affairs Organizational Benchmarking Survey | 1 | 0 | 1 | 0.0 | 0.0% |
| Tool’s Walkthrough | Tool’s Walk Through | 5 | 5 | 0 | Inf | 100.0% |
| Tool’s Walkthrough | Tool’s Walk Through Call | 1 | 1 | 0 | Inf | 100.0% |
| VIP Day | West Virginia University Virtual Impact Workshops | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Scenario Planning | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Scenario Planning Workshop for Francesco Cesareo and Cabinet | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Future Visioning | Presentation Date: 01/19/2021 | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Blueprint for Growth: The Future of Adult and Graduate Education | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Blueprint for Growth: The Future of Adult and Graduate Education | Presentation Date: 11/20/2020 | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Customized Presentation | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Customized Presentation | 5/18/21 | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Customized Presentation | 6/6/21 | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Future of Work | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Future Visioning | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Future Visioning | Presentation Date: 11/20/20 | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Future Visioning | Presentation Date: 12/11/20 | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Future Visioning | Presentation date: 9/28/20 | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Global Engagement | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | Radical Business Models | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | HESF | State of the Union | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Virtual Meeting | SAF | Tackling Institutional Blind Spots in Advance of the 2020 Election | Presentation date: 10/15/20 | 1 | 0 | 1 | 0.0 | 0.0% |
| Webinar | NA | 2 | 2 | 0 | Inf | 100.0% |
| NA | NA | 2 | 2 | 0 | Inf | 100.0% |
Correlation betwen Event Groupings, and Renewal
- Most of the categories have a positive association with Renewal - suprising that Events works the other way, but we saw above that nearly all of the units consumed events and therefore the drops are also there!
- SL Led, Events, and Onsites/PLW all have high positive correlations: they are consumed together.
Multivariate Regression (still kind of simple)
Note: Looking at all decision types.
- Nothing is significant.
| Target_Binary | |||
|---|---|---|---|
| Predictors | Odds Ratios | CI | p |
| (Intercept) | 2.65 | 0.61 – 14.32 | 0.213 |
| II Events Volume | 0.68 | 0.39 – 1.09 | 0.125 |
| II SLLed Volume | 1.85 | 0.48 – 12.34 | 0.430 |
|
II ResearchInterview Volume |
4437346889626451615246842600442068060002800424064482806.00 | 0.00 – NA | 0.995 |
| II PLW Onsite Volume | 4.96 | 0.82 – 378.18 | 0.268 |
| II ExpertCall Volume | 1.48 | 0.92 – 3.51 | 0.230 |
| II Service Volume | 0.00 | NA – 10440297064330968171486000486244480660026220846686646826806808624286620808662420866.00 | 0.996 |
| Observations | 44 | ||
| R2 Tjur | 0.284 | ||