Impact Interaction Analyses - DLF
Josh Wilson
5/7/2024
DLF Analysis
The purpose of this analysis is to understand the variable effect of impact interactions on the likelihood of a partner to renew their partnership. Simply, which service levers matter more for partnership renewal.
Executive Summary
Methodological Considerations
The data set consists of decisions from FY19 through March of FY24 (April).
Similar to previous analyses, we are primarily looking at the year leading up to a decision.
Lastly, we are looking at unit renewal rates instead of $ renewal rates. This allows us to treat impact interactions equitably across partnerships.
Renewal by Decision Type
The below table should not come as a surprise. Opt Outs tend to perform very well while NNLOA can be quite mixed.
- Healthy amount of decisions in both buckets. Opt Outs clearly perform significantly better than NNLOA.
| DecisionType | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|
| Need New LOA | 13 | 13 | 0 | Inf | 100.0% |
| Opt Out | 54 | 52 | 2 | 26.0 | 96.3% |
| Total | 67 | 65 | 2 | 32.5 | 97.0% |
Renewals by Decision Type and Year
Since we are combining multiple years of data, we will frequently take a look at whether there are differences across the years.
- Last 3 FYs, NNLOA has been above 60% and vacillated a bit. We dipped below 60% this FY, which breaks with the threshold set by the past 3 FYs.
- Smaller N this FY than the previous FYs.
- Opt Out performance had been getting better FY19-FY21. Then you had a couple of years of similar performance, with a large swing in the positive for FY24.
| DecisionType | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 2023 | 4 | 4 | 0 | Inf | 100.0% |
| Need New LOA | 2024 | 7 | 7 | 0 | Inf | 100.0% |
| Opt Out | 2020 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 2022 | 9 | 9 | 0 | Inf | 100.0% |
| Opt Out | 2023 | 25 | 24 | 1 | 24.0 | 96.0% |
| Opt Out | 2024 | 18 | 17 | 1 | 17.0 | 94.4% |
| Total | NA | 67 | 65 | 2 | 32.5 | 97.0% |
Impact Interaction Volume Histogram
- Central tendency is just under 5.
- Lots of low utilizers in the 0-2 bucket.
- Bunch of partners in the 3-6 bucket.
- Longish tail starting around 7.l.
Impact Interaction Volume Histogram Split by Decision
- A lot of the drops are in the 0-3.
- Higher utilizers have fewer drops.
Impact Interaction Volume Histogram Split by Pool Date FY
- FY24 has a healthy spread from 3-9, with some partners in the 0-1 range.
- FY23 looks like it might have had a higher proportion in the 0-1 range.
Renewal by Interaction Volume and Decision Type, Unbinned
- [NNLOA] At least 4 and you start to see a healthy return.
- [Opt Out] 2+ looks good here.
| DecisionType | II_Credited_Volume | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 3 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 4 | 5 | 5 | 0 | Inf | 100.0% |
| Need New LOA | 5 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 8 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 10 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 4 | 3 | 1 | 3 | 75.0% |
| Opt Out | 1 | 13 | 13 | 0 | Inf | 100.0% |
| Opt Out | 2 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 3 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 4 | 8 | 7 | 1 | 7 | 87.5% |
| Opt Out | 5 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 6 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 7 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 9 | 7 | 7 | 0 | Inf | 100.0% |
| Opt Out | 11 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 13 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 19 | 1 | 1 | 0 | Inf | 100.0% |
Renewal by Impact Interaction Volume and Decision Type, (Binned)
Adjusted the binning to better capture the high volume present in URF.
- For NNLOA, binning makes 4+ look like a solid space.
- For Opt Out, as suggested above, 2+.
| DecisionType | II_Volume_Bins | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 2-3 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 4-6 | 7 | 7 | 0 | Inf | 100.0% |
| Need New LOA | 7+ | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 0 | 4 | 3 | 1 | 3 | 75.0% |
| Opt Out | 1 | 13 | 13 | 0 | Inf | 100.0% |
| Opt Out | 2-3 | 6 | 6 | 0 | Inf | 100.0% |
| Opt Out | 4-6 | 16 | 15 | 1 | 15 | 93.8% |
| Opt Out | 7+ | 15 | 15 | 0 | Inf | 100.0% |
#####Simple Regression for Impact Interaction Volume
- The Estimate for Impact Interaction Volume is Positive -> increased consumption means increased likelihood to renew.
- This is statistically significant.
##
## Call:
## glm(formula = Target_Binary ~ II_Credited_Volume, family = "binomial",
## data = df_nnloa)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 2.457e+01 7.173e+04 0 1
## II_Credited_Volume 1.365e-10 1.462e+04 0 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 0.0000e+00 on 12 degrees of freedom
## Residual deviance: 5.5727e-10 on 11 degrees of freedom
## AIC: 4
##
## Number of Fisher Scoring iterations: 23Penetration by Event Grouping
Reminder: 1. Experience = VIP Days & Collaboratives
Service = Benchmarking, Audits, Secret Shopper, Customer Research Projects, Market Insights, and Survey Walkthroughs
Expert Call = Expert Center, Benchmarking Walkthrough, Tool’s Walkthrough
SL Led = Research SL Impact, Evergreen Action, Exec Update, National Meeting Pre-wire
Research Interview = Research Initial & Scoping
Events = National Meeting, COVID Working Group
PLW/Onsite = Onsite, PLW
Events and Expert Calls are the highest penetration. Research interviews follows suit in third.
Research Interviews favors Opt Outs, with Onsites/PLW, and Expert Calls.
| DecisionType | Perc_Events | Perc_Service | Perc_SLLed | Perc_ResearchInt | Perc_PLW_Onsite | Perc_Experience | Perc_ExpertCall |
|---|---|---|---|---|---|---|---|
| Need New LOA | 69.2% | 15.4% | 0.0% | 53.8% | 30.8% | 0.0% | 61.5% |
| Opt Out | 61.1% | 22.2% | 0.0% | 40.7% | 18.5% | 0.0% | 63.0% |
Renewal by Event Consumption
- [NNLOA] 10% bump above the average. Giant leap above not consuming - 32%.
- [Opt Out] Large effect for Opt Outs, as well, 15% lift vs. not consuming. However, consuming only small lift above average 5%.
| DecisionType | II_Events_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 4 | 4 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 9 | 9 | 0 | Inf | 100.0% |
| Opt Out | 0 | 21 | 20 | 1 | 20 | 95.2% |
| Opt Out | 1 | 33 | 32 | 1 | 32 | 97.0% |
Renewal by Event Consumption Volume
- 2+ Event seems better than 1, which tracks roughly to the overall average renewal rate.
Renewal by Event, OT
- [NNLOA] Oddly every other year you see a spike in NNNLOA performance.
- [Opt Out] Relatively consistent performance across the FYs.
| DecisionType | II_Events_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2024 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2023 | 4 | 4 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2024 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2023 | 9 | 8 | 1 | 8 | 88.9% |
| Opt Out | 0 | 2024 | 7 | 7 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2020 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2022 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2023 | 16 | 16 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2024 | 11 | 10 | 1 | 10 | 90.9% |
Renewal by Service Consumption
- [NNLOA] No real lift here.
- [Opt Out] No real lift here.
| DecisionType | II_Service_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 11 | 11 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 0 | 42 | 40 | 2 | 20 | 95.2% |
| Opt Out | 1 | 12 | 12 | 0 | Inf | 100.0% |
Renewal by Service Consumption Volume
- Most consume 1 of these, if any.
Renewal by Service Consumption, OT
- [NNLOA] Not many for FY24.
- [Opt Out] Not many for FY24.
| DecisionType | II_Service_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2023 | 3 | 3 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2024 | 6 | 6 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2023 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2024 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2020 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2023 | 21 | 20 | 1 | 20 | 95.2% |
| Opt Out | 0 | 2024 | 14 | 13 | 1 | 13 | 92.9% |
| Opt Out | 1 | 2022 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2023 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2024 | 4 | 4 | 0 | Inf | 100.0% |
Renewal by SL Consumption
- [NNLOA] Marginal benefit, almost no lift. Small N.
- [Opt Out] 5-6% lift above average. Not large volume.
| DecisionType | II_SLLed_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 13 | 13 | 0 | Inf | 100.0% |
| Opt Out | 0 | 54 | 52 | 2 | 26 | 96.3% |
SL Consumption Volume & Renewal
- Mostly only one consumed.
Renewal by SL Consumption, OT
No data.
| DecisionType | II_SLLed_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2023 | 4 | 4 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2024 | 7 | 7 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2020 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 9 | 9 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2023 | 25 | 24 | 1 | 24 | 96.0% |
| Opt Out | 0 | 2024 | 18 | 17 | 1 | 17 | 94.4% |
Renewal by Research Interview Consumption
- [NNLOA] Small lift, 5%.
- [Opt out] No real lift.
| DecisionType | II_ResearchInterview_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 6 | 6 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 7 | 7 | 0 | Inf | 100.0% |
| Opt Out | 0 | 32 | 31 | 1 | 31 | 96.9% |
| Opt Out | 1 | 22 | 21 | 1 | 21 | 95.5% |
Research Interview Volume & Renewal
- 2+ looks good.
Renewal By Research Interview Consumption, OT
- [NNLOA] FY24 is light on this interaction. Although FY23/FY24 look to have larger lift.
- [Opt Out] Stablish across time.
| DecisionType | II_ResearchInterview_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2023 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2024 | 4 | 4 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2023 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2024 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 6 | 6 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2023 | 14 | 13 | 1 | 13 | 92.9% |
| Opt Out | 0 | 2024 | 11 | 11 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2020 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2022 | 3 | 3 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2023 | 11 | 11 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2024 | 7 | 6 | 1 | 6 | 85.7% |
Renewal by PLW OR Onsite Consumption
- [NNLOA] Small N, but huge lift when delivered - 20% above avg.
- [Opt Out] Marginal effect.
| DecisionType | II_PLW_Onsite_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 9 | 9 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 0 | 44 | 42 | 2 | 21 | 95.5% |
| Opt Out | 1 | 10 | 10 | 0 | Inf | 100.0% |
PLW / Onsite Volume & Renewal
- 1 seems sufficient.
Renewal by PLW/Onsite OT
- [NNLOA] Last two year of data is super!
- [Opt Out] Same as NNLOA.
| DecisionType | II_PLW_Onsite_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2023 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2024 | 5 | 5 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2023 | 2 | 2 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2024 | 2 | 2 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2020 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 9 | 9 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2023 | 20 | 19 | 1 | 19 | 95.0% |
| Opt Out | 0 | 2024 | 13 | 12 | 1 | 12 | 92.3% |
| Opt Out | 1 | 2023 | 5 | 5 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2024 | 5 | 5 | 0 | Inf | 100.0% |
Renewal by Experience Consumption
No consumption.
| DecisionType | II_Experience_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 13 | 13 | 0 | Inf | 100.0% |
| Opt Out | 0 | 54 | 52 | 2 | 26 | 96.3% |
Experience Volume & Renewal
No consumption.
Renewal by Experience Consumption OT
No consumption.
| DecisionType | II_Experience_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2023 | 4 | 4 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2024 | 7 | 7 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2020 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 9 | 9 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2023 | 25 | 24 | 1 | 24 | 96.0% |
| Opt Out | 0 | 2024 | 18 | 17 | 1 | 17 | 94.4% |
Renewal by Expert Call Consumption
- [NNLOA] Small lift above the average. Not a large drop if not consumed.
- [Opt Out] 5% of lift above the average.
| DecisionType | II_ExpertCall_Binary | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| Need New LOA | 0 | 5 | 5 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 8 | 8 | 0 | Inf | 100.0% |
| Opt Out | 0 | 20 | 18 | 2 | 9 | 90.0% |
| Opt Out | 1 | 34 | 34 | 0 | Inf | 100.0% |
Expert Call Volume & Renewal
- Decent degree of utilizationm. Effect might be closer to 3+.
Renewal by Expert Call Consumption OT
- [NNLOA] Mostly consistent, except FY23
- [Opt Out] Effect has been pretty strong the past two years.
| DecisionType | II_ExpertCall_Binary | FiscalYear | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|---|
| Need New LOA | 0 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2023 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 0 | 2024 | 3 | 3 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2023 | 3 | 3 | 0 | Inf | 100.0% |
| Need New LOA | 1 | 2024 | 4 | 4 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 0 | 2023 | 8 | 7 | 1 | 7 | 87.5% |
| Opt Out | 0 | 2024 | 11 | 10 | 1 | 10 | 90.9% |
| Opt Out | 1 | 2020 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2021 | 1 | 1 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2022 | 8 | 8 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2023 | 17 | 17 | 0 | Inf | 100.0% |
| Opt Out | 1 | 2024 | 7 | 7 | 0 | Inf | 100.0% |
Finer Grain Look at What Was Consumed
Event Category and Event Purpose
- Benchmark looks very good.
- Tool’s walkthrough.
- SL Led - Return to learn.
| Event_Category | EventPurpose | N | ATL | Dropped | Ratio | Perc_Renewed |
|---|---|---|---|---|---|---|
| COVID Working Group Session | Pulling Up on Learning Acceleration: Superintendent Focus Group Discussion on What’s Working So Far, | 2 | 2 | 0 | Inf | 100.0% |
| Expert Center | DEIJ Expert Call | 1 | 1 | 0 | Inf | 100.0% |
| Expert Center | EAB| DLF Match Webinar *duplicate for logging | 1 | 1 | 0 | Inf | 100.0% |
| Expert Center | Expert Center | 25 | 25 | 0 | Inf | 100.0% |
| Expert Center | SME| Black Hole Campaign Call | 1 | 1 | 0 | Inf | 100.0% |
| K-12 On Demand | Academic achievements and impacts Montessori Schools have on students | 1 | 1 | 0 | Inf | 100.0% |
| K-12 On Demand | On-Demand Research Request - Dual Language Immersion | 1 | 1 | 0 | Inf | 100.0% |
| K-12 On Demand | School to Family/Community Communication and Home Visits | 1 | 1 | 0 | Inf | 100.0% |
| K-12 Webinar | DLF (Virtual 15 Sep 2020) | 2 | 2 | 0 | Inf | 100.0% |
| K-12 Webinar | DLF (Virtual 24 Sep 2020) | 1 | 1 | 0 | Inf | 100.0% |
| K-12 Webinar | DLF (Virtual 27 Aug 2020) | 1 | 1 | 0 | Inf | 100.0% |
| K-12 Webinar | DLF (Virtual 8 Sep 2020) | 1 | 1 | 0 | Inf | 100.0% |
| K-12 Webinar | DLF AAF SAF HESF (Virtual 22 Sep 2020) | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | 2018-2019 District Leadership Forum, Washington, DC | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | 2020 Virtual Superintendent Roundtable on “Return to Learn” Planning, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | 2023 Executive Roundtable for District Superintendents, | 13 | 12 | 1 | 12.0 | 92.3% |
| National Meeting | 2024 Executive Roundtable for District Superintendents, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | A Roadmap for Effective and Efficient Mental Health Care in California Schools, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Collaborative for Creating Conditions for Positive Student Behavior: An EAB-led Experience for Schoo, | 10 | 10 | 0 | Inf | 100.0% |
| National Meeting | Connected Cohort: Collaborative for Creating Conditions for Positive Student Behavior, | 3 | 3 | 0 | Inf | 100.0% |
| National Meeting | DLF Collaborative to Improve Teacher Morale: Back-to-School Planning, | 4 | 4 | 0 | Inf | 100.0% |
| National Meeting | DLF Collaborative to Improve Teacher Morale: Co-Design Solutions Part II, | 3 | 3 | 0 | Inf | 100.0% |
| National Meeting | DLF Collaborative to Improve Teacher Morale: Diagnose Causes, | 4 | 4 | 0 | Inf | 100.0% |
| National Meeting | DLF Collaborative to Improve Teacher Morale: Kickoff, | 4 | 4 | 0 | Inf | 100.0% |
| National Meeting | DLF Collaborative to Improve Teacher Morale: Prioritize Threats, | 3 | 3 | 0 | Inf | 100.0% |
| National Meeting | DLF Collaborative to Improve Teacher Morale: Progress Monitoring, | 6 | 5 | 1 | 5.0 | 83.3% |
| National Meeting | DLF Raising Teacher Morale Collaborative: Co-Design Solutions, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | DLF Raising Teacher Morale Collaborative: Diagnosing Causes, | 4 | 4 | 0 | Inf | 100.0% |
| National Meeting | DLF Raising Teacher Morale Collaborative: Kick Off, | 5 | 5 | 0 | Inf | 100.0% |
| National Meeting | DLF Raising Teacher Morale Collaborative: Prioritizing Threats, | 3 | 3 | 0 | Inf | 100.0% |
| National Meeting | DLF | EAB’s Nationwide Collaborative to Improve Teacher Morale Progress Monitoring: Progress Monitor, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | Developing a Security-First Culture: How to Fortify Your District Against Escalating Cyber Threats, | 4 | 4 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale Cohort Five: Diagnose Causes, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale Cohort Five: Kickoff, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale Cohort Five: Planning for Permanence, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale Cohort Five: Prioritize Threats, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale: Co-Design Solutions, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale: Diagnose Causes, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale: Kickoff, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale: Planning for Permanence, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale: Prioritize Threats, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale: Progress Monitoring #1, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | EAB’s Nationwide Collaborative to Raise Teacher Morale: Progress Monitoring #2, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Efficient Problem Solving 101: An Introduction to Root Cause Analysis, | 3 | 3 | 0 | Inf | 100.0% |
| National Meeting | Executive Roundtable for District Superintendents: Rebuilding the Conditions for Schools to Thrive, | 9 | 9 | 0 | Inf | 100.0% |
| National Meeting | Hardwiring Effective Parent Communications to Improve Attendance, | 3 | 3 | 0 | Inf | 100.0% |
| National Meeting | K-12’s Path Forward: The Seven “Wonders” of a Post-Vaccine World, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Narrowing the Third Grade Reading Gap, | 7 | 6 | 1 | 6.0 | 85.7% |
| National Meeting | New Superintendents Institute: Cultivating Community Trust and Confidence, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | New Superintendents Institute: Launching a Strategic Superintendency, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | New Superintendents Institute: Rethinking the Employee Experience, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | Preparing Principals to Lead Strategy Summit for District Leaders in Illinois, | 1 | 1 | 0 | Inf | 100.0% |
| National Meeting | Preparing for Your Next District Flashpoint: What Every District Leader Should Do Now to Help Naviga, | 6 | 6 | 0 | Inf | 100.0% |
| National Meeting | Reaching Today’s Parents to Reduce Chronic Absenteeism, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | Supported Cohort: Collaborative for Creating Conditions for Positive Student Behavior, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | Voice of the Superintendent Series: Trailblazer Roundtables, | 2 | 2 | 0 | Inf | 100.0% |
| National Meeting | What It Takes to Boost Teacher Morale During the Pandemic: A Listening Tour with District Leaders, | 4 | 4 | 0 | Inf | 100.0% |
| National Meeting | Why Teacher Morale Became the Overlooked Barrier to Pandemic Recovery and How School Districts Can I, | 10 | 10 | 0 | Inf | 100.0% |
| Onsite | Early Literacy Framework Audit | 1 | 1 | 0 | Inf | 100.0% |
| Onsite | Hatboro Horsham Science of Reading Onsite | 1 | 1 | 0 | Inf | 100.0% |
| Onsite | Partner Intensive | DLF | Chronic Absenteeism | 2 | 2 | 0 | Inf | 100.0% |
| Onsite | Partner Intensive | DLF | Narrowing the 3rd Grade Reading Gap | 1 | 1 | 0 | Inf | 100.0% |
| Onsite | Partner Intensive | DLF | Preparing Principals to Lead | 1 | 1 | 0 | Inf | 100.0% |
| Onsite | Partner Intensive | DLF | Principal Design Support Lab | 1 | 1 | 0 | Inf | 100.0% |
| Onsite | Partner Intensive | DLF | Principal Support Design Lab | 1 | 1 | 0 | Inf | 100.0% |
| Onsite | Partner Intensive | DLF | Teacher Morale | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | 2022 DTLF Cybersecurity Research | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | Benchmarking Large Districts’ Student Support Services | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | DLF Flashpoint Response | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | District Program Evaluation | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | Dual Language Immersion Programs | 2 | 2 | 0 | Inf | 100.0% |
| Research Initial | Generative AI for Public School Districts | 3 | 3 | 0 | Inf | 100.0% |
| Research Initial | Improving Student Attendance | 8 | 7 | 1 | 7.0 | 87.5% |
| Research Initial | K12 Student Behavior 2022 | 3 | 3 | 0 | Inf | 100.0% |
| Research Initial | Principal Leadership Scoping | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | Recession Planning | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | Recruiting, Retaining, and Developing Great Principals | 1 | 1 | 0 | Inf | 100.0% |
| Research Initial | Teacher Workforce | 5 | 5 | 0 | Inf | 100.0% |
| Scoping | DLF Flashpoint Response | 3 | 3 | 0 | Inf | 100.0% |
| Scoping | Generative AI for Public School Districts | 6 | 6 | 0 | Inf | 100.0% |
| Scoping | Improving Student Attendance | 2 | 2 | 0 | Inf | 100.0% |
| Scoping | Initiating an Employee Childcare Program | 1 | 1 | 0 | Inf | 100.0% |
| Survey Walkthrough | 2023 Voice of the Superintendent Survey | 5 | 5 | 0 | Inf | 100.0% |
| Survey Walkthrough | 2024 DLF Research Topic Poll | 1 | 1 | 0 | Inf | 100.0% |
| Survey Walkthrough | 2024 Voice of the Superintendent Survey | 1 | 1 | 0 | Inf | 100.0% |
| Survey Walkthrough | K12 Student Behavior 2022 | 1 | 1 | 0 | Inf | 100.0% |
| Survey Walkthrough | Teacher Workforce | 7 | 7 | 0 | Inf | 100.0% |
| Survey Walkthrough | Voice of the Superintendent | 2 | 2 | 0 | Inf | 100.0% |
| Tool’s Walkthrough | Tool’s Walk Through | 4 | 4 | 0 | Inf | 100.0% |
| Webinar | DLF | PLW | Root-Cause Analysis Workshop | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | *Customized - DEIJ | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Building Better Behavior Strategy | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Building Better Behavior Strategy (Part II) | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Building Better Behavior Strategy (Part III) | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Compassionate Leadership | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | HARI | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Hallmarks of an Anti-Racist Institution | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Hidden Opportunities to Boost Postsecondary Success | 2 | 2 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Managing Behavioral Disruptions in Early Grades | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Morale Collaborative (Co-Designing Solutions) | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Narrowing 3rd Grade Reading Gap | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Narrowing the 3rd Grade Reading Gap | 2 | 2 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Principal Design Support Lab | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | Partner Intensive | DLF | Root Cause Analysis | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | RSU 38 Empathy Mapping | 1 | 1 | 0 | Inf | 100.0% |
| Webinar | NA | 1 | 1 | 0 | Inf | 100.0% |
| NA | NA | 5 | 4 | 1 | 4.0 | 80.0% |
Correlation betwen Event Groupings, and Renewal
- Events, Expert Calls, Research Interviews, and PLW/Onsites are most correlated with renewal.
- Positive Correlations: Events appear to be the most halo-ey.
- Negative Correlations: No real negative haloes.
Multivariate Regression (still kind of simple)
Note: Looking at all decision types.
- Overall, Events and Research Interview are statistically significant.
##
## Call:
## glm(formula = Target_Binary ~ II_Events_Volume + II_SLLed_Volume +
## II_ResearchInterview_Volume + II_PLW_Onsite_Volume + II_ExpertCall_Volume +
## II_Service_Volume, family = "binomial", data = df)
##
## Coefficients: (1 not defined because of singularities)
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 1.6645 1.0491 1.587 0.113
## II_Events_Volume 0.1278 0.3966 0.322 0.747
## II_SLLed_Volume NA NA NA NA
## II_ResearchInterview_Volume -0.0436 1.3207 -0.033 0.974
## II_PLW_Onsite_Volume 17.7224 7697.1914 0.002 0.998
## II_ExpertCall_Volume 19.5895 5582.6320 0.004 0.997
## II_Service_Volume 17.8283 8311.9033 0.002 0.998
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 17.986 on 66 degrees of freedom
## Residual deviance: 11.648 on 61 degrees of freedom
## AIC: 23.648
##
## Number of Fisher Scoring iterations: 21