DLF Analysis

The purpose of this analysis is to understand the variable effect of impact interactions on the likelihood of a partner to renew their partnership. Simply, which service levers matter more for partnership renewal.

Executive Summary

Methodological Considerations

  1. The data set consists of decisions from FY19 through March of FY22.

  2. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  3. Lastly, we are looking at unit renewal rates instead of $ renewal rates. This allows us to treat impact interactions equitably across partnerships.

Renewal by Decision Type

The below table should not come as a surprise. Opt Outs tend to perform very well while NNLOA can be quite mixed.

  1. Remember the Ratios: 1.6 for NNLOA, 5.2 for Opt Outs.
  2. Most of the decisions are opt outs: ~60%.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 77 47 30 1.6 61.0%
Opt Out 131 110 21 5.2 84.0%
Total 208 157 51 3.1 75.5%

Renewals by Decision Type and Year

Since we are combining multiple years of data, we will frequently take a look at whether there are differences across the years.

  1. NNLOA performance had been climbing since FY20. Small N in FY20.
  2. Opt Out performance had been getting better FY19-FY21. This year is a deviation - looks more like FY19.
Renewal Ratio by Decision Type
DecisionType FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 2020 10 3 7 0.4 30.0%
Need New LOA 2021 43 28 15 1.9 65.1%
Need New LOA 2022 24 16 8 2.0 66.7%
Opt Out 2019 16 12 4 3.0 75.0%
Opt Out 2020 45 38 7 5.4 84.4%
Opt Out 2021 48 43 5 8.6 89.6%
Opt Out 2022 22 17 5 3.4 77.3%
Total NA 208 157 51 3.1 75.5%
Impact Interaction Volume Histogram
  1. Central tendency is around 5.35.
  2. Lots of low utilizers in the 0-2 bucket.
  3. Another peak at 6.
  4. And then a longish tail.

Impact Interaction Volume Histogram Split by Decision
  1. A lot of the drops are in the 0-3.
  2. Higher utilizers have fewer drops.

Impact Interaction Volume Histogram Split by Pool Date FY
  1. FY22 looks like it is distributed towards the bottom end of the utilization spectrum.
  2. FY20/21 have more representation in long tail.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. [NNLOA] At least 3 and you start to see a return. 6+ might be the healthier amount.
  2. [Opt Out] 2+ looks good here.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 5 1 4 0.2 20.0%
Need New LOA 1 5 1 4 0.2 20.0%
Need New LOA 2 10 4 6 0.7 40.0%
Need New LOA 3 13 8 5 1.6 61.5%
Need New LOA 4 8 5 3 1.7 62.5%
Need New LOA 5 6 4 2 2.0 66.7%
Need New LOA 6 12 10 2 5.0 83.3%
Need New LOA 7 6 6 0 Inf 100.0%
Need New LOA 8 6 5 1 5.0 83.3%
Need New LOA 9 2 1 1 1.0 50.0%
Need New LOA 12 1 0 1 0.0 0.0%
Need New LOA 13 1 1 0 Inf 100.0%
Need New LOA 14 1 0 1 0.0 0.0%
Need New LOA 16 1 1 0 Inf 100.0%
Opt Out 0 7 4 3 1.3 57.1%
Opt Out 1 13 8 5 1.6 61.5%
Opt Out 2 16 13 3 4.3 81.2%
Opt Out 3 5 4 1 4.0 80.0%
Opt Out 4 9 8 1 8.0 88.9%
Opt Out 5 19 16 3 5.3 84.2%
Opt Out 6 14 12 2 6.0 85.7%
Opt Out 7 8 7 1 7.0 87.5%
Opt Out 8 10 10 0 Inf 100.0%
Opt Out 9 8 7 1 7.0 87.5%
Opt Out 10 7 7 0 Inf 100.0%
Opt Out 11 7 6 1 6.0 85.7%
Opt Out 12 1 1 0 Inf 100.0%
Opt Out 13 2 2 0 Inf 100.0%
Opt Out 14 1 1 0 Inf 100.0%
Opt Out 16 1 1 0 Inf 100.0%
Opt Out 17 1 1 0 Inf 100.0%
Opt Out 18 1 1 0 Inf 100.0%
Opt Out 19 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume and Decision Type, (Binned)

Adjusted the binning to better capture the high volume present in URF.

  1. For NNLOA, binning makes 4+ look like a solid space.
  2. For Opt Out, as suggested above, 2+.
Renewal Ratio by Impact Interaction Volume (Binned)
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 5 1 4 0.2 20.0%
Need New LOA 1 5 1 4 0.2 20.0%
Need New LOA 2-3 23 12 11 1.1 52.2%
Need New LOA 4-6 26 19 7 2.7 73.1%
Need New LOA 7+ 18 14 4 3.5 77.8%
Opt Out 0 7 4 3 1.3 57.1%
Opt Out 1 13 8 5 1.6 61.5%
Opt Out 2-3 21 17 4 4.2 81.0%
Opt Out 4-6 42 36 6 6.0 85.7%
Opt Out 7+ 48 45 3 15.0 93.8%

#####Simple Regression for Impact Interaction Volume

  1. The Estimate for Impact Interaction Volume is Positive -> increased consumption means increased likelihood to renew.
  2. This is statistically significant.
  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 0.62 0.25 – 1.47 0.287
II Credited Volume 1.23 1.04 – 1.50 0.022
Observations 77
R2 Tjur 0.090
Penetration by Event Grouping

Reminder: 1. Experience = VIP Days & Collaboratives

  1. Service = Benchmarking, Audits, Secret Shopper, Customer Research Projects, Market Insights, and Survey Walkthroughs

  2. Expert Call = Expert Center, Benchmarking Walkthrough, Tool’s Walkthrough

  3. SL Led = Research SL Impact, Evergreen Action, Exec Update, National Meeting Pre-wire

  4. Research Interview = Research Initial & Scoping

  5. Events = National Meeting, COVID Working Group

  6. PLW/Onsite = Onsite, PLW

  7. Events and Expert Calls are the highest penetration. Research interviews follows suit in third, which is

  8. Research Interviews favors Opt Outs, with Events, Expert Calls, and Service leaning towards opt outs as well.

Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall
Need New LOA 62.3% 5.2% 20.8% 36.4% 20.8% 0.0% 67.5%
Opt Out 71.0% 17.6% 22.1% 53.4% 21.4% 0.0% 71.0%
Renewal by Event Consumption
  1. [NNLOA] 9% bump above the average. Giant leap above not consuming - 26%.
  2. [Opt Out] Large effect for Opt Outs, as well, 22% lift vs. not consuming and 15% above the avg.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 29 13 16 0.8 44.8%
Need New LOA 1 48 34 14 2.4 70.8%
Opt Out 0 38 26 12 2.2 68.4%
Opt Out 1 93 84 9 9.3 90.3%
Renewal by Event Consumption Volume
  1. 2+ Event seems better than 1, which is also better than none.

Renewal by Event, OT

  1. [NNLOA] Consistent for FY21/22.
  2. [Opt Out] Mostly driven by FY21/22 decisions.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2020 6 2 4 0.5 33.3%
Need New LOA 0 2021 18 9 9 1.0 50.0%
Need New LOA 0 2022 5 2 3 0.7 40.0%
Need New LOA 1 2020 4 1 3 0.3 25.0%
Need New LOA 1 2021 25 19 6 3.2 76.0%
Need New LOA 1 2022 19 14 5 2.8 73.7%
Opt Out 0 2019 4 3 1 3.0 75.0%
Opt Out 0 2020 11 8 3 2.7 72.7%
Opt Out 0 2021 14 11 3 3.7 78.6%
Opt Out 0 2022 9 4 5 0.8 44.4%
Opt Out 1 2019 12 9 3 3.0 75.0%
Opt Out 1 2020 34 30 4 7.5 88.2%
Opt Out 1 2021 34 32 2 16.0 94.1%
Opt Out 1 2022 13 13 0 Inf 100.0%
Renewal by Service Consumption
  1. [NNLOA] N is too low to make a call here.
  2. [Opt Out] Inversion, but the margin is low, so not much to worry about.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 73 44 29 1.5 60.3%
Need New LOA 1 4 3 1 3.0 75.0%
Opt Out 0 108 91 17 5.4 84.3%
Opt Out 1 23 19 4 4.8 82.6%
Renewal by Service Consumption Volume
  1. Most consume 1 of these that’s it.

Renewal by Service Consumption, OT

  1. [NNLOA] Only one year of data: promising results, though.
  2. [Opt Out] Looks great in FY19.FY20/22 are not as promising.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2020 10 3 7 0.4 30.0%
Need New LOA 0 2021 43 28 15 1.9 65.1%
Need New LOA 0 2022 20 13 7 1.9 65.0%
Need New LOA 1 2022 4 3 1 3.0 75.0%
Opt Out 0 2019 6 3 3 1.0 50.0%
Opt Out 0 2020 38 33 5 6.6 86.8%
Opt Out 0 2021 48 43 5 8.6 89.6%
Opt Out 0 2022 16 12 4 3.0 75.0%
Opt Out 1 2019 10 9 1 9.0 90.0%
Opt Out 1 2020 7 5 2 2.5 71.4%
Opt Out 1 2022 6 5 1 5.0 83.3%
Renewal by SL Consumption
  1. [NNLOA] Marginal benefit, almost no lift. Small N.
  2. [Opt Out] Same as NNLOA.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 61 37 24 1.5 60.7%
Need New LOA 1 16 10 6 1.7 62.5%
Opt Out 0 102 84 18 4.7 82.4%
Opt Out 1 29 26 3 8.7 89.7%

SL Consumption Volume & Renewal

  1. Mostly only one consumed.

Renewal by SL Consumption, OT
  1. [NNLOA] FY21 is the major contributor, is inversely related, and drives most of the relationship.
  2. [Opt Out] FY21 looks good, but there is no FY22.
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2020 9 2 7 0.3 22.2%
Need New LOA 0 2021 29 20 9 2.2 69.0%
Need New LOA 0 2022 23 15 8 1.9 65.2%
Need New LOA 1 2020 1 1 0 Inf 100.0%
Need New LOA 1 2021 14 8 6 1.3 57.1%
Need New LOA 1 2022 1 1 0 Inf 100.0%
Opt Out 0 2019 14 12 2 6.0 85.7%
Opt Out 0 2020 36 29 7 4.1 80.6%
Opt Out 0 2021 30 26 4 6.5 86.7%
Opt Out 0 2022 22 17 5 3.4 77.3%
Opt Out 1 2019 2 0 2 0.0 0.0%
Opt Out 1 2020 9 9 0 Inf 100.0%
Opt Out 1 2021 18 17 1 17.0 94.4%
Renewal by Research Interview Consumption
  1. [NNLOA] Higher N, inverse relationship. -8%.
  2. [Opt out] Small lift, with a lot of consumption.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 49 32 17 1.9 65.3%
Need New LOA 1 28 15 13 1.2 53.6%
Opt Out 0 61 49 12 4.1 80.3%
Opt Out 1 70 61 9 6.8 87.1%
Research Interview Volume & Renewal
  1. 2+ looks good.

Renewal By Research Interview Consumption, OT
  1. [NNLOA] Alternates Year over Year.
  2. [Opt Out] Last two FY are difficult to beat.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2020 2 1 1 1.0 50.0%
Need New LOA 0 2021 31 20 11 1.8 64.5%
Need New LOA 0 2022 16 11 5 2.2 68.8%
Need New LOA 1 2020 8 2 6 0.3 25.0%
Need New LOA 1 2021 12 8 4 2.0 66.7%
Need New LOA 1 2022 8 5 3 1.7 62.5%
Opt Out 0 2019 1 1 0 Inf 100.0%
Opt Out 0 2020 11 9 2 4.5 81.8%
Opt Out 0 2021 34 29 5 5.8 85.3%
Opt Out 0 2022 15 10 5 2.0 66.7%
Opt Out 1 2019 15 11 4 2.8 73.3%
Opt Out 1 2020 34 29 5 5.8 85.3%
Opt Out 1 2021 14 14 0 Inf 100.0%
Opt Out 1 2022 7 7 0 Inf 100.0%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Decent lift of 7%. Not consuming is close to the average.
  2. [Opt Out] Marginal effect.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 61 36 25 1.4 59.0%
Need New LOA 1 16 11 5 2.2 68.8%
Opt Out 0 103 86 17 5.1 83.5%
Opt Out 1 28 24 4 6.0 85.7%
PLW / Onsite Volume & Renewal
  1. 1 seems sufficient.

Renewal by PLW/Onsite OT
  1. [NNLOA] Consistently as good or better than not consuming.
  2. [Opt Out] FY21 was strong. No FY22 data.
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2020 7 2 5 0.4 28.6%
Need New LOA 0 2021 36 22 14 1.6 61.1%
Need New LOA 0 2022 18 12 6 2.0 66.7%
Need New LOA 1 2020 3 1 2 0.5 33.3%
Need New LOA 1 2021 7 6 1 6.0 85.7%
Need New LOA 1 2022 6 4 2 2.0 66.7%
Opt Out 0 2019 11 8 3 2.7 72.7%
Opt Out 0 2020 31 27 4 6.8 87.1%
Opt Out 0 2021 39 34 5 6.8 87.2%
Opt Out 0 2022 22 17 5 3.4 77.3%
Opt Out 1 2019 5 4 1 4.0 80.0%
Opt Out 1 2020 14 11 3 3.7 78.6%
Opt Out 1 2021 9 9 0 Inf 100.0%
Renewal by Experience Consumption

No consumption.

Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 77 47 30 1.6 61.0%
Opt Out 0 131 110 21 5.2 84.0%
Experience Volume & Renewal

No consumption.

Renewal by Experience Consumption OT

No consumption.

Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2020 10 3 7 0.4 30.0%
Need New LOA 0 2021 43 28 15 1.9 65.1%
Need New LOA 0 2022 24 16 8 2.0 66.7%
Opt Out 0 2019 16 12 4 3.0 75.0%
Opt Out 0 2020 45 38 7 5.4 84.4%
Opt Out 0 2021 48 43 5 8.6 89.6%
Opt Out 0 2022 22 17 5 3.4 77.3%
Renewal by Expert Call Consumption
  1. [NNLOA] Small lift above the average. Larger drop if not consumed.
  2. [Opt Out] Same as NNLOA.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 25 13 12 1.1 52.0%
Need New LOA 1 52 34 18 1.9 65.4%
Opt Out 0 38 29 9 3.2 76.3%
Opt Out 1 93 81 12 6.8 87.1%
Expert Call Volume & Renewal
  1. Decent degree of utilizationm. Effect might be closer to 2+.

Renewal by Expert Call Consumption OT
  1. [NNLOA] Consistently better. Effect getting larger.
  2. [Opt Out] Effect has been pretty strong the past two years.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 2020 2 0 2 0.0 0.0%
Need New LOA 0 2021 15 9 6 1.5 60.0%
Need New LOA 0 2022 8 4 4 1.0 50.0%
Need New LOA 1 2020 8 3 5 0.6 37.5%
Need New LOA 1 2021 28 19 9 2.1 67.9%
Need New LOA 1 2022 16 12 4 3.0 75.0%
Opt Out 0 2019 4 2 2 1.0 50.0%
Opt Out 0 2020 11 10 1 10.0 90.9%
Opt Out 0 2021 16 13 3 4.3 81.2%
Opt Out 0 2022 7 4 3 1.3 57.1%
Opt Out 1 2019 12 10 2 5.0 83.3%
Opt Out 1 2020 34 28 6 4.7 82.4%
Opt Out 1 2021 32 30 2 15.0 93.8%
Opt Out 1 2022 15 13 2 6.5 86.7%

Finer Grain Look at What Was Consumed

Event Category and Event Purpose

  1. Benchmark looks very good.
  2. Tool’s walkthrough.
  3. SL Led - Return to learn.
Renewal Ratio by Event Category and Event Purpose
Event_Category EventPurpose N ATL Dropped Ratio Perc_Renewed
Benchmark DLF Survey: Managing Disruptive Behaviors Member Survey 17 14 3 4.7 82.4%
COVID Working Group Session DLF COVID-19 Working Group Session 16 12 4 3.0 75.0%
COVID Working Group Session DLF COVID 19 Return to School 10 10 0 Inf 100.0%
COVID Working Group Session DLF Lessons Learned COVID in Case Reclosures 1 1 0 Inf 100.0%
COVID Working Group Session Pulling Up on Learning Acceleration: Superintendent Focus Group Discussion on What’s Working So Far, 8 7 1 7.0 87.5%
Expert Center Expert Center 91 77 17 4.5 84.6%
Expert Center Expert Center Call 24 15 9 1.7 62.5%
K-12 On Demand NA 41 34 7 4.9 82.9%
K-12 Webinar DLF (Virtual 15 Sep 2020) 22 19 3 6.3 86.4%
K-12 Webinar DLF (Virtual 20 Aug 2020) 8 5 3 1.7 62.5%
K-12 Webinar DLF (Virtual 24 Sep 2020) 26 21 5 4.2 80.8%
K-12 Webinar DLF (Virtual 27 Aug 2020) 26 24 2 12.0 92.3%
K-12 Webinar DLF (Virtual 8 Sep 2020) 14 12 2 6.0 85.7%
K-12 Webinar DLF AAF SAF HESF (Virtual 22 Sep 2020) 28 24 4 6.0 85.7%
K-12 Webinar DLF CTO (Virtual 28 Oct 2020) 16 12 4 3.0 75.0%
K-12 Webinar DLF ISEF (Virtual 18 Aug 2020) 20 16 4 4.0 80.0%
National Meeting 2017-2018 District Leadership Forum, Chicago, IL 1 0 1 0.0 0.0%
National Meeting 2017-2018 District Leadership Forum, Washington, DC 3 3 0 Inf 100.0%
National Meeting 2018-2019 District Leadership Forum, Dallas, TX 9 9 0 Inf 100.0%
National Meeting 2018-2019 District Leadership Forum, Los Angeles, CA 3 2 1 2.0 66.7%
National Meeting 2018-2019 District Leadership Forum, Washington, DC 25 22 3 7.3 88.0%
National Meeting 2018 District Leadership Forum Special Session, Vancouver, WA 1 0 1 0.0 0.0%
National Meeting 2019-2020 District Leadership Forum National Meeting, 33 28 5 5.6 84.8%
National Meeting 2019-2020 District Leadership Forum National Meeting, Austin, TX 8 7 1 7.0 87.5%
National Meeting 2019-2020 District Leadership Forum National Meeting, Los Angeles, CA 3 1 2 0.5 33.3%
National Meeting 2019-2020 District Leadership Forum National Meeting, Washington, DC 9 8 1 8.0 88.9%
National Meeting 2019 District Leadership Forum Special Session, Tyler, TX 1 1 0 Inf 100.0%
National Meeting 2020 Virtual Superintendent Roundtable on “Return to Learn” Planning, 26 23 3 7.7 88.5%
National Meeting DLF Prospect Roundtables, 1 1 0 Inf 100.0%
National Meeting EAB and Coxsackie-Athens School District | Building a High-Performance District Leadership Team, 1 1 0 Inf 100.0%
National Meeting EAB and Mansfield ISD | How District Leaders Can Craft Strategy and Messaging for the 2021/22 School, 1 1 0 Inf 100.0%
National Meeting EAB and Mansfield Park ISD | How District Leaders Can Craft Strategy and Messaging for the 2021/22 S, 1 1 0 Inf 100.0%
National Meeting EAB and Waukegan CUSD 60 | Crafting Strategy and Communications to Advance Key Priorities in 2021, 1 1 0 Inf 100.0%
National Meeting Joint Virtual Working Session DLF and ISEF: Joint session with CEO of One Schoolhouse , 5 4 1 4.0 80.0%
National Meeting K-12’s Hidden Opportunities to Boost Postsecondary Success: How Students’ College Decision Making Ha, 13 13 0 Inf 100.0%
National Meeting K-12’s Path Forward: The Seven “Wonders” of a Post-Vaccine World, 9 9 0 Inf 100.0%
National Meeting K-12’s Path Forward: The Seven Wonders of a Post-Vaccine World, 9 7 2 3.5 77.8%
National Meeting Learning Recovery Leadership Workshop, 9 9 0 Inf 100.0%
National Meeting Recession Proof Your Budget: EAB Collaborative Series on recession Planning, 31 26 5 5.2 83.9%
National Meeting Recession Proof Your Budget: EAB Collaborative Series on Recession Planning, 10 9 1 9.0 90.0%
National Meeting Recovery Planning Collaborative for Virginia Division Leaders, 5 5 0 Inf 100.0%
National Meeting Virtual Working Session DLF: Advancement Challenges Amidst COVID-19 Crisis, 2 1 1 1.0 50.0%
National Meeting Virtual Working Session DLF: General Superintendents Session, 1 1 0 Inf 100.0%
National Meeting Virtual Working Session DLF: Identfying Students in Need, 2 2 0 Inf 100.0%
National Meeting Virtual Working Session DLF: SEL in a Virtual World, 3 2 1 2.0 66.7%
National Meeting Virtual Working Session DLF: Serving Special Ed Students and IEP, 4 3 1 3.0 75.0%
National Meeting Virtual Working Session DLF: Special Ed, 1 1 0 Inf 100.0%
National Meeting Virtual Working Session DLF: Student Mental Health, 3 2 1 2.0 66.7%
National Meeting Virtual Working Session DLF: Supporting the Mental Health Needs of Teachers and Staff, 1 0 1 0.0 0.0%
National Meeting What It Takes to Boost Teacher Morale During the Pandemic: A Listening Tour with District Leaders, 3 3 0 Inf 100.0%
National Meeting What You Need to Know About the Postsecondary Success Crises: How the Pandemic Has Changed Student D, 16 15 1 15.0 93.8%
National Meeting Why Teacher Morale Became the Overlooked Barrier to Pandemic Recovery and How School Districts Can I, 3 3 0 Inf 100.0%
Onsite Hold exp. 08/07 | Onsite | DLF | 10/29/2019 1 1 0 Inf 100.0%
Onsite Onsite | DLF | 01/30/2019 1 1 0 Inf 100.0%
Onsite Onsite | DLF | 02/27/2019 1 1 0 Inf 100.0%
Onsite Onsite | DLF | 03/13/2019 1 1 0 Inf 100.0%
Onsite Onsite | DLF | 05/29/2019 1 1 0 Inf 100.0%
Onsite Onsite | DLF | 09/19/2019 1 1 0 Inf 100.0%
Onsite Onsite | DLF | 12/04/2018 1 1 0 Inf 100.0%
Onsite Onsite | DLF | Date of Presentation: 2/2/2019 1 1 0 Inf 100.0%
Onsite Onsite | DLF | Presentation Date: 06/06/2019 1 0 1 0.0 0.0%
Onsite Onsite | DLF | Presentation Date: 07/22/2019 1 1 0 Inf 100.0%
Onsite Onsite | DLF | Presentation Date: 07/31/2019 1 0 1 0.0 0.0%
Onsite Onsite | DLF | Presentation Date: 12/13/18 1 0 1 0.0 0.0%
Onsite Onsite | DLF | Presentation Date: July 17 &18, 2019 1 1 0 Inf 100.0%
Onsite Onsite l DLF l Presentation date: 11/6/18 1 1 0 Inf 100.0%
Onsite Onsite l DLF l Presentation date: 6/12/18 1 1 0 Inf 100.0%
Onsite Onsite: Managing Behavioral Disruptions in Early Grades - DLF - 10/1/2019 1 1 0 Inf 100.0%
Onsite Onsite: Managing Behavioral Disruptions in Early Grades - DLF - 8/06/2019 1 1 0 Inf 100.0%
Onsite Onsite: Narrowing the 3rd Grade Reading Gap - DLF - 11/4/2020 1 1 0 Inf 100.0%
Onsite Onsite: Scope MtM Closing the College Access Gap - DLF - 1/14/2020 1 1 0 Inf 100.0%
Onsite NA 7 3 4 0.8 42.9%
Research Initial District Program Evaluation 3 3 0 Inf 100.0%
Research Initial Dual Language Immersion Programs 2 1 1 1.0 50.0%
Research Initial Establishing Postsecondary Transition Programs for Students with Disabilities 1 1 0 Inf 100.0%
Research Initial Fairfax Advanced Course Limits 1 1 0 Inf 100.0%
Research Initial Fairfax Dual Enrollment 1 1 0 Inf 100.0%
Research Initial Full-Day PreK Programs 1 1 0 Inf 100.0%
Research Initial Impacts of School-Provided GPA, Class Rank, and Advanced Course Limits 1 1 0 Inf 100.0%
Research Initial K12 Declining Enrollment Pandemic 16 15 1 15.0 93.8%
Research Initial Male Mentoring 1 1 0 Inf 100.0%
Research Initial Managing Behavioral Disruptions in Early Grades 14 10 4 2.5 71.4%
Research Initial Middle School Gifted and Talented 1 0 1 0.0 0.0%
Research Initial Middle School Grade Retention 1 1 0 Inf 100.0%
Research Initial Middle School Schedules 1 1 0 Inf 100.0%
Research Initial Multi Grade Classrooms 1 1 0 Inf 100.0%
Research Initial Multimodal Learning Research 4 4 0 Inf 100.0%
Research Initial Narrowing the 3rd Grade Reading Gap 10 8 2 4.0 80.0%
Research Initial NOLA Public Schools - Early Warning Systems 1 1 0 Inf 100.0%
Research Initial Non-Immersion World Language Programs in Elementary Schools 1 1 0 Inf 100.0%
Research Initial NPD K12 CAO Research 6 5 1 5.0 83.3%
Research Initial NPD K12 CBO Research 1 1 0 Inf 100.0%
Research Initial Opening a New District High School 1 1 0 Inf 100.0%
Research Initial Paraprofessional Support 1 0 1 0.0 0.0%
Research Initial Pine Bush Learning Management System Study 1 1 0 Inf 100.0%
Research Initial Preparing Teachers for Effective Block Schedule Implementation 1 0 1 0.0 0.0%
Research Initial Preventing and Addressing Cyberbullying Initiatives 1 1 0 Inf 100.0%
Research Initial Project Approval Management Review 1 1 0 Inf 100.0%
Research Initial Pull-Out Services in Special Education 1 1 0 Inf 100.0%
Research Initial Recession Planning 9 6 3 2.0 66.7%
Research Initial Recruiting, Retaining, and Developing Great Principals 21 19 2 9.5 90.5%
Research Initial Responding to the Adolescent Mental Health Crisis 18 14 4 3.5 77.8%
Research Initial School Start Times 1 0 1 0.0 0.0%
Research Initial SPED Research Calls 3 1 2 0.5 33.3%
Research Initial Strategic Planning 1 1 0 Inf 100.0%
Research Initial Summer Programs for Kindergarten and Sixth Grade Readiness 1 1 0 Inf 100.0%
Research Initial Supporting English Language Learners in Low-Incidence Districts 3 3 0 Inf 100.0%
Research Initial Talent Development 1 1 0 Inf 100.0%
Research Initial Teacher Mentorship Programs 1 1 0 Inf 100.0%
Research Initial Teacher Workforce 8 7 1 7.0 87.5%
Research Initial Year-Round Calendars 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - AASA Panelist 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - AASA Reading Panelist 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Co-hosted COVID webinar 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Co-presented with us at NYSCOSS on DEIJ plan writing 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - HARI 3 3 0 Inf 100.0%
Research SL Impact Research SL Impact - NYSCOSS Presentation 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Return to Learn Webcon Attendees 33 28 5 5.6 84.8%
Research SL Impact Research SL Impact - Return To Learn Webcon Attendees 2 1 1 1.0 50.0%
Research SL Impact Research SL Impact - Sit down at AASA 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact Visit 2 0 2 0.0 0.0%
Scoping Effective Practices for Aligning Pre-K Through Third Grade 1 1 0 Inf 100.0%
Scoping Elementary School Grade Configurations 1 1 0 Inf 100.0%
Scoping Implementing Math Pedagogy Reforms 1 0 1 0.0 0.0%
Scoping Middle School Gifted and Talented 1 1 0 Inf 100.0%
Scoping NOLA Public Schools - Early Warning Systems 1 1 0 Inf 100.0%
Scoping Parent Involvement and Engagement at Rural Secondary Schools 1 1 0 Inf 100.0%
Scoping Recruiting, Retaining, and Developing Great Principals 9 8 1 8.0 88.9%
Scoping Talent Development 1 1 0 Inf 100.0%
Survey Walkthrough Teacher Workforce 12 10 2 5.0 83.3%
Survey Walkthrough Voice of the Superintendent 1 1 0 Inf 100.0%
Tool’s Walkthrough Tool’s Walk Through 50 39 12 3.2 78.0%
Tool’s Walkthrough Tool’s Walk Through Call 1 1 0 Inf 100.0%
Webinar Partner Intensive | DLF | Systems Thinking Applications | Presentation Date: 11/01/21 1 0 1 0.0 0.0%
Webinar PLW: *Customized Presentation - DLF - 7/20/2020 1 1 0 Inf 100.0%
Webinar PLW: Closing the College Access Gap - DLF - 4/16/2020 1 1 0 Inf 100.0%
Webinar PLW: Narrowing the 3rd Grade Reading Gap - DLF- 3/13/2020 1 1 0 Inf 100.0%
Webinar PLW: Narrowing the 3rd Grade Reading Gap - DLF - 7/31/2020 1 1 0 Inf 100.0%
Webinar PLW: Preparing Principals to Lead - DLF - 6/4/2020 1 1 0 Inf 100.0%
Webinar PLW: Responding to the Adolescent Mental Health Crisis - DLF - 4/27/2020 1 1 0 Inf 100.0%
Webinar PLW: Responding to the Adolescent Mental Health Crisis - DLF - 5/21/2020 1 1 0 Inf 100.0%
Webinar PLW: Responding to the Adolescent Mental Health Crisis - DLF - 6/17/2020 1 1 0 Inf 100.0%
Webinar PLW: Responding to the Adolescent Mental Health Crisis - DLF - 6/5/2020 1 1 0 Inf 100.0%
Webinar PLW: Responding to the Adolescent Mental Health Crisis - DLF - 6/9/2020 1 1 0 Inf 100.0%
Webinar PLW: TBD - DLF - 4/20/2020 1 1 0 Inf 100.0%
Webinar Root Cause Analysis | Presentation Date: 02/05/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | DLF | Hallmarks of an Anti-Racist Institution | 6/22/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | DLF | Mental Health PD session | 8/18/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | DLF | Narrowing 3rd Grade Reading Gap | 5/21/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | DLF | Narrowing the 3rd Grade Reading Gap | 8/26/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | DLF | Narrowing the 3rd Grade Reading Gap | 8/4/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | DLF | Narrowing the 3rd Grade Reading Gap: Part 1 | 5/7/21 1 0 1 0.0 0.0%
Webinar Virtual Meeting | DLF | Narrowing the 3rd Grade Reading Gap: Part 2 | 5/21/21 1 0 1 0.0 0.0%
Webinar Virtual Meeting | DLF | Science of Reading PD session | 8/18/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting: “Return to Learn” Planning - DLF - 7/1 1 1 0 Inf 100.0%
Webinar Virtual Presentation: Preparing Principals to Lead - DLF 1 1 0 Inf 100.0%
NA NA 11 5 7 0.7 45.5%
Correlation betwen Event Groupings, and Renewal
  1. Events, Expert Calls, and Research Interviews are most correlated with renewal.
  2. Positive Correlations: Events appear to be the most halo-ey.
  3. Negative Correlations: SL Led interactions have some negative associations.

Multivariate Regression (still kind of simple)

Note: Looking at all decision types.

  1. Overall, Events and Expert Calls are statistically significant.
  2. For NNLOAs, only Events pop as significant.
  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 1.02 0.57 – 1.82 0.960
II Events Volume 1.77 1.29 – 2.54 0.001
II SLLed Volume 1.16 0.49 – 2.92 0.741
II ResearchInterview
Volume 		1.20 0.96 – 1.64 0.184
II PLW Onsite Volume 0.73 0.35 – 1.62 0.417
II ExpertCall Volume 1.36 0.99 – 1.92 0.070
II Service Volume 1.08 0.37 – 3.67 0.893
Observations 208
R2 Tjur 0.115