PAE Analysis

The purpose of this analysis is to understand the variable effect of impact interactions on the likelihood of a partner to renew their partnership. Simply, which service levers matter more for partnership renewal.

Executive Summary

Methodological Considerations

  1. The data set consists of decisions from FY19 through March of FY22.

  2. Similar to previous analyses, we are primarily looking at the year leading up to a decision.

  3. We are not looking at ‘integrated’ partnerships. That is, we are focusing on utilization at the account where the renewal decision is made.

  4. Lastly, we are looking at unit renewal rates instead of $ renewal rates. This allows us to treat impact interactions equitably across partnerships.

Renewal by Decision Type

The below table should not come as a surprise. Opt Outs tend to perform very well while NNLOA can be quite mixed.

  1. Remember the Ratios: 1.1 for NNLOA, 11.5 for Opt Outs.
  2. Opt Outs perform exceedingly well, while NNLOA are hovering just above 50%.
  3. Very similar to AAF.
Renewal Ratio by Decision Type
DecisionType N ATL Dropped Ratio Perc_Renewed
Need New LOA 202 107 95 1.1 53.0%
Opt Out 100 92 8 11.5 92.0%
Total 302 199 103 1.9 65.9%

Renewals by Decision Type and Year

Since we are combining multiple years of data, we will frequently take a look at whether there are differences across the years.

  1. NNLOA performance has been climbing since FY19!
  2. FY19 and FY22 are lower on total decisions being made. Perhaps this is a part of a cycle.
  3. Opt Outs have been relatively consistent, with this current FY being in the middle.
Renewal Ratio by Decision Type
DecisionType FiscalYear N ATL Dropped Ratio Perc_Renewed
Need New LOA 2019 18 5 13 0.4 27.8%
Need New LOA 2020 77 36 41 0.9 46.8%
Need New LOA 2021 73 41 32 1.3 56.2%
Need New LOA 2022 34 25 9 2.8 73.5%
Opt Out 2019 20 20 0 Inf 100.0%
Opt Out 2020 35 32 3 10.7 91.4%
Opt Out 2021 25 22 3 7.3 88.0%
Opt Out 2022 20 18 2 9.0 90.0%
Total NA 302 199 103 1.9 65.9%
Renewal Rate by Account Segment
  1. Over 40% of the decisions lie with Large P&P, then an even split between the remaining 3 domestic segments..
  2. Large P&P has peformed head and shoulders above the rest..
  3. Small incremental gains from Regional Private to Selective.
Renewal Ratio by Account Segment
AccountSegment N ATL Dropped Ratio Perc_Renewed
Canada 8 5 3 1.7 62.5%
Large Public & Private 136 104 32 3.2 76.5%
Regional Private 49 24 25 1.0 49.0%
Regional Public 55 31 24 1.3 56.4%
Selective 54 35 19 1.8 64.8%
Renewal Rate by Account Segment & Fiscal Year
  1. Large P&P has alternating years of higher and lower performance.
  2. Regional Private has shown some resilience since a nadir in FY20.
  3. Regional Publics have been the steady the past two years.
  4. Selectives have rebounded to FY19 highs - perhaps this is a 3 year cycle thing?
Renewal Ratio by Account Segment
AccountSegment FiscalYear N ATL Dropped Ratio Perc_Renewed
Canada 2019 1 1 0 Inf 100.0%
Canada 2020 2 2 0 Inf 100.0%
Canada 2021 2 0 2 0.0 0.0%
Canada 2022 3 2 1 2.0 66.7%
Large Public & Private 2019 14 10 4 2.5 71.4%
Large Public & Private 2020 46 37 9 4.1 80.4%
Large Public & Private 2021 50 35 15 2.3 70.0%
Large Public & Private 2022 26 22 4 5.5 84.6%
Regional Private 2019 7 3 4 0.8 42.9%
Regional Private 2020 24 8 16 0.5 33.3%
Regional Private 2021 13 9 4 2.2 69.2%
Regional Private 2022 5 4 1 4.0 80.0%
Regional Public 2019 9 5 4 1.2 55.6%
Regional Public 2020 21 10 11 0.9 47.6%
Regional Public 2021 16 10 6 1.7 62.5%
Regional Public 2022 9 6 3 2.0 66.7%
Selective 2019 7 6 1 6.0 85.7%
Selective 2020 19 11 8 1.4 57.9%
Selective 2021 17 9 8 1.1 52.9%
Selective 2022 11 9 2 4.5 81.8%
Renewal by Decision Type & Segment
  1. Opt Out performance in high in Large P&P and Regional Publics, while lower in the other Segments.
  2. Regional space is very low on NNLOA.
  3. Opt Out has low N in Regional Private, which is low relative to the proportion of total opt outs occupy in the other segments.
Renewal Ratio by Account Segment
AccountSegment DecisionType N ATL Dropped Ratio Perc_Renewed
Canada Need New LOA 3 1 2 0.5 33.3%
Canada Opt Out 5 4 1 4.0 80.0%
Large Public & Private Need New LOA 84 54 30 1.8 64.3%
Large Public & Private Opt Out 52 50 2 25.0 96.2%
Regional Private Need New LOA 42 18 24 0.8 42.9%
Regional Private Opt Out 7 6 1 6.0 85.7%
Regional Public Need New LOA 37 14 23 0.6 37.8%
Regional Public Opt Out 18 17 1 17.0 94.4%
Selective Need New LOA 36 20 16 1.2 55.6%
Selective Opt Out 18 15 3 5.0 83.3%
CRP Inclusion
  1. Renewal Rate looks pretty close to the Opt Out rate, which is great.
Renewal Ratio by CRP Inclusion
GRP3-4 N ATL Dropped Ratio Perc_Renewed
0 274 172 102 1.7 62.8%
1 28 27 1 27.0 96.4%
Impact Interaction Volume Histogram
  1. Central tendency has jumped from about 4 to nearly 7.5.
  2. There is long tail with heavy utilizers.
  3. There is also a decent number of lower utilizers.

Impact Interaction Volume Histogram Split by Decision
  1. Super users definitively renew at a pretty high rate; the cut off could be pretty high for this forum.
  2. Lower utilization still has some partners who renew and from a proportion standpoint, look similar to the middle of the graph.

Impact Interaction Volume Histogram Split by Pool Date FY
  1. Many of the high bars are comprised of multiple FY decisions, which can provide some measure of comfort in making conclusions which are more general.
  2. A lot less purple, FY22, resides toward the bottom part of the graph, in the 0-4 section. 5-10 seems to hold most of FY22 mass.

Renewal by Interaction Volume and Decision Type, Unbinned
  1. [NNLOA] You don’t start to see much lift until the 10+ region. With a nice blip at 4.
  2. [Opt Out] A decent number are in the 10+ region, and there is not much to suggest that it makes a difference.
Renewal Ratio by Impact Interaction Volume
DecisionType II_Credited_Volume N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 1 2 0.5 33.3%
Need New LOA 1 5 1 4 0.2 20.0%
Need New LOA 2 12 4 8 0.5 33.3%
Need New LOA 3 16 7 9 0.8 43.8%
Need New LOA 4 22 16 6 2.7 72.7%
Need New LOA 5 17 7 10 0.7 41.2%
Need New LOA 6 16 9 7 1.3 56.2%
Need New LOA 7 22 8 14 0.6 36.4%
Need New LOA 8 20 10 10 1.0 50.0%
Need New LOA 9 16 10 6 1.7 62.5%
Need New LOA 10 17 11 6 1.8 64.7%
Need New LOA 11 5 3 2 1.5 60.0%
Need New LOA 12 10 8 2 4.0 80.0%
Need New LOA 13 3 2 1 2.0 66.7%
Need New LOA 14 4 2 2 1.0 50.0%
Need New LOA 15 3 1 2 0.5 33.3%
Need New LOA 16 4 3 1 3.0 75.0%
Need New LOA 17 1 1 0 Inf 100.0%
Need New LOA 18 2 1 1 1.0 50.0%
Need New LOA 21 1 1 0 Inf 100.0%
Need New LOA 22 1 1 0 Inf 100.0%
Need New LOA 23 1 0 1 0.0 0.0%
Need New LOA 25 1 0 1 0.0 0.0%
Opt Out 0 3 3 0 Inf 100.0%
Opt Out 1 3 3 0 Inf 100.0%
Opt Out 2 2 2 0 Inf 100.0%
Opt Out 3 5 5 0 Inf 100.0%
Opt Out 4 11 9 2 4.5 81.8%
Opt Out 5 8 6 2 3.0 75.0%
Opt Out 6 13 12 1 12.0 92.3%
Opt Out 7 10 10 0 Inf 100.0%
Opt Out 8 11 10 1 10.0 90.9%
Opt Out 9 4 4 0 Inf 100.0%
Opt Out 10 4 4 0 Inf 100.0%
Opt Out 11 6 5 1 5.0 83.3%
Opt Out 12 4 4 0 Inf 100.0%
Opt Out 13 2 2 0 Inf 100.0%
Opt Out 14 2 1 1 1.0 50.0%
Opt Out 15 5 5 0 Inf 100.0%
Opt Out 16 3 3 0 Inf 100.0%
Opt Out 17 1 1 0 Inf 100.0%
Opt Out 18 1 1 0 Inf 100.0%
Opt Out 19 1 1 0 Inf 100.0%
Opt Out 24 1 1 0 Inf 100.0%
Renewal by Impact Interaction Volume and Decision Type, (Binned)

Adjusted the binning to better capture the high volume present in AAF.

  1. For NNLOA, increased impact interaction volume does tend to be associated with increased likelihood to renew.
  2. For NNLOA, 4+ is where this binning suggests there is a cutoff.
  3. For Opt Out, most of the mass is in high utilization and therefore the drops are as well.
Renewal Ratio by Impact Interaction Volume (Binned)
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 3 1 2 0.5 33.3%
Need New LOA 1 5 1 4 0.2 20.0%
Need New LOA 2-3 28 11 17 0.6 39.3%
Need New LOA 4-6 55 32 23 1.4 58.2%
Need New LOA 7+ 111 62 49 1.3 55.9%
Opt Out 0 3 3 0 Inf 100.0%
Opt Out 1 3 3 0 Inf 100.0%
Opt Out 2-3 7 7 0 Inf 100.0%
Opt Out 4-6 32 27 5 5.4 84.4%
Opt Out 7+ 55 52 3 17.3 94.5%

Renewal by Impact Interaction Volume and Account Segment, (Binned)

  1. 4+ cutoff driven a lot by Large P&P and Regional Publics.
  2. For Regional Private and Selectives, 7+ consumption seems to be the point at which renewal improves.
Renewal Ratio by Impact Interaction Volume (Binned)
AccountSegment II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Canada 2-3 1 1 0 Inf 100.0%
Canada 4-6 4 3 1 3.0 75.0%
Canada 7+ 3 1 2 0.5 33.3%
Large Public & Private 1 5 3 2 1.5 60.0%
Large Public & Private 2-3 15 10 5 2.0 66.7%
Large Public & Private 4-6 40 32 8 4.0 80.0%
Large Public & Private 7+ 76 59 17 3.5 77.6%
Regional Private 2-3 11 5 6 0.8 45.5%
Regional Private 4-6 11 4 7 0.6 36.4%
Regional Private 7+ 27 15 12 1.2 55.6%
Regional Public 0 4 3 1 3.0 75.0%
Regional Public 1 3 1 2 0.5 33.3%
Regional Public 2-3 5 1 4 0.2 20.0%
Regional Public 4-6 21 15 6 2.5 71.4%
Regional Public 7+ 22 11 11 1.0 50.0%
Selective 0 2 1 1 1.0 50.0%
Selective 2-3 3 1 2 0.5 33.3%
Selective 4-6 11 5 6 0.8 45.5%
Selective 7+ 38 28 10 2.8 73.7%

Regional Private Renewal by Impact Interaction Volume (Binned) & Decision Type

  1. Seems to confirm that utilization does not drive renewal for this segment. 7+ was masking Opt Outs.
Regional Private Renewal Ratio by Decision Type & Impact Interaction Volume (Binned)
DecisionType II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
Need New LOA 2-3 11 5 6 0.8 45.5%
Need New LOA 4-6 10 4 6 0.7 40.0%
Need New LOA 7+ 21 9 12 0.8 42.9%
Opt Out 4-6 1 0 1 0.0 0.0%
Opt Out 7+ 6 6 0 Inf 100.0%

Regional Private Renewal Ratio by Impact Interaction Volume (Binned) & Opex

  1. No real pattern to suggest there are differences across opex bins.
Regional Private Renewal Ratio by Impact Interaction Volume (Binned) & Opex
OpEx_Bins_Private II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
0 to 38M 2-3 2 1 1 1.0 50.0%
0 to 38M 4-6 4 2 2 1.0 50.0%
0 to 38M 7+ 6 3 3 1.0 50.0%
38M to 71M 2-3 4 4 0 Inf 100.0%
38M to 71M 4-6 3 0 3 0.0 0.0%
38M to 71M 7+ 6 4 2 2.0 66.7%
71M to 127M 2-3 5 0 5 0.0 0.0%
71M to 127M 4-6 3 2 1 2.0 66.7%
71M to 127M 7+ 10 4 6 0.7 40.0%
127M+ 4-6 1 0 1 0.0 0.0%
127M+ 7+ 5 4 1 4.0 80.0%

Regional Public Renewal Ratio by Impact Interaction Volume(Binned) & Opex

  1. A lot of the action is in the highest OpEx band. Not much of a pattern here.
  2. Lower two bands of OpEx are problematic.
Regional Public Renewal Ratio by Impact Interaction Volume(Binned) & Opex
OpEx_Bins_Public II_Volume_Bins N ATL Dropped Ratio Perc_Renewed
0 to 50M 0 1 0 1 0.0 0.0%
0 to 50M 4-6 1 0 1 0.0 0.0%
0 to 50M 7+ 1 0 1 0.0 0.0%
50M to 90M 4-6 1 0 1 0.0 0.0%
50M to 90M 7+ 3 0 3 0.0 0.0%
90M to 131M 0 1 1 0 Inf 100.0%
90M to 131M 1 2 0 2 0.0 0.0%
90M to 131M 4-6 4 4 0 Inf 100.0%
90M to 131M 7+ 2 1 1 1.0 50.0%
131M+ 0 1 1 0 Inf 100.0%
131M+ 1 1 1 0 Inf 100.0%
131M+ 2-3 5 1 4 0.2 20.0%
131M+ 4-6 15 11 4 2.8 73.3%
131M+ 7+ 16 10 6 1.7 62.5%
NA 0 1 1 0 Inf 100.0%

#####Simple Regression for Impact Interaction Volume

  1. The Estimate for Impact Interaction Volume is Positive -> increased consumption means increased likelihood to renew.
  2. This is borderline statistically significant.
  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 1.30 0.81 – 2.09 0.274
II Credited Volume 1.05 1.00 – 1.12 0.066
Observations 302
R2 Tjur 0.012
Penetration by Event Grouping
  1. Expert Calls, Service, and Market Insights are the highest penetration.
  2. Not many large differences between NNLOA and Opt Outs - Expert Calls may be the highest.

Reminder: 1. Experience = VIP Days & Collaboratives 2. Service = Benchmarking, Audits, Secret Shopper, Customer Research Projects, Market Insights, and Survey Walkthroughs 3. Expert Call = Expert Center, Benchmarking Walkthrough, Tool’s Walkthrough 4. SL Led = Research SL Impact, Evergreen Action, Exec Update, National Meeting Pre-wire 5. Research Interview = Research Initial & Scoping 6. Events = National Meeting, COVID Working Group 7. PLW/Onsite = Onsite, PLW

Percent of Renewals w/ At Least 1 Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall Perc_MrktInsights
Need New LOA 52.5% 82.7% 44.1% 40.6% 25.7% 0.0% 73.3% 79.2%
Opt Out 57.0% 83.0% 45.0% 47.0% 21.0% 1.0% 64.0% 77.0%
Renewal by Event Consumption
  1. [NNLOA] Compelling evidence for the value of events, although not attending just happens to be much worse than the average.
  2. [Opt Out] Even here, with super high renewal, there seems to be marginal benefit.
Renewal Ratio by Event Consumption
DecisionType II_Events_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 96 46 50 0.9 47.9%
Need New LOA 1 106 61 45 1.4 57.5%
Opt Out 0 43 39 4 9.8 90.7%
Opt Out 1 57 53 4 13.2 93.0%
Renewal by Event Consumption Volume
  1. You see drops at all ends of the spectrum.
  2. Some Institutions are using events heavily in the final year, without obvious benefit.

Renewal by Event, OT

  1. For those not attending events, FY21 and FY22 show increases in performance form FY20.
  2. Attending an event has mixed results, with a serious drop in FY21, and outstanding performance in FY22.
Renewal Ratio by Event Consumption
II_Events_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
0 2019 19 13 6 2.2 68.4%
0 2020 68 34 34 1.0 50.0%
0 2021 30 21 9 2.3 70.0%
0 2022 22 17 5 3.4 77.3%
1 2019 19 12 7 1.7 63.2%
1 2020 44 34 10 3.4 77.3%
1 2021 68 42 26 1.6 61.8%
1 2022 32 26 6 4.3 81.2%
Renewal by Service Consumption
  1. [NNLOA] Very similar pattern to Events. Consuming is just above average while not consuming puts folks well below.
  2. [Opt Out] Marginal bump, with most of the Opt Outs using this service.
Renewal Ratio by Service Consumption
DecisionType II_Service_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 35 15 20 0.8 42.9%
Need New LOA 1 167 92 75 1.2 55.1%
Opt Out 0 17 15 2 7.5 88.2%
Opt Out 1 83 77 6 12.8 92.8%
Renewal by Service Consumption Volume
  1. A lot of consumption, with drops across the spectrum.

Renewal by Service Consumption, OT

  1. Steady rates for both decision types before FY22, with FY22 being a positive outlier.
Renewal Ratio by Service Consumption
II_Service_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
0 2019 4 2 2 1.0 50.0%
0 2020 20 10 10 1.0 50.0%
0 2021 17 9 8 1.1 52.9%
0 2022 11 9 2 4.5 81.8%
1 2019 34 23 11 2.1 67.6%
1 2020 92 58 34 1.7 63.0%
1 2021 81 54 27 2.0 66.7%
1 2022 43 34 9 3.8 79.1%
Renewal by SL Consumption
  1. [NNLOA] Smallest difference thus far. Unlikely to be meaningful as the percent renewed is so close to the average for both.
  2. [Opt Out] Decent sized positive effect, given the other opt out differences..
Renewal Ratio by SL-Led Consumption
DecisionType II_SLLed_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 113 58 55 1.1 51.3%
Need New LOA 1 89 49 40 1.2 55.1%
Opt Out 0 55 49 6 8.2 89.1%
Opt Out 1 45 43 2 21.5 95.6%

SL Consumption Volume & Renewal

  1. See relatively high volumes of SL Led interactions.
  2. Drops scattered throughout.

Renewal by SL Consumption, OT
  1. FY22 has shown the most benefit from these types of interactions - potential small upward trend.
Renewal Ratio by SL-Led Consumption
II_SLLed_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
0 2019 18 12 6 2.0 66.7%
0 2020 49 28 21 1.3 57.1%
0 2021 61 35 26 1.3 57.4%
0 2022 40 32 8 4.0 80.0%
1 2019 20 13 7 1.9 65.0%
1 2020 63 40 23 1.7 63.5%
1 2021 37 28 9 3.1 75.7%
1 2022 14 11 3 3.7 78.6%
Renewal by Research Interview Consumption
  1. [NNLOA] Similar to SL Led. Unlikely to be of statistical significance.
  2. [Opt Out] Marginal bump for Opt Outs, same degree as NNLOA.
Renewal Ratio by Research Interview Consumption
DecisionType II_ResearchInterview_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 120 62 58 1.1 51.7%
Need New LOA 1 82 45 37 1.2 54.9%
Opt Out 0 53 48 5 9.6 90.6%
Opt Out 1 47 44 3 14.7 93.6%
Research Interview Volume & Renewal
  1. Lots of high consumption in this impact interaction.

Renewal By Research Interview Consumption, OT
  1. Seeing an increasing renewal trend if this is not consumed.
  2. Until FY22, we were seeing a strong downward trend for Research Interviews.
Renewal Ratio by Research Interview Consumption
II_ResearchInterview_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
0 2019 12 6 6 1.0 50.0%
0 2020 38 20 18 1.1 52.6%
0 2021 89 58 31 1.9 65.2%
0 2022 34 26 8 3.2 76.5%
1 2019 26 19 7 2.7 73.1%
1 2020 74 48 26 1.8 64.9%
1 2021 9 5 4 1.2 55.6%
1 2022 20 17 3 5.7 85.0%
Renewal by PLW OR Onsite Consumption
  1. [NNLOA] Showing some lift above the average; not consuming is very close to the average. Not much benefit.
  2. [Opt Out] Inversion here!
Renewal Ratio by PLW OR Onsite Consumption
DecisionType II_PLW_Onsite_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 150 78 72 1.1 52.0%
Need New LOA 1 52 29 23 1.3 55.8%
Opt Out 0 79 73 6 12.2 92.4%
Opt Out 1 21 19 2 9.5 90.5%
PLW / Onsite Volume & Renewal
  1. No institution gets these at a large volume. Decent sized chunk at 1, more than we saw with AAF.

Renewal by PLW/Onsite OT
  1. Increasing Renewal trend for consumption.
  2. Not consuming has a peak in FY22, although this year might just be healthier, and there was a downward trend before it.
Renewal Ratio by PLW OR Onsite Consumption
II_PLW_Onsite_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
0 2019 25 18 7 2.6 72.0%
0 2020 83 50 33 1.5 60.2%
0 2021 79 49 30 1.6 62.0%
0 2022 42 34 8 4.2 81.0%
1 2019 13 7 6 1.2 53.8%
1 2020 29 18 11 1.6 62.1%
1 2021 19 14 5 2.8 73.7%
1 2022 12 9 3 3.0 75.0%
Renewal by Experience Consumption

Basically no usage.

Renewal Ratio by Experience Consumption
DecisionType II_Experience_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 202 107 95 1.1 53.0%
Opt Out 0 99 91 8 11.4 91.9%
Opt Out 1 1 1 0 Inf 100.0%
Experience Volume & Renewal

No Usage.

Renewal by Experience Consumption OT

No Usage.

Renewal Ratio by Experience Consumption
II_Experience_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
0 2019 38 25 13 1.9 65.8%
0 2020 112 68 44 1.5 60.7%
0 2021 97 62 35 1.8 63.9%
0 2022 54 43 11 3.9 79.6%
1 2021 1 1 0 Inf 100.0%
Renewal by Expert Call Consumption
  1. [NNLOA] First interaction where we’ve seen a NNLOA inversion. Still centered pretty closely to the mean, so no cause for concern. More evidence here would suggest unhealthy partners are more likely to use this type of interaction.
  2. [Opt Out] Inversion Here. Again, small number of drops all in the utilization bucket.
Renewal Ratio by Expert Call Consumption
DecisionType II_ExpertCall_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 54 31 23 1.3 57.4%
Need New LOA 1 148 76 72 1.1 51.4%
Opt Out 0 36 36 0 Inf 100.0%
Opt Out 1 64 56 8 7.0 87.5%
Expert Call Volume & Renewal
  1. This is another area where we are seeing a lot of impact interaction volume.
  2. You see drops across the board.

Renewal by Expert Call Consumption OT
  1. FY22 is the first year that it flips, every other year not consuming has higher renewal than consuming. FY21 is a strong data point supporting this insight.
Renewal Ratio by Expert Call Consumption
II_ExpertCall_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
0 2019 8 6 2 3.0 75.0%
0 2020 40 26 14 1.9 65.0%
0 2021 26 23 3 7.7 88.5%
0 2022 16 12 4 3.0 75.0%
1 2019 30 19 11 1.7 63.3%
1 2020 72 42 30 1.4 58.3%
1 2021 72 40 32 1.2 55.6%
1 2022 38 31 7 4.4 81.6%
Renewal by Market Insights Consumption
  1. [NNLOA] Similar to Service, not much lift above the mean, but a decent drop below if not consumed.
  2. [Opt Out] Practically no difference between consumption and not.
Renewal Ratio by Market Insights Consumption
DecisionType II_MarketResearch_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 42 19 23 0.8 45.2%
Need New LOA 1 160 88 72 1.2 55.0%
Opt Out 0 23 21 2 10.5 91.3%
Opt Out 1 77 71 6 11.8 92.2%
Market Insights by Opex Bins (Public)
  1. Looks like below the 130M there are a lot of drops even when market insights is consumed.
Renewal Ratio by Market Insights Consumption & Opex
OpEx_Bins_Public II_MarketResearch_Binary N ATL Dropped Ratio Perc_Renewed
0 to 50M 0 7 2 5 0.4 28.6%
0 to 50M 1 12 6 6 1.0 50.0%
50M to 90M 0 2 0 2 0.0 0.0%
50M to 90M 1 18 5 13 0.4 27.8%
90M to 131M 0 4 1 3 0.3 25.0%
90M to 131M 1 13 6 7 0.9 46.2%
131M+ 0 29 16 13 1.2 55.2%
131M+ 1 115 70 45 1.6 60.9%
NA 1 2 1 1 1.0 50.0%
Market Insights Volume Distribution
  1. Heavy consumption.
  2. Again, drops scattered at all parts of the distribution.

##### Market Insights Volume By Opex Band

  1. 131M+ is the highest presence across the distribution.
  2. Unlikely to see the smallest schools as super users.

##### Renewal by Market Insights Over Time

  1. With the exception of FY22, relatively consistent performance if utilized.
Renewal Ratio by Market Insights Consumption
II_MarketResearch_Binary FiscalYear N ATL Dropped Ratio Perc_Renewed
0 2019 6 4 2 2.0 66.7%
0 2020 26 14 12 1.2 53.8%
0 2021 20 11 9 1.2 55.0%
0 2022 13 11 2 5.5 84.6%
1 2019 32 21 11 1.9 65.6%
1 2020 86 54 32 1.7 62.8%
1 2021 78 52 26 2.0 66.7%
1 2022 41 32 9 3.6 78.0%

Finer Grain Look at What Was Consumed

Event Category and Event Purpose

  1. Some of the national meeting series have had strong performance - Resizing / Designing Auld Degree…
  2. Research - Blueprint for growth, crossing the chasm, both showed solid results.
  3. Benchmarking survey.
Renewal Ratio by Event Category and Event Purpose
Event_Category EventPurpose N ATL Dropped Ratio Perc_Renewed
Benchmark COE Org Benchmarking Survey COEB2012-2016 28 19 9 2.1 67.9%
Benchmarking Walkthrough Benchmarking Walkthrough 2 1 1 1.0 50.0%
COVID Working Group Session PAE COVID-19 Working Group - Responding to Impact on Marketing and Recruitment 5 1 4 0.2 20.0%
COVID Working Group Session PAE COVID-19 Working Group - Supporting Transition to Remote Instruction 7 5 2 2.5 71.4%
Diagnostic or Audit COE Org Benchmarking Survey COEB2012-2016 4 3 1 3.0 75.0%
Diagnostic or Audit Industry Futures: MBA 1 1 0 Inf 100.0%
Evergreen Action Evergreen Action 18 9 9 1.0 50.0%
Executive Update Executive Update 6 6 0 Inf 100.0%
Expert Center Expert Center 129 82 53 1.5 63.6%
Expert Center Expert Center Call 74 43 32 1.3 58.1%
Market Research NA 181 116 76 1.5 64.1%
National Meeting 2017-2018 COE Executive Forum, Baltimore, MD 2 1 1 1.0 50.0%
National Meeting 2017-2018 COE Executive Forum, Marina del Rey, CA 2 1 1 1.0 50.0%
National Meeting 2017-2018 COE Executive Forum, Washington, DC 3 3 0 Inf 100.0%
National Meeting 2018 COE Special Session, Chicago, IL 2 2 0 Inf 100.0%
National Meeting 2019 COE Forum Executive Summit, 6 5 1 5.0 83.3%
National Meeting 2019 COE Forum Executive Summit, Chicago, IL 6 4 2 2.0 66.7%
National Meeting 2019 COE Forum Executive Summit, Santa Monica, CA 15 13 2 6.5 86.7%
National Meeting 2019 COE Forum Executive Summit, Washington, DC 32 22 10 2.2 68.8%
National Meeting 2020 Professional & Adult Education Forum National Meeting Series, 49 29 21 1.4 59.2%
National Meeting Designing Programs for the Adult Degree Completer, 14 11 3 3.7 78.6%
National Meeting Digital Revolution Series, 5 4 1 4.0 80.0%
National Meeting FinTech Series, 7 6 1 6.0 85.7%
National Meeting PAE Executive Pop-Ups, 14 10 4 2.5 71.4%
National Meeting PAE Virtual Working Group: Program Development to Aid the Pandemic Economic Recovery, 16 10 6 1.7 62.5%
National Meeting Professional and Adult Education Forum: Certificate Launches Virtual Workshop, 4 3 1 3.0 75.0%
National Meeting Program Design Principles for Adult Degree Completion, 5 3 2 1.5 60.0%
National Meeting Re-sizing the Adult Degree Completion Market: A Roundtable for Professional and Adult Education Lead, 37 29 8 3.6 78.4%
National Meeting Revisiting Marketing After a Year of Remote Operations, 6 4 2 2.0 66.7%
National Meeting Smart Manufacturing Series, 6 3 3 1.0 50.0%
National Meeting Virtual Working Session PAE: Program Development and Planning During COVID19, 5 2 3 0.7 40.0%
National Meeting Virtual Working Session PAE: Responding to COVID19’s Impact on Marketing and Recruitement, 6 2 4 0.5 33.3%
National Meeting Virtual Working Session PAE: Supporting Transition to Remote Instruction, 7 5 2 2.5 71.4%
National Meeting Working Session for Public R1 Professional and Adult Education Leaders, 5 3 2 1.5 60.0%
National Meeting Pre-Wire National Meeting Pre-Wire 6 4 2 2.0 66.7%
Onsite Deliver Undergraduate Opportunity Analysis and Graduate Portfolio Diagnostic 1 0 1 0.0 0.0%
Onsite Onsite | AF| Presentation Date: 12/11/2018 1 0 1 0.0 0.0%
Onsite Onsite | COE | 04/11/2019 1 0 1 0.0 0.0%
Onsite Onsite | COE | 05/09/2019 1 1 0 Inf 100.0%
Onsite Onsite | COE | 09/04/2019 1 1 0 Inf 100.0%
Onsite Onsite | COE | 3/12/2018 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation Date: 02/28/2019 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation Date: 04/01/2019 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation Date: 05/14/2019 1 0 1 0.0 0.0%
Onsite Onsite | COE | Presentation Date: 06/04/2019 1 0 1 0.0 0.0%
Onsite Onsite | COE | Presentation Date: 06/19/2019 1 0 1 0.0 0.0%
Onsite Onsite | COE | Presentation Date: 07/16/2019 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation Date: 10/23/2019 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation Date: 10/25/2019 1 0 1 0.0 0.0%
Onsite Onsite | COE | Presentation Date: 10/9/2018 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation Date: 11/13/2018 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation date: 3/12/2018 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation Date: 3/28/2019 1 0 1 0.0 0.0%
Onsite Onsite | COE | Presentation Date: 7/2/2019 1 1 0 Inf 100.0%
Onsite Onsite | COE | Presentation Date: 8/23/2018 1 0 1 0.0 0.0%
Onsite Onsite | COEF 1 0 1 0.0 0.0%
Onsite Onsite | COEF | 07/24/18 1 0 1 0.0 0.0%
Onsite Onsite | COEF | 08/03/18 1 1 0 Inf 100.0%
Onsite Onsite | COEF | 12/11/2018 1 0 1 0.0 0.0%
Onsite Onsite | HESF | Presentation Date: 07/23/2019 1 1 0 Inf 100.0%
Onsite Onsite | Marketing Across the Program Lifecycle| 11/15 1 1 0 Inf 100.0%
Onsite Onsite | State of the Union: New Rules of Competition 1 1 0 Inf 100.0%
Onsite Onsite l COE l 1/15/18 1 1 0 Inf 100.0%
Onsite Onsite l COE l Presentation date: 1/10/2019 1 0 1 0.0 0.0%
Onsite Onsite l COE l Presentation date: 10/4/18 1 0 1 0.0 0.0%
Onsite Onsite l COE l Presentation date: 12/4/18 1 1 0 Inf 100.0%
Onsite Onsite Marketing Across the Program Lifecycle 3/3 1 0 1 0.0 0.0%
Onsite Onsite: *Customized Presentation - PAE - 3/5/2020 1 1 0 Inf 100.0%
Onsite Onsite: COE | Presentation Date: 6/5/2018 1 0 1 0.0 0.0%
Onsite Onsite: Higher Ed in the New Learning Economy - PAE - 9/18/2019 1 1 0 Inf 100.0%
Onsite UC Berkeley COEF - Credentials for an Unpredictable Market 1 1 0 Inf 100.0%
Onsite UC Berkeley COEF customized onsite | adult learner market and trends 1 0 1 0.0 0.0%
Onsite NA 7 6 1 6.0 85.7%
Research Initial 360 Degree Program Scalability Analysis for the ELOC Program 1 0 1 0.0 0.0%
Research Initial Adult Degree Completion Implementation Research 2 2 0 Inf 100.0%
Research Initial Blueprint for Growth 20 15 5 3.0 75.0%
Research Initial Central Washington University - Agribusiness Certificate 1 1 0 Inf 100.0%
Research Initial Clemson University - PFS - Environmental Engineering 5 4 1 4.0 80.0%
Research Initial COE Org Benchmarking Survey COEB2012-2016 1 1 0 Inf 100.0%
Research Initial Competitor Profiles for Master’s-Level Applied Science Programs 2 2 0 Inf 100.0%
Research Initial Considerations for a Post-Baccalaureate Computer Science Certificate Program 5 4 1 4.0 80.0%
Research Initial Considerations for an MSED in Differentiated Instruction Program 1 1 0 Inf 100.0%
Research Initial Considerations for the Bachelor of Science in Product Design 3 1 2 0.5 33.3%
Research Initial Crossing the Chasm 29 23 7 3.3 79.3%
Research Initial Dean Growth Strategy 11 8 3 2.7 72.7%
Research Initial Dominican University - Master’s-Level Professional Writing or Cross Cultural Stu 1 1 0 Inf 100.0%
Research Initial Employer Demand and Competitive Landscape of Master’s-Level Dietetics Programs 2 1 1 1.0 50.0%
Research Initial Employer Demand for an Online GIS Graduate Certificate 5 4 1 4.0 80.0%
Research Initial Future of the Hybrid Campus 1 1 0 Inf 100.0%
Research Initial Industry Futures 5 4 1 4.0 80.0%
Research Initial Industry Futures 2021 5 4 1 4.0 80.0%
Research Initial Institutional Reckoning 2 2 0 Inf 100.0%
Research Initial James Madison University: Civic Leadership Certificate 1 1 0 Inf 100.0%
Research Initial Kansas State University Environmental Communication 2 2 0 Inf 100.0%
Research Initial Market Considerations for a Master’s-Level Public Administration Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Bachelor’s-Level Applied Human Movement Program 2 1 1 1.0 50.0%
Research Initial Market Demand for a Bachelor’s-Level Creative Mass Media Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Bachelor’s-Level Exercise Science Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Bachelor’s-Level Global Studies Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Bachelor’s-Level Health Care Administration Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Bachelor’s-Level Information Technology Program 2 1 1 1.0 50.0%
Research Initial Market Demand for a Bachelor’s-Level Interdisciplinary Studies Program 2 1 1 1.0 50.0%
Research Initial Market Demand for a Bachelor’s-Level LGBT Studies Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Bachelor’s-Level Marine Studies Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Bachelor’s-Level Professional and Creative Writing Program 1 0 1 0.0 0.0%
Research Initial Market Demand for a Bachelor’s Degree or Certificate Program in Librarianship 2 2 0 Inf 100.0%
Research Initial Market Demand for a Corporate Entrepreneurship Training 2 2 0 Inf 100.0%
Research Initial Market Demand for a Doctor of Design in Human-Centered Design 4 3 1 3.0 75.0%
Research Initial Market Demand for a Doctor of Education 1 1 0 Inf 100.0%
Research Initial Market Demand for a Doctor of Industrial Technology Program 1 0 1 0.0 0.0%
Research Initial Market Demand for a Doctoral-Level Educational Leadership Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Doctoral Level Occupational Therapy Program 2 2 0 Inf 100.0%
Research Initial Market Demand for a Graduate Certificate in Pharmacovigilance 1 1 0 Inf 100.0%
Research Initial Market Demand for a Hybrid Master’s-Level Spanish Journalism Program 1 0 1 0.0 0.0%
Research Initial Market Demand for a Master’s-Level Computing Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Master’s-Level Data Science Program 1 0 1 0.0 0.0%
Research Initial Market Demand for a Master’s-Level English Program 1 0 1 0.0 0.0%
Research Initial Market Demand for a Master’s-Level Food Science Program 5 3 2 1.5 60.0%
Research Initial Market Demand for a Master’s-Level Health Administration Program 2 1 1 1.0 50.0%
Research Initial Market Demand for a Master’s-Level Program Assessment and Evaluation Program 2 1 1 1.0 50.0%
Research Initial Market Demand for a Master’s-Level School Counseling Program 1 0 1 0.0 0.0%
Research Initial Market Demand for a Master’s-Level School Psychology Program 1 0 1 0.0 0.0%
Research Initial Market Demand for a Master of Science in Law 2 1 1 1.0 50.0%
Research Initial Market Demand for a Master’s-Level Physician Assistant Program 1 1 0 Inf 100.0%
Research Initial Market Demand for a Media Management Certificate Program 2 1 1 1.0 50.0%
Research Initial Market Demand for a Medical Interpretation Certificate Program 2 1 1 1.0 50.0%
Research Initial Market Demand for a Non-Credit Certificate in Digital Marketing 1 1 0 Inf 100.0%
Research Initial Market Demand for a Post-Baccalaureate Certificate in Health Leadership 1 0 1 0.0 0.0%
Research Initial Market Demand for a Post-Baccalaureate Pre-Medical Certificate Program 4 4 0 Inf 100.0%
Research Initial Market Demand for a Professional Science Master’s in Engineering 2 2 0 Inf 100.0%
Research Initial Market Demand for a Technology Entrepreneurship Graduate Certificate 1 1 0 Inf 100.0%
Research Initial Market Demand for an Accelerated Bachelor’s to Master’s Public Health Program 1 1 0 Inf 100.0%
Research Initial Market Demand for an Artificial Intelligence Graduate Certificate 1 0 1 0.0 0.0%
Research Initial Market Demand for an Online and Hybrid M.Ed. in Special Education with an Emphas 4 2 2 1.0 50.0%
Research Initial Market Demand for an Online Bachelor of Arts in Health Studies Program 1 0 1 0.0 0.0%
Research Initial Market Demand for an Online Graduate Certificate in Journalism 2 1 1 1.0 50.0%
Research Initial Market Demand for an Online Master’s-Level Landscape Architecture Program 2 2 0 Inf 100.0%
Research Initial Market Demand for an Online Master’s-Level Nutrition Program 1 1 0 Inf 100.0%
Research Initial Market Demand for an Online Master’s-Level Research Administration Program 1 1 0 Inf 100.0%
Research Initial Market Demand for an Online Master’s-Level Nursing Program 1 1 0 Inf 100.0%
Research Initial Market Demand for an Online Version of the Master of Science in Nutrition Progra 3 2 1 2.0 66.7%
Research Initial Market Demand for an Undergraduate Certificate in Public Relations 1 1 0 Inf 100.0%
Research Initial Market Demand for BA in Computer Science 2 2 0 Inf 100.0%
Research Initial Market Demand for Bachelor’s- and Master’s-Level Applied Linguistics and TESOL P 2 2 0 Inf 100.0%
Research Initial Market Demand for Bachelor’s-Level Risk, Insurance, and Safety Management 1 1 0 Inf 100.0%
Research Initial Market Demand for BFA in Animation: Webster University 1 0 1 0.0 0.0%
Research Initial Market Demand for Master’s-Level Animal Behavior Programs 2 2 0 Inf 100.0%
Research Initial Market Demand for Master’s-Level Art Therapy Program 1 1 0 Inf 100.0%
Research Initial Market Demand for Master’s-Level Environmental Policy Programs 1 1 0 Inf 100.0%
Research Initial Market Demand for Master’s-Level Medical Management Program 1 1 0 Inf 100.0%
Research Initial Market Demand for Master’s-Level Nursing Admistrati 1 1 0 Inf 100.0%
Research Initial Market Demand for Master’s-Level Public Administration and Nonprofit Management 1 1 0 Inf 100.0%
Research Initial Market Demand for Master’s-Level Public Policy Programs 3 2 1 2.0 66.7%
Research Initial Market Demand for Master’s in Supply Chain Management 1 1 0 Inf 100.0%
Research Initial Market Demand for Part-Time Master of Accounting Program 2 2 0 Inf 100.0%
Research Initial Market Demand for PhD Level Clinical Psychology Prorgams 3 2 1 2.0 66.7%
Research Initial Market Viability for Master’s-Level Health Informatics Program 2 2 0 Inf 100.0%
Research Initial Market Viability of a Bachelor’s-Level Environmental Science Program 2 2 0 Inf 100.0%
Research Initial Market Viability of a Bachelor’s-Level Kinesiology Program 2 1 1 1.0 50.0%
Research Initial Market Viability of a Bachelor’s-Level Sustainable Urban Planning and Design Pro 1 1 0 Inf 100.0%
Research Initial Market Viability of a College Admissions Counseling Certificate Program 3 2 1 2.0 66.7%
Research Initial Market Viability of a Doctoral-Level Psychology Program (PsyD) 1 0 1 0.0 0.0%
Research Initial Market Viability of a Forensic Psychology Certificate Program 1 1 0 Inf 100.0%
Research Initial Market Viability of a Graduate-Level Urban Food Systems Program 3 3 0 Inf 100.0%
Research Initial Market Viability of a Master’s- or Certificate-Level Data Analytics Program 1 1 0 Inf 100.0%
Research Initial Market Viability of a Master’s-Level Applied Statistics Program 1 1 0 Inf 100.0%
Research Initial Market Viability of a Master’s-level Chemistry Program 1 1 0 Inf 100.0%
Research Initial Market Viability of a Master’s Applied Chemical Science and Technology Program 2 1 1 1.0 50.0%
Research Initial Market Viability of a Master’s-Level Pharmacology and Drug Development Program 1 1 0 Inf 100.0%
Research Initial Market Viability of a Non-Credit Creative Executive Production Certificate 1 0 1 0.0 0.0%
Research Initial Market Viability of an Online Bachelor of Arts General Program 1 1 0 Inf 100.0%
Research Initial Market Viability of an Online Bachelor’s-Level Business Economics Program 2 0 2 0.0 0.0%
Research Initial Market Viability of an Online Master’s-Level Music Therapy Program 1 1 0 Inf 100.0%
Research Initial Market Viability of an Online Master’s-Level Regulatory Science Program 1 1 0 Inf 100.0%
Research Initial Market Viability of Online ABET-Accredited Engineering or Comp Sci Program 2 2 0 Inf 100.0%
Research Initial Market Viability of Self-Paced and Enroll Anytime Courses 1 1 0 Inf 100.0%
Research Initial Master’s-Level early childhood deaf education and linguistics 2 0 2 0.0 0.0%
Research Initial Online Faculty Compensation 1 1 0 Inf 100.0%
Research Initial PAE 2020 National Meeting Series 2 1 1 1.0 50.0%
Research Initial PAE 2021 Agenda Setting 1 1 0 Inf 100.0%
Research Initial PAE Nursing 2021 1 1 0 Inf 100.0%
Research Initial Portland State University Master’s in International and Global Studies 1 1 0 Inf 100.0%
Research Initial Principles of Program Design 3 3 0 Inf 100.0%
Research Initial Royal Roads University - Master’s-Level Intercultural Teaching and Learning 1 1 0 Inf 100.0%
Research Initial Self-Paced Online Courses 1 0 1 0.0 0.0%
Research Initial Single Program Analysis of Master of Anesthesiologist Assistant 1 1 0 Inf 100.0%
Research Initial Single Program Analysis Speech Pathology 2 1 1 1.0 50.0%
Research Initial St. Mary’s College of California - Biotech MS 2 2 0 Inf 100.0%
Research Initial UC Santa Cruz - Creative Technologies SPA 2 0 2 0.0 0.0%
Research Initial University of Central Florida - Hospitality Management 1 0 1 0.0 0.0%
Research Initial University of Central Florida - PFS - Data Analytics 1 1 0 Inf 100.0%
Research Initial University of Massachusetts-Amherst - Master’s-Level Accounting Transitions 3 1 2 0.5 33.3%
Research Initial University of North Texas - Engineering Management 5 2 3 0.7 40.0%
Research Initial University of North Texas Executive Hospitality Administraiton 9 7 2 3.5 77.8%
Research Initial University of Wyoming GIS 1 0 1 0.0 0.0%
Research Initial Veteran research 1 0 1 0.0 0.0%
Research Initial Webster University: STEM Ed Certificate 1 0 1 0.0 0.0%
Research Initial Wichita State University-BA and MA in Spanish 2 1 1 1.0 50.0%
Research SL Impact Research SL Impact - (PAE) Coffee w/ Flavius Lilly 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Bachelor’s in Data Science Report Delivery 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Bi-Monthly Strategy Conversation 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - cabinet overview - focus on market research 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Call 4 3 1 3.0 75.0%
Research SL Impact Research SL Impact - Call - Clarkson - PAE - Colleen Thapalia 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Call - graduate programs/strategy 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Call - online program growth 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Call - UCLA - PAEF - Kamina Smith 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Call - UNC Chapel Hill - PAEF - Jack Rodenfels 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Call with Ellen & Kara to discuss PAE Membership 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Call with Excelsior College 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Campus Visit 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Check-in 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Check In 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - CNHS Degree Completion conversation 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - COE Action Planning 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - COE Forum Discussion 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - CSU LB PAE Web Audit Debrief | Call with Anna Behar-Russell 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - DSU market research PA 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Enrollment State of the Union 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - ERAU WW COB Meeting Part 2 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - ERAU WW COB Meeting Part 3 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Evening Program Pilot Discussion 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Expert Brainstorming Call 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Expert Call 10 8 2 4.0 80.0%
Research SL Impact Research SL Impact - Follow-up Call 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - FSC Impact Call 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Impact on Org Structures 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Impact Visit on campus 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - In-Person Meeting 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - In Person Lunch Meeting with Manohar Singh 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - in person onsite and visit 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - In Person Visit 2 2 0 Inf 100.0%
Research SL Impact Research SL Impact - in person visit + dinner 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - INT Org Benchmarking Submission 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - LRP Check-in 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - market insights completion and walkthrough 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Market Insights Walkthrough 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Market Opp Scan Meeting 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Mass Art COE PA w SME 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Meeting 10 7 3 2.3 70.0%
Research SL Impact Research SL Impact - Meeting - Baylor - PAE - Sandy Bennett 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Meeting - UC Berkeley - PAE - Carol Sirney 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Meeting - UC Berkeley - PAE - Ramu Nagappan 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Meeting with ERAU WW COB 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Monthly Check-in 2 1 1 1.0 50.0%
Research SL Impact Research SL Impact - Muhlenberg Market Insights Call 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Networking Call for Justin with IU 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Organizational Benchmarking Tools Walkthrough 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - PA - Carnegie Mellon - PAEF - expert call 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - PAE Check-In 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - PAE Check In 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - PAE Executive Update 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - PAE Forum Overview 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - PAE IPV 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - PAE Onsite Presentation with David Vuletich 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - PAEF Overview and Discussion 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - PAEF Planning Discussion 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Philanthropy Market Insights Report Call 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Quarterly Call 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Research call with Justin Louder 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Research Call with Online Education Team 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Research Call with Stephen Turner 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Review of URM Enrollment Resources 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Second Exec Roundtable Topic 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - SF PAE grad school dean meeting 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - SF UCM PAE 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - SF UCM PAE SL Impact 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Shelby Newport In-Person Visit and Research Walkthrough 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Student of the Future Seminar 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - UM Flint, PAE (COE) - Stephen Turner, Associate Provost - In person visit 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - UNLV PAE | SL Impact Interaction with Joseph Miera 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - UNLV PAE Call with Joe, Heidi, and Tami 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Walk Through of Blue Print for Growth 1 1 0 Inf 100.0%
Research SL Impact Research SL Impact - Web & Mobile Walkthrough 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact - Working Groups Campus Meeting 1 0 1 0.0 0.0%
Research SL Impact Research SL Impact Visit 48 30 18 1.7 62.5%
Scoping Blueprint for Growth 4 3 1 3.0 75.0%
Scoping Blueprint for Growth: Future of Low-Cost and Free 1 1 0 Inf 100.0%
Scoping Industry Futures 2021 2 2 0 Inf 100.0%
Scoping PAE 2021 Agenda Setting 4 3 1 3.0 75.0%
Scoping PAE 2022 Agenda Setting 2 2 0 Inf 100.0%
Scoping University of Central Florida - PFS - Data Analytics 1 1 0 Inf 100.0%
Tool’s Walkthrough Tool’s Walk Through 23 16 7 2.3 69.6%
Tool’s Walkthrough Tool’s Walk Through Call 3 2 1 2.0 66.7%
VIP Day West Virginia University Virtual Impact Workshops 1 1 0 Inf 100.0%
Web & Moble Audit 1 Page 1 1 0 Inf 100.0%
Web & Moble Audit 1 page (MPA Program Landing Page) 1 1 0 Inf 100.0%
Web & Moble Audit 2 pages 3 3 0 Inf 100.0%
Web & Moble Audit 2 pages (Sociology Department) 1 1 0 Inf 100.0%
Web & Moble Audit 3 pages 1 1 0 Inf 100.0%
Web & Moble Audit 3 Pages 1 0 1 0.0 0.0%
Web & Moble Audit Full Inquiry Audit of BA Psych and MBA 1 0 1 0.0 0.0%
Web & Moble Audit Homepage; B.S.B.A., Global Business (General Business); Master of Science in Management 1 0 1 0.0 0.0%
Web & Moble Audit Market Research Request: Web Presence 1 1 0 Inf 100.0%
Web & Moble Audit member request for “secret shopper” website audit 1 1 0 Inf 100.0%
Web & Moble Audit Mobile Audit 1 1 0 Inf 100.0%
Web & Moble Audit Web audit 1 0 1 0.0 0.0%
Web & Moble Audit Web Audit 2 1 1 1.0 50.0%
Web & Moble Audit Web Audit (3 pages) 1 0 1 0.0 0.0%
Web & Moble Audit Web Audit Request 1 1 0 Inf 100.0%
Web & Moble Audit Web Audit Request Form 1 0 1 0.0 0.0%
Web & Moble Audit Web Presence Audit 1 1 0 Inf 100.0%
Web & Moble Audit Web Presence Audit: Non-Credit Programs (3 Pages) 1 1 0 Inf 100.0%
Web & Moble Audit Web/Mobile Audit 2 0 2 0.0 0.0%
Web & Moble Audit Website Audit 1 1 0 Inf 100.0%
Webinar Baylor PAEF | Alternative Credential Session 1 1 0 Inf 100.0%
Webinar Onsite | Credentials for an Unpredictable Market| 1/13 1 0 1 0.0 0.0%
Webinar Partner Intensive | PAE | Blueprint for Growth: Certificates 1 1 0 Inf 100.0%
Webinar Partner Intensive | PAE | Customized Presentation 1 1 0 Inf 100.0%
Webinar Partner Intensive | PAE | Growth Strategy 1 1 0 Inf 100.0%
Webinar Partner Intensive | PAE | Organizational Design 1 1 0 Inf 100.0%
Webinar PLW | COE | 04/25/2019 1 1 0 Inf 100.0%
Webinar PLW | COE | Presentation Date: 03/26/2019 1 1 0 Inf 100.0%
Webinar PLW | COEF | 07/27/2018 1 0 1 0.0 0.0%
Webinar PLW | COEF | 08/28/2018 1 1 0 Inf 100.0%
Webinar PLW | COEF | 11/19/2018 1 0 1 0.0 0.0%
Webinar PLW | COEF |11/12/2018 1 0 1 0.0 0.0%
Webinar PLW: Marketing and Recruiting Across the Adult Student Lifecycle - PAE - 4/16/2020 1 1 0 Inf 100.0%
Webinar U Calgary - PAE - Certificate/Credentials | RESCHEDULEED 3/19/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | GRP | COVID SOTU for Board | Presentation date: 10/22/20 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | *Customized Presentation 1 0 1 0.0 0.0%
Webinar Virtual Meeting | PAE | Blueprint for Growth 2 2 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Blueprint for Growth | 2/17/21 1 0 1 0.0 0.0%
Webinar Virtual Meeting | PAE | Blueprint for Growth | 3/22/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Blueprint for Growth: Adult Degree Completion | Date of Presentation: 4/06/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Certificates Innovation Workshop 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Certificates Innovation Workshop| Date of Presentation 7/14/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Customized Presentation 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Customized Presentation| Date of Presentation 07/14/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Designing for Growth: Building Your Alternative Credential Portfolio 1 0 1 0.0 0.0%
Webinar Virtual Meeting | PAE | Designing for Growth: Building Your Alternative Credential Portfolio | 2/24/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Designing for Growth: Building Your Alternative Credential Portfolio | Presentation Date: 1/7/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Designing for Growth: Building Your Alternative Credential Portfolio | Presentation date: 9/15/20 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Designing for Growth: Building Your Alternative Credential Portfolio| Date of presentation: 3/25/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Designing for Growth: Building Your Alternative Credential Portfolio| Date of Presentation: 6/24/21 1 0 1 0.0 0.0%
Webinar Virtual Meeting | PAE | Higher Ed in the New Learning Economy 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Partnering With Industry| Date of presentation 3/17/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Partnering with Industry| Date of Presentation: 5/04/21 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAE | Re-sizing the Adult Degree Completer Market 1 1 0 Inf 100.0%
Webinar Virtual Meeting | PAEF/BAF | national market for online ed 1 0 1 0.0 0.0%
Webinar Virtual Meeting: Custom Student Mental Health: SAF: University of Miami 1 1 0 Inf 100.0%
Webinar Virtual Presentation | PAE | Designing for Growth: Building Your Alternative Credential Portfolio | Presentation dates: 8/3/20 and 8/6/20 1 1 0 Inf 100.0%
Webinar Virtual Presentation: Partnering with Industry: Employer Partnership Trends and Models for Success - PAE - 8/28/2020 1 0 1 0.0 0.0%
Webinar Virtual Presentation| PAE| COVID-19 Outbreak’s Impacts Across the Professional and Adult Portfolio| 10/27 1 1 0 Inf 100.0%
Webinar NA 4 3 1 3.0 75.0%
NA NA 6 4 2 2.0 66.7%
Correlation betwen Event Groupings, and Renewal
  1. No strong correlations. It doesn’t look like consuming one type of interaction leads to others.
  2. Many small positive correlations with Renewal. No standouts.

Multivariate Regression (still kind of simple)

Note: Looking at all decision types.

  1. Nothing pops as significant.
  Target_Binary
Predictors Odds Ratios CI p
(Intercept) 1.32 0.81 – 2.16 0.261
II Events Volume 1.08 0.86 – 1.36 0.523
II SLLed Volume 1.06 0.86 – 1.33 0.577
II ResearchInterview
Volume 		1.11 0.98 – 1.29 0.120
II PLW Onsite Volume 1.05 0.70 – 1.62 0.817
II ExpertCall Volume 1.01 0.88 – 1.16 0.931
II MarketResearch Volume 1.05 0.95 – 1.17 0.324
Observations 302
R2 Tjur 0.016
Stop Here

MPOC & Contact Engagement

  1. MPOC engagement appears to matter for NNLOAs more so than it does for Opt Outs.
  2. MPOCs tend to be present for SL Led, Expert Calls, and Events. and these interactions have lift
  3. Unrealized value appears to lie in Events and Onsites. Perhaps onsites can be used surgically in this manner -> not necessarily goaled.
  4. The total number of engaged contacts is associated with higher likelihood to renew.
  5. If the MPOC is not engaged, you can try and up the overall number of engaged contacts to make up the difference.
Impact Interactions Penetration for MPOCs
  1. SL Led and Expert Calls have the highest engagement amongst MPOCs.
  2. Larger differences between NNLOA and Opt Outs, for Events and Service, is unexpected.
Percent of Renewals w/ At Least 1 MPOC Interaction in a Group
DecisionType Perc_Events Perc_Service Perc_SLLed Perc_ResearchInt Perc_PLW_Onsite Perc_Experience Perc_ExpertCall Perc_MrktInsights
Need New LOA 36.1% 38.6% 24.8% 11.9% 13.4% 0.0% 45.0% 37.6%
Opt Out 31.0% 39.0% 24.0% 14.0% 9.0% 1.0% 35.0% 34.0%
Renewal by MPOC Egnagement
  1. Doesn’t seem to matter whether the MPOC is engaged or not.
Renewal Ratio by MPOC Engagement
DecisionType II_MPOC N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 43 20 23 0.9 46.5%
Need New LOA 1 159 87 72 1.2 54.7%
Opt Out 0 34 30 4 7.5 88.2%
Opt Out 1 66 62 4 15.5 93.9%
Renewal by MPOC Volume
  1. MPOC volume does not seem to do the trick, since the ratio of Red to Blue remains pretty consistent.
Total Contact Engagement
  1. Seems to be that the more contacts engaged, the better.
Renewal Ratio by Total Contact Engagement
II_Contacts_Bins N ATL Dropped Ratio Perc_Renewed
0 6 4 2 2.0 66.7%
1 49 32 17 1.9 65.3%
2 71 38 33 1.2 53.5%
3-4 106 73 33 2.2 68.9%
5-6 41 30 11 2.7 73.2%
7+ 29 22 7 3.1 75.9%
Total Contact Engagement w/ and w/o MPOC

If you cannot get the MPOC engaged, is there a certain number of other contact which would cover the spread?

  1. The data suggests we might fare better when we are not engaged with the MPOC. Makes me wonder about the data integrity here.
Renewal Ratio by Total Contact Engagement by MPOC Engagement
II_MPOC II_Contacts_Bins N ATL Dropped Ratio Perc_Renewed
0 0 6 4 2 2.0 66.7%
0 1 16 11 5 2.2 68.8%
0 2 11 5 6 0.8 45.5%
0 3-4 23 14 9 1.6 60.9%
0 5-6 13 10 3 3.3 76.9%
0 7+ 8 6 2 3.0 75.0%
1 1 33 21 12 1.8 63.6%
1 2 60 33 27 1.2 55.0%
1 3-4 83 59 24 2.5 71.1%
1 5-6 28 20 8 2.5 71.4%
1 7+ 21 16 5 3.2 76.2%
Renewal by MPOC Engagement in Events
  1. [NNLOA] Not much of an effect here.
Renewal Ratio by MPOC Event Engagement
DecisionType II_Events_MPOC_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 129 65 64 1.0 50.4%
Need New LOA 1 73 42 31 1.4 57.5%
Opt Out 0 69 62 7 8.9 89.9%
Opt Out 1 31 30 1 30.0 96.8%
Renewal by MPOC Engagement in Service
  1. [NNLOA] Pretty large effect for Service consumption!
Renewal Ratio by MPOC Service Engagement
DecisionType II_Service_MPOC_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 124 64 60 1.1 51.6%
Need New LOA 1 78 43 35 1.2 55.1%
Opt Out 0 61 53 8 6.6 86.9%
Opt Out 1 39 39 0 Inf 100.0%
Renewal by MPOC Engagement in SL Led Impact Interactions
  1. [NNLOA] Like events, not a very large effect.
Renewal Ratio by MPOC SL Led Engagement
DecisionType II_SLLed_MPOC_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 152 82 70 1.2 53.9%
Need New LOA 1 50 25 25 1.0 50.0%
Opt Out 0 76 69 7 9.9 90.8%
Opt Out 1 24 23 1 23.0 95.8%
Renewal by MPOC Engagement in Expert Call Engagement
  1. [NNLOA] Similar to Events and SL Led, not a large effect.
Renewal Ratio by MPOC Expert Call Engagement
DecisionType II_ExpertCall_MPOC_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 111 65 46 1.4 58.6%
Need New LOA 1 91 42 49 0.9 46.2%
Opt Out 0 65 61 4 15.2 93.8%
Opt Out 1 35 31 4 7.8 88.6%
Renewal by MPOC Engagement in PLW / Onsite
  1. [NNLOA] Small lift from consuming this type of interaction.
Renewal Ratio by MPOC Onsite Engagement
DecisionType II_PLW_Onsite_MPOC_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 175 94 81 1.2 53.7%
Need New LOA 1 27 13 14 0.9 48.1%
Opt Out 0 91 83 8 10.4 91.2%
Opt Out 1 9 9 0 Inf 100.0%
Renewal by MPOC Engagement in Experience
  1. [NNLOA] N too small to make a call.
Renewal Ratio by MPOC Experience Engagement
DecisionType II_Experience_MPOC_Binary N ATL Dropped Ratio Perc_Renewed
Need New LOA 0 202 107 95 1.1 53.0%
Opt Out 0 99 91 8 11.4 91.9%
Opt Out 1 1 1 0 Inf 100.0%