The Impact of Increased Course Loads on College Performance

Course Load and Time-to-Degree

• Nationally, graduation rates are a problem (~60%)
• Also a problem: time to degree, especially at publics (5.2 years vs. 4.8 for private)
• And *especially at CSUs (4-year grad rates ~20-30%)
• Nationally we estimate cost of low graduation rates (just to students) nationally at about $41.7b per annual cohort
• Cutting TTD requires higher courseload. But is there a performance cost?

Time Demands and Performance

• Literature is of two minds on the likely effects of increasing courseload
• Time allocation model: increased load should reduce time spent on each class, reducing performance.
• Mixed evidence on this from employment, sports (Ruhm 1997, Oettinger 1999, Stinebrickner & Stinebrickner 2003, Aries et al. 2004, Kalenkoski & Pabliona 2008, Emerson et al. 2009, Darolia 2014)

Time Demands and Performance

• “Academic momentum”: Increased course load should focus students on school. Substitute away from other time demands, positive identity effects
• Support for this generally, but literature is often causally weak (Volkwein and Lorang 1996, Knight 2004, Cornwell, Lee, and Mustard 2005, Attewell, Heil, and Reisel 2012, Attewell and Monaghan 2016, Belfield, Jenkins, and Lahr 2016, Venit 2017)

Course Loads and GPA

• Evidence of course load on within-class performance (GPA) rather than mechanically-related things like time-to-degree slim or weak (Szafran 2001, Venit 2017)
• Best evidence on this is experimental but combined course load with many other interventions (Scott-Clayton 2010 on PROMISE, Scrivener et al. 2015 on CUNY)
• Is there a cost of reduced performance as a result of increased courseload?
• We use administrative longitudinal data on two incoming freshman cohorts at CSU Fullerton to attempt a causal estimate

Identification

• Identification runs into some obvious problems:
• Ability bias
• Time-varying shocks to demand on time (work)
• Bounding of course load
• Endogenous course load/course mix selection (Volkwein and Lorang 1996)

Identification

\[ GPA_{ijt} = \beta_{0i} + \beta_{1j}+\beta_2 Class_{it}+\beta_3 X_{it} + \varepsilon_{it} \]

\[ GPA_{ijt}^{STD} = \beta_{0i}+\beta_2 Class_{it}+\beta_3 X_{it} + \varepsilon_{it} \]

\[ Class_{it}= \gamma_{0i}+\gamma_{ij}+ \gamma_2 X_{it} + \nu_{it} \]

Inclusion of: time-invariant (\(\beta_{0i}, \gamma_{0i}\)) and variant (\(X_{it}\)) background characteristics, difficulty of courses (\(\beta_{1j}, \gamma_{ij}\)), courseload (\(Class_{it}\)), consumption and family shocks (\(\varepsilon_{it}, \nu_{it}\))

Identification

Fixed effects deal with \(\beta_{0i}, \gamma_{0i}\), use proxy controls to address \(\beta_{1j}, \gamma_{ij}\) as possible, effect bounding simulation to consider impact of \(\varepsilon_{it}, \nu_{it}\)

Much of within-student variation in course load at CSUF is due to bottlenecks (Moore & Tan 2018), suggesting that much within-student variation is exogenous.

Data

• Entering Freshman cohorts at CSUF in 2010 (3,874 students) and 2011 (4,141), follow up to Spring 2017
• Observe nearly 60,000 courses taken by these students after dropping summer and any non-full-time semesters
• Observe grades in each class, student background, course loads, and characteristics of other non-sample students in the same courses
• Plenty of within-student variation: about 30% of semesters, the non-modal number of courses is taken

Summary Statistics

Classes Taken

Distribution of Standardized GPA

Baseline Results

Baseline Results

• Consistent finding of no negative effect
• Effect declines as we control for background \(\rightarrow\) better students take more courses
• But even with fixed effects the effect is positive
• A slight Simpson’s paradox - effect concentrated in non-grads but lower than pooled
• Suggests differences in the ways that graduates and non-graduates respond to shocks
• What else can we learn about the endogeneity of \(Class_{it}\)?

Predicting \(Class_{it}\)

Adjusting for Bias from Time-Varying Information

And yet…

• We see little evidence that selection on observables is hiding a true negative effect
• But big hole in the data: no information on work pressure/hours
• We simulate a binary unobserved confounder \(Z_{it}\) positively correlated with \(GPA_{it}^{STD}\) and \(Class_{it}\)
• In practice, \(Correl(Z_{it},GPA_{it}^{STD})\in [0,.43], Correl(Z_{it},Class_{it})\in[0,.82]\).

Sensitivity Analysis

• Include in model, see how strong correlations must be to produce a negative coefficient on \(Class_{it}\)

\[ Z_{it} = I(.5+\delta_1 GPA_{it}^{STD}+\delta_2(I(Class_{it} \geq 5)-.5)\geq U_{it}) \]

\[ U_{it}\sim Unif[0,1], \delta_1 \in \{0,.1,...,1\}, \delta_2 \in\{0,.1,...,1\} \]

Simulation Results

Simulation Results

• It’s not too hard to produce a negative result - a (negative) correlation between omitted work pressures and standardized GPA of .1 or stronger, combined with correlations between omitted work pressures of .2 or stronger produce a statistically significant negative result 95% of the time

• But these significant results are still small

• We call a “meaningfully large” result .2 SDs of within-class GPA, or about half the difference between B and B+

• To get this we need correlations of .430 with course load and .531 with GPA - much less plausible!

Is This Just Good Students?

• OK sure, no overall big negative effect. But surely that’s a negative effect among struggling students who couldn’t handle more, and maybe neutral or positive at the top?

• We perform a quantile analysis to see the impact on grades at different parts of the distribution

Is This Just Good Students?

Recap

• Positive correlation between course load and GPA
• Positive effect of within-student variation in course load and GPA
• Test for measured confounders does not make the effect negative
• Simulation for unmeasured confounders indicates that the unmeasured confounder must be very strong
• Our interpretation of choice: the effect is non-negative (positive? maybe)

Additional Results

Controls = Race, gender, admissions, HSGPA, fin. aid, GPA

Conclusion

• We find no evidence to back up the worry that increased course loads will significantly harm performance
• Time-use implications for substitutability of study time (or maybe school just doesn’t take much time; Arum & Roksa 2011) and evidence in favor of academic momentum
• This is not just a good-student effect or a raw correlation
• Seems like you can increase loads without harming performance
• Note: this is all pre-pandemic data. Who knows how it generalizes to now