STA320 Final Exam Team 4

1 Research Question & Response Variable (Grace Parsons)

The research questions are, how do exam format, practice frequency, feedback timing, and class time structure affect student performance in an introductory statistics course, and how do these effects vary among students in different colleges. The response variable is the student’s exam grade in the introductory statistics course.

2 Factors, Levels, and Design Choice (Kevin Campoverde)

The design used was a blocked full \(2^4\) factorial design with the following four factors and levels: exam format(standard, modified), practice frequency(low, high), feedback timing (delayed, immediate), and class time structure (lecture, interactive). There are two levels of treatment per factor, producing a total of 16 treatment combinations. Three blocks are implemented because of concerns surrounding possible differences among the following colleges; Science and Math, Education, and Arts and Humanities. Randomization should be executed within each block, giving a total of 48 experimental runs. A possible restriction could arise from the difficulty of randomizing every student to the class time structure factor.

3 Analysis Results (Kevin Campoverde)

A blocked \(2^4\) factorial design was utilized to determine whether exam format, practice frequency, feedback timing, and class time structure affected student exam grades. The ANOVA results were statistically significant, \(F(15, 32) = 30.78\), \(p < .001\), indicating a significant amount of the variation in exam grades was associated with the set of factorial effects. Significant effects included the following interactions: Format x Frequency, \(F(1, 32) = 97.48\), \(p < .001\), \(\eta_p^2 \approx .753\); Format x Timing, \(F(1, 32) = 7.15\), \(p = .012\), \(\eta_p^2 \approx .183\); and Format x Timing x Structure, \(F(1, 32) = 251.85\), \(p < .001\), \(\eta_p^2 \approx .887\). The interaction plots showed nonparallel lines, supporting that the effect of one factor depends on the level of another factor. Main effects were not interpreted alone due to the significance of the interactions. The four-way interaction, Format x Frequency x Timing x Structure, was not statistically significant, \(F(1, 32) = 0.00\), \(p = .991\).

4 Power (Grace Parsons)

The post-hoc power for the four-way interaction, using eta2=0.0020559, (f ≈ 0.0454), df = 1, α = 0.05, and N = 48, was approximately 0.061 (6.07%), indicating very low power to detect this effect.

5 Residual Analysis (Joshua Xavier)

The residual histogram is a bell-shaped distribution centered near zero, supporting the normality assumption. There is a slight right skew, but it is not significant and can be ignored. The residuals vs. run order plot shows no funnel or fan shape distribution. Points are scattered randomly above and below zero throughout the entire graph. The spread is roughly consistent for all three blocks (black = Block 1, red = Block 2, green = Block 3), with no block sitting noticeably above or below the zero line, supporting assumptions of homoscedasticity. There are a couple of points near ±6 but no extreme outliers. Based on the analysis of the residual plots, the assumptions of normality and homoscedasticity are met.

6 Factorial Regularities (Grace Parsons)

The results display sparsity because only a small number of effects are statistically significant compared to the total number of possible effects in the 2^4 factorial. Most of the higher-order interactions are not significant, but a few effects, including exam format, practice frequency, exam format x practice frequency, and exam format x feedback timing x class time structure, were found to be significant. The hereditary principle is partially supported. The significantly higher order interaction, exam format x feedback timing x class time structure (\(p\) = 2.2 \(\times\) 10^-16) involves factors that also show an effect at lower levels. Exam format has a strong main effect (\(p\) = 3.217 \(\times\) 10^-9), and feedback timing and class time structure appear in lower-order interactions, even though they are not significant. This suggests that the important higher-order effect is built from factors that show some influence individually, or in smaller combinations, which is consistent with the heredity principle. The results from the ANOVA follow hierarchy because the strongest effects appear at lower orders and immediate orders, such as exam format, practice frequency, exam format x practice frequency, while the higher order interactions are less common and more selective. However, the exam format x feedback timing x class time structure interaction is extremely strong (\(p\) = 2.2 \(\times\) 10^-16), which shows that hierarchy is not extremely strict in this system. Overall, the experiment demonstrates clear sparsity, partial heredity, and mostly follows hierarchy, with one higher-order interaction breaking the strict hierarchy.

7 Limitations & What You’d Do Next (Joshua Xavier)

This design only tests each factor at two levels (High/Low), so it can only detect linear effects. Any curvature in the relationship between a factor and exam performance would go undetected.

The blocking assumes treatment effects are the same across all three colleges. If exam format affects science students differently from humanities students, that difference gets added to the error. A follow-up experiment that includes college as a factor with interaction terms would let us test if this is true or not. H0: Exam format affects all students equally Ha: Exam format affects students differently depending on the college they are a part of If the interaction between the college a student is in and the exam format is significant, we reject the null hypothesis and conclude that exam format affects students differently depending on college.

Since this is a full 2⁴ factorial, there is no confounding, but a follow-up screening experiment using a fractional factorial would introduce aliasing. The fraction would need to be chosen carefully to avoid aliasing two-factor interactions with main effects, since those are most likely to be active based on the heredity principle.

The ExamFormatPracticeFrequencyClassTimeStructure three-way interaction term (effect = -10) and the ExamFormat*PracticeFrequency two-way interaction terms have the largest effect in the model, but it is difficult to interpret and act on. A new model with all factors and interaction terms would be a good next step to confirm relationships or possibly build a reduced model