Team 4 Statapult
Final Project IE 5342 - Fall 2021
F. Gersdorff, H. Swerdloff, J. Rosila Mares
Part 1
The Siege Setup
We performed a designed experiment on the effect of Release Angle on the distance in which a ball is thrown (measured at its furthest distance). Specifically, we studied whether the settings of 175, 180, and 185 degrees significantly differ in their mean distance thrown. Since pulling the lever back takes additional work, we investigated whether this made a significant difference on the mean distance thrown. The other factors were set to constant values of the following:
  Fire Angle = 90°
  Bungee Position = 200mm
  Pin Elevation = 200mm
  Cup Elevation = 300mm
To test this hypothesis, we used a completely randomized design with an α = 0.05.
Sample Size
To save resources during the preparation for the siege, we determined how many samples should be collected to detect a mean difference with a medium effect (i.e. 50% of the standard deviation) with a probability of 75%. The chart below shows the Balanced One-way Analysis of Variance Power Calculation.
The results of the sample size calculation showed the need to run 14 launches per Release Angle.
Trial Run Order & Results
We knew that in order to reduce the margin of error in our sampling process, we would randomize our design for collecting catapult data. In order to achieve this we leveraged the built-in capabilities of r through the design.crd function.
The final randomized run order is shown below with their respective distance measures:
| Release Angle | Distance Traveled |
|---|---|
| 180 | 270 |
| 185 | 275 |
| 185 | 281 |
| 175 | 272 |
| 180 | 283 |
| 175 | 269 |
| 180 | 281 |
| 185 | 286 |
| 180 | 266 |
| 175 | 265 |
| 185 | 279 |
| 185 | 282 |
| 175 | 260 |
| 185 | 278 |
| 185 | 281 |
| 175 | 264 |
| 180 | 270 |
| 175 | 267 |
| 180 | 277 |
| 180 | 275 |
| 180 | 267 |
| 185 | 297 |
| 180 | 285 |
| 185 | 278 |
| 175 | 259 |
| 180 | 264 |
| 175 | 268 |
| 185 | 286 |
| 185 | 295 |
| 180 | 263 |
| 185 | 288 |
| 175 | 268 |
| 180 | 250 |
| 180 | 273 |
| 175 | 265 |
| 185 | 273 |
| 175 | 255 |
| 175 | 268 |
| 180 | 268 |
| 175 | 283 |
| 175 | 254 |
| 185 | 275 |
Hypothesis Testing
We needed to then understand if Release Angle plays a significant role in the distance the ball travels. To complete this task, we ran a One-way ANOVA test.
Hypotheses
The hypotheses for the ANOVA were as follows:
 Null:     H0: μ 175 = μ 180 = μ 185
 Alternate: H1: μi ≠μj for at least one pair (i,j)
ANOVA
The ANOVA was then run in R.
Which showed that at least one of the means was significantly different than the others.
## Df Sum Sq Mean Sq F value Pr(>F)
## angles 2 2096 1048.1 16.46 6.55e-06 ***
## Residuals 39 2483 63.7
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1We can see that the p-value less than 0.001 , and with an α = 0.05, we reject the Null hypothesis. There is evidence to support the claim of a significant difference between at least one of the three sample mean Release Angle launch distances.
Residuals
The residuals were then assessed for normality and constant variance. From the charts below the residuals are indeed homoscedastic and normal.
Pairwise Comparisons
Pairwise comparisons were conducted to understand which pairs were statistically significantly different. The following hypotheses were created for a Tukey’s Test.
 Null:     H0: μi = μj
 Alternate: H1: μi ≠μj for all i ≠j.
The Tukey’s Test was then run in R.
With the following results:
## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = distance ~ angles, data = dat)
##
## $angles
## diff lwr upr p adj
## 180-175 5.357143 -1.989822 12.70411 0.1907605
## 185-175 16.928571 9.581607 24.27554 0.0000053
## 185-180 11.571429 4.224464 18.91839 0.0012648We can see that the pairwise comparisons resulted in two of the three comparisons being significant to an α = 0.05.
| Release Angle Comparison | P-value |
|---|---|
| 180 - 175 | 0.191 |
| 185 - 175 | < 0.001 * |
| 185 - 180 | 0.001 * |
| " * " denotes a significant p-value | |
For the ‘185-175’ and ‘185-180’ comparisons, we reject the Null hypothesis. There is evidence to support the claim of a significant difference between the sample mean distances.
For the ‘180-175’ comparison, we fail to reject the Null hypothesis. There is not enough evidence to support the claim of a significant difference between the sample mean distances.
Conclusions & Recommendations
Release Angle was shown to be a significant factor for some of the pair-wise comparisons but not all. It was not shown to be significant when pulled between 175° to the 180° position.
It is also recommended to study other Release Angles to understand the impact they play on ball distance traveled throughout the range of arm motion.
Part 2
The Designed Experiment
In Part 2, we performed a designed experiment on the effect of Pin Elevation and Release Angle on the distance in which a ball is thrown (measured at its furthest distance). Specifically, we studied whether the settings of 100, 150 and 200mm for Pin Elevation and the settings of 110°, 140° and 170° for Release Angle significantly differ in their mean distance thrown.
The other factors were set to constant values of the following:
  Fire Angle = 100°
  Bungee Position = 150mm
  Cup Elevation = 250mm
The settings of Pin Elevation at 100, 150 and 200mm were investigated as a fixed effect, as well as settings of the Release Angle corresponding to 110°, 140° and 170° as a random effect. We replicated the design three times.
Model Equation
The model equation is: \(\quad y_{ijk}\) = \(\mu\) + \(\tau_i\) + \(\beta_j\) + \((\tau\beta)_{ij}\) + \(\epsilon_{ijk}\)
 Where \(\tau_i\) is the fixed effect Pin Elevation
  \(\beta_j\) is the random effect Release Angle
  \((\tau\beta)_{ij}\) is the interaction effect of Pin Elevation * Release Angle
  and \(\epsilon_{ijk}\) is the error term
Hypotheses
The Hypotheses we will test are:
Pin Elevation Main Fixed Effect:
\(\quad H_0\) : \(\tau_i\) = 0 \(\forall\) i
\(\quad H_a\) : \(\tau_i \neq\) 0 for at least one \(\ i\)
Release Angle Main Random Effect:
\(\quad H_0\) : \(\beta_j\) = 0 \(\forall\) j
\(\quad H_a\) : \(\beta_j \neq\) 0 for at least one \(\ j\)
Pin Elevation * Release Angle Interaction Effect:
\(\quad H_0\) : \((\tau\beta)_{ij}\) = 0 \(\forall\) ij
\(\quad H_a\) : \((\tau\beta)_{ij} \neq\) 0 for at least one \(\ ij\)
\(\quad\)at a significance level of \(\alpha\) = 0.05
Ranomized Layout & Results
The proposed layout with a randomized run order and the recorded observations were:
| pinElevation | releaseAngle | distance |
|---|---|---|
| 200 | 140 | 220 |
| 200 | 170 | 329 |
| 150 | 170 | 271 |
| 200 | 110 | 69 |
| 150 | 110 | 56 |
| 150 | 140 | 183 |
| 150 | 110 | 60 |
| 200 | 110 | 75 |
| 200 | 140 | 215 |
| 150 | 140 | 181 |
| 100 | 110 | 53 |
| 200 | 170 | 340 |
| 150 | 140 | 182 |
| 200 | 140 | 217 |
| 100 | 170 | 209 |
| 150 | 170 | 272 |
| 100 | 110 | 48 |
| 150 | 110 | 64 |
| 100 | 140 | 144 |
| 100 | 170 | 204 |
| 100 | 110 | 50 |
| 100 | 170 | 204 |
| 200 | 110 | 68 |
| 200 | 170 | 334 |
| 150 | 170 | 269 |
| 100 | 140 | 146 |
| 100 | 140 | 144 |
Mixed Effects Model
The result of the General ANOVA Mixed Effects Model are:
| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| pinElevation | 2 | 24568.9630 | 12284.481482 | 5.57883 | 0.0696395 |
| releaseAngle | 2 | 199786.7407 | 99893.370370 | 10413.59459 | 0.0000000 |
| pinElevation:releaseAngle | 4 | 8807.9259 | 2201.981481 | 229.55019 | 0.0000000 |
| Residual | 18 | 172.6667 | 9.592593 | NA | NA |
The interaction effect is significant with a p-value < 0.001 at a significance level of \(\alpha\) = 0.05. We stop here and do not analyze the main effects. We can see this interaction in the following plot where the slope of the release angles are not equal to each other.
Residuals
The residuals were then assessed for normality and constant variance. From the charts below the residuals are indeed homoscedastic and normal.
All Code
# Setup Libraries
library(dplyr)
library(tidyr)
library(readr)
library(knitr)
library(agricolae)
library(lawstat)
library(car)
library(GAD)
library(BSDA)
library(pwr)
library(WebPower)
library(ggplot2)
library(ggfortify)
library(ggpubr)
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
# Part 1
effectiveReleaseAngle = c(175,180,185)
firingAngle = 90 #degrees
bungeePos = 200 #mm
pinElevation = 200 #mm
cupElevation = 300 #mm
alpha = 0.05
#------------------------------------------------------------------------------
# Determine how many samples should be collected to detect a mean
# difference with a medium effect (i.e. 50% of the standard deviation)
# and a pattern of maximum variability with a probability of 75%.
power1 = 0.75
#d is the range of means divided by sigma, k is the number of populations
d = mean(diff(effectiveReleaseAngle))
sd1 = sd(effectiveReleaseAngle)
# sigma = 5
# power = .75
# u = k - 1
# u = 2
# a = 0.05
kgroups = length(effectiveReleaseAngle)
#The effect is given by d/2 when k is even
#The effect is given by d*sqrt(k^2-1)/(2*k) when k is odd
fCal = d/sd1*sqrt(kgroups^2-1)/(2*kgroups)
sampleNumber <- pwr.anova.test(n = NULL,
k = kgroups,
f = fCal,
sig.level = alpha,
power = power1)
plot(sampleNumber)#------------------------------------------------------------------------------
# Propose a layout using the number of samples from part (a) with
# randomized run order
n = ceiling(sampleNumber$n)
# set seed here for reproducibility
design <- design.crd(trt = effectiveReleaseAngle,
r = n,
seed = 1234)
design$book
#------------------------------------------------------------------------------
# Collect data and record observations on layout proposed in part (b)
recordedDistance <- c(270,275,281,272,283,269,281,
286,266,265,279,282,260,278,
281,264,270,267,277,275,267,
297,285,278,259,264,268,286,
295,263,288,268,250,273,265,
273,255,268,268,283,254,275)
#------------------------------------------------------------------------------
# Perform hypothesis test and check residuals. Be sure to comment and
# take corrective action if necessary.
dat <- data.frame(angles = design$book$effectiveReleaseAngle,
distance = recordedDistance)
dataov <- aov(distance~angles,data = dat)
summary(dataov)
datHSD <- TukeyHSD(dataov)
datHSD
plot(datHSD)autoplot(dataov)#------------------------------------------------------------------------------
# If the null hypothesis is rejected, investigate pairwise comparisons.
#------------------------------------------------------------------------------
# State conclusions and make recommendation.
#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
# Part 2
# Determine effect of Pin Elevations and Release Angle,
# when Fire Angle at 100 Degrees
#
# Bungee Position is 150m & Cup Elevation is 250mm
#
# Pin Elevation:Fixed Effect, 100, 150, 200 mm.
# Release Angle:Random Effect, 110, 140, 170 degrees.
# model equation:
# distance~pinElevation+releaseAngle+pinElevation*releaseAngle
# init
pinElevation <- as.fixed(c(100,150,200))
releaseAngle <- as.random(c(110,140,170))
# 3x3 factorial with 3 replications
trts<-c(3,3)
design3x3 <- design.ab(trt=trts, r=3, design="crd",seed=1234)
# replace names & values
names(design3x3$book)[names(design3x3$book) == c("A","B")] <- c("pinElevation","releaseAngle")
design3x3$book$pinElevation <- pinElevation[match(design3x3$book$pinElevation,1:3)]
design3x3$book$releaseAngle <- releaseAngle[match(design3x3$book$releaseAngle,1:3)]
design3x3$book
distanceRecorded <- c(220,329,271,69,56,183,60,75,215,
181,53,340,182,217,209,272,48,64,
144,204,50,204,68,334,269,146,144)
dat2 <- data.frame(pinElevation = design3x3$book$pinElevation,
releaseAngle = design3x3$book$releaseAngle,
distance = distanceRecorded)
dat2aov <- aov(distance~pinElevation+releaseAngle+pinElevation*releaseAngle,data = dat2)
gad(dat2aov)
autoplot(dat2aov)#pinElevation vs releaseAngle
interaction.plot(x.factor = dat2$pinElevation, #x-axis variable
trace.factor = dat2$releaseAngle, #variable for lines
response = dat2$distance, #y-axis variable
fun = median, #metric to plot
ylab = "distance",
xlab = "pinElevation",
col = c("green", "blue", "purple"),
lty = 1, #line type
lwd = 2, #line width
trace.label = "releaseAngle")dat2 %>% group_by(pinElevation, releaseAngle) %>%
summarise(
count = n(),
mean = mean(distance, na.rm = TRUE),
sd = sd(distance, na.rm = TRUE))