Part 1

The Siege Setup

We performed a designed experiment on the effect of Release Angle on the distance in which a ball is thrown (measured at its furthest distance). Specifically, we studied whether the settings of 175, 180, and 185 degrees significantly differ in their mean distance thrown. Since pulling the lever back takes additional work, we investigated whether this made a significant difference on the mean distance thrown. The other factors were set to constant values of the following:

  Fire Angle = 90°
  Bungee Position = 200mm
  Pin Elevation = 200mm
  Cup Elevation = 300mm

To test this hypothesis, we used a completely randomized design with an α = 0.05.

Sample Size

To save resources during the preparation for the siege, we determined how many samples should be collected to detect a mean difference with a medium effect (i.e. 50% of the standard deviation) with a probability of 75%. The chart below shows the Balanced One-way Analysis of Variance Power Calculation.

The results of the sample size calculation showed the need to run 14 launches per Release Angle.

Trial Run Order & Results

We knew that in order to reduce the margin of error in our sampling process, we would randomize our design for collecting catapult data. In order to achieve this we leveraged the built-in capabilities of r through the design.crd function.

The final randomized run order is shown below with their respective distance measures:

Release Angle Distance Traveled
180 270
185 275
185 281
175 272
180 283
175 269
180 281
185 286
180 266
175 265
185 279
185 282
175 260
185 278
185 281
175 264
180 270
175 267
180 277
180 275
180 267
185 297
180 285
185 278
175 259
180 264
175 268
185 286
185 295
180 263
185 288
175 268
180 250
180 273
175 265
185 273
175 255
175 268
180 268
175 283
175 254
185 275

Hypothesis Testing

We needed to then understand if Release Angle plays a significant role in the distance the ball travels. To complete this task, we ran a One-way ANOVA test.

Hypotheses

The hypotheses for the ANOVA were as follows:
 Null:     H0: μ 175 = μ 180 = μ 185
 Alternate: H1: μi ≠ μj for at least one pair (i,j)

ANOVA

The ANOVA was then run in R.

Which showed that at least one of the means was significantly different than the others.

##             Df Sum Sq Mean Sq F value   Pr(>F)    
## angles       2   2096  1048.1   16.46 6.55e-06 ***
## Residuals   39   2483    63.7                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

We can see that the p-value less than 0.001 , and with an α = 0.05, we reject the Null hypothesis. There is evidence to support the claim of a significant difference between at least one of the three sample mean Release Angle launch distances.

Residuals

The residuals were then assessed for normality and constant variance. From the charts below the residuals are indeed homoscedastic and normal.

Pairwise Comparisons

Pairwise comparisons were conducted to understand which pairs were statistically significantly different. The following hypotheses were created for a Tukey’s Test.
 Null:     H0: μi = μj
 Alternate: H1: μi ≠ μj for all i ≠ j.

The Tukey’s Test was then run in R.

With the following results:

##   Tukey multiple comparisons of means
##     95% family-wise confidence level
## 
## Fit: aov.default(formula = distance ~ angles, data = dat)
## 
## $angles
##              diff       lwr      upr     p adj
## 180-175  5.357143 -1.989822 12.70411 0.1907605
## 185-175 16.928571  9.581607 24.27554 0.0000053
## 185-180 11.571429  4.224464 18.91839 0.0012648

We can see that the pairwise comparisons resulted in two of the three comparisons being significant to an α = 0.05.

Release Angle Comparison P-value
180 - 175 0.191
185 - 175 < 0.001 *
185 - 180 0.001 *
" * " denotes a significant p-value

For the ‘185-175’ and ‘185-180’ comparisons, we reject the Null hypothesis. There is evidence to support the claim of a significant difference between the sample mean distances.

For the ‘180-175’ comparison, we fail to reject the Null hypothesis. There is not enough evidence to support the claim of a significant difference between the sample mean distances.

Conclusions & Recommendations

Release Angle was shown to be a significant factor for some of the pair-wise comparisons but not all. It was not shown to be significant when pulled between 175° to the 180° position.

It is also recommended to study other Release Angles to understand the impact they play on ball distance traveled throughout the range of arm motion.

Part 2

The Designed Experiment

In Part 2, we performed a designed experiment on the effect of Pin Elevation and Release Angle on the distance in which a ball is thrown (measured at its furthest distance). Specifically, we studied whether the settings of 100, 150 and 200mm for Pin Elevation and the settings of 110°, 140° and 170° for Release Angle significantly differ in their mean distance thrown.

The other factors were set to constant values of the following:

  Fire Angle = 100°
  Bungee Position = 150mm
  Cup Elevation = 250mm

The settings of Pin Elevation at 100, 150 and 200mm were investigated as a fixed effect, as well as settings of the Release Angle corresponding to 110°, 140° and 170° as a random effect. We replicated the design three times.

Model Equation

The model equation is: \(\quad y_{ijk}\) = \(\mu\) + \(\tau_i\) + \(\beta_j\) + \((\tau\beta)_{ij}\) + \(\epsilon_{ijk}\)

 Where \(\tau_i\) is the fixed effect Pin Elevation
  \(\beta_j\) is the random effect Release Angle
  \((\tau\beta)_{ij}\) is the interaction effect of Pin Elevation * Release Angle
  and \(\epsilon_{ijk}\) is the error term

Hypotheses

The Hypotheses we will test are:
Pin Elevation Main Fixed Effect:
\(\quad H_0\) : \(\tau_i\) = 0 \(\forall\) i
\(\quad H_a\) : \(\tau_i \neq\) 0 for at least one \(\ i\)
Release Angle Main Random Effect:
\(\quad H_0\) : \(\beta_j\) = 0 \(\forall\) j
\(\quad H_a\) : \(\beta_j \neq\) 0 for at least one \(\ j\)
Pin Elevation * Release Angle Interaction Effect:
\(\quad H_0\) : \((\tau\beta)_{ij}\) = 0 \(\forall\) ij
\(\quad H_a\) : \((\tau\beta)_{ij} \neq\) 0 for at least one \(\ ij\)
\(\quad\)at a significance level of \(\alpha\) = 0.05

Ranomized Layout & Results

The proposed layout with a randomized run order and the recorded observations were:

pinElevation releaseAngle distance
200 140 220
200 170 329
150 170 271
200 110 69
150 110 56
150 140 183
150 110 60
200 110 75
200 140 215
150 140 181
100 110 53
200 170 340
150 140 182
200 140 217
100 170 209
150 170 272
100 110 48
150 110 64
100 140 144
100 170 204
100 110 50
100 170 204
200 110 68
200 170 334
150 170 269
100 140 146
100 140 144

Mixed Effects Model

The result of the General ANOVA Mixed Effects Model are:

Df Sum Sq Mean Sq F value Pr(>F)
pinElevation 2 24568.9630 12284.481482 5.57883 0.0696395
releaseAngle 2 199786.7407 99893.370370 10413.59459 0.0000000
pinElevation:releaseAngle 4 8807.9259 2201.981481 229.55019 0.0000000
Residual 18 172.6667 9.592593 NA NA

The interaction effect is significant with a p-value < 0.001 at a significance level of \(\alpha\) = 0.05. We stop here and do not analyze the main effects. We can see this interaction in the following plot where the slope of the release angles are not equal to each other.

Residuals

The residuals were then assessed for normality and constant variance. From the charts below the residuals are indeed homoscedastic and normal.

Part 3

The Designed Experiment

In Part 3, we performed a designed experiment to determine the effect of the available factors of Fire Angle, Bungee Position, Release Angle, Pin Elevation, and Cup Elevation on the distance in which a ball is thrown (measured at its furthest distance). We designed this experiment as a single replicate of a 2^5 factorial design with the low and high level of the factors being as follows:

Factor Low Level (-1) High Level (+1)
Fire Angle 90 degrees 110 degrees
Bungee Position 130mm 150mm
Release Angle 140 degrees 170 degrees
Pin Elevation 125mm 175mm
Cup Elevation 225mm 275mm

Randomized Layout & Results

The proposed layout with a randomized run order was generated. Trial runs were taken and the data was collected. The observations were recorded and are presented in the table below.

Fire Angle Bungee Position Release Angle Pin Elevation Cup Elevation Response
110 130 170 175 225 237
110 150 170 175 225 260
110 150 170 175 275 343
90 150 170 175 275 189
110 130 140 175 225 135
110 150 140 125 275 154
90 150 140 175 225 121
90 150 170 175 225 153
110 130 170 175 275 312
90 150 140 175 275 152
110 130 140 175 275 171
110 150 140 125 225 114
90 150 170 125 275 169
90 130 140 175 225 114
110 150 140 175 225 145
90 150 170 125 225 135
90 130 170 175 225 149
110 150 140 175 275 197
90 130 140 125 225 98
90 150 140 125 275 134
90 130 170 125 275 157
90 130 140 125 275 125
110 150 170 125 275 262
110 130 170 125 225 180
90 130 170 125 225 124
90 130 140 175 275 143
110 130 140 125 225 101
90 130 170 175 275 183
110 130 170 125 275 238
110 150 170 125 225 199
90 150 140 125 225 104
110 130 140 125 275 140

Model Equation

The model equation for this experiment is as follows:

Distance = A + B + C + D + E + AB + AC + AD + AE + BC + BD + BE + CD + CE + DE + ABC + ABD + ABE + ACD + ACE + ADE + BCD + BCE + BDE + CDE + ABCD + ABCE + ABDE + ACDE + BCDE + ABCDE

Where: A = Fire Angle; B = Bungee Position; C = Release Angle; D = Pin Elevation; E = Cup Elevation

Significant Factors/Interactions

The half-normal plot for all of the factors and interactions was created which identified the significant factors.

The significant terms were found to be:

Signficant Factors/Interactions
Fire_Angle:Release_Angle:Cup_Elevation
Fire_Angle:Bungee_Position
Fire_Angle:Release_Angle:Pin_Elevation
Release_Angle:Cup_Elevation
Release_Angle:Pin_Elevation
Fire_Angle:Cup_Elevation
Bungee_Position
Fire_Angle:Pin_Elevation
Pin_Elevation
Fire_Angle:Release_Angle
Cup_Elevation
Fire_Angle
Release_Angle

Final ANOVA for Model Equation

As we reduced the model, the insignificant factors/interactions were built into the error term. The following represents the ANOVA with only significant factors and interactions built in (to an alpha of 0.05)

##                                        Df Sum Sq Mean Sq F value   Pr(>F)    
## Bungee_Position                         1   1568    1568   71.73 1.08e-07 ***
## Pin_Elevation                           1  10153   10153  464.44 2.64e-14 ***
## Cup_Elevation                           1  15313   15313  700.45 7.30e-16 ***
## Fire_Angle                              1  27495   27495 1257.72  < 2e-16 ***
## Release_Angle                           1  40755   40755 1864.28  < 2e-16 ***
## Fire_Angle:Bungee_Position              1    288     288   13.17 0.001917 ** 
## Release_Angle:Cup_Elevation             1    545     545   24.91 9.47e-05 ***
## Release_Angle:Pin_Elevation             1    741     741   33.90 1.62e-05 ***
## Fire_Angle:Cup_Elevation                1   1152    1152   52.70 9.50e-07 ***
## Fire_Angle:Pin_Elevation                1   2016    2016   92.22 1.66e-08 ***
## Fire_Angle:Release_Angle                1  11476   11476  524.96 9.11e-15 ***
## Fire_Angle:Release_Angle:Cup_Elevation  1    264     264   12.10 0.002682 ** 
## Fire_Angle:Release_Angle:Pin_Elevation  1    406     406   18.58 0.000421 ***
## Residuals                              18    394      22                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Final Model Equation

\[ \begin{aligned} \operatorname{\widehat{response}} &= 169.94 + 7(\operatorname{Bungee\_Position}) + 17.81(\operatorname{Pin\_Elevation}) + 21.88(\operatorname{Cup\_Elevation})\ + \\ &\quad 29.31(\operatorname{Fire\_Angle}) + 35.69(\operatorname{Release\_Angle}) + 3(\operatorname{Bungee\_Position}_{\operatorname{Fire\_Angle}} \times \operatorname{Fire\_Angle}_{\operatorname{Bungee\_Position}}) + 4.13(\operatorname{Cup\_Elevation}_{\operatorname{Release\_Angle}} \times \operatorname{Release\_Angle}_{\operatorname{Cup\_Elevation}})\ + \\ &\quad 4.81(\operatorname{Pin\_Elevation}_{\operatorname{Release\_Angle}} \times \operatorname{Release\_Angle}_{\operatorname{Pin\_Elevation}}) + 6(\operatorname{Cup\_Elevation}_{\operatorname{Fire\_Angle}} \times \operatorname{Fire\_Angle}_{\operatorname{Cup\_Elevation}}) + 7.94(\operatorname{Pin\_Elevation}_{\operatorname{Fire\_Angle}} \times \operatorname{Fire\_Angle}_{\operatorname{Pin\_Elevation}}) + 18.94(\operatorname{Fire\_Angle} \times \operatorname{Release\_Angle})\ + \\ &\quad 2.87(\operatorname{Cup\_Elevation}_{\operatorname{Fire\_Angle}} \times \operatorname{Fire\_Angle}_{\operatorname{Release\_Angle}} \times \operatorname{Release\_Angle}_{\operatorname{Cup\_Elevation}}) + 3.56(\operatorname{Pin\_Elevation}_{\operatorname{Fire\_Angle}} \times \operatorname{Fire\_Angle}_{\operatorname{Release\_Angle}} \times \operatorname{Release\_Angle}_{\operatorname{Pin\_Elevation}}) \end{aligned} \]

All Code

# Setup Libraries
library(dplyr)
library(tidyr)
library(readr)
library(knitr)
library(agricolae)
library(lawstat)
library(car)
library(GAD)
library(BSDA)
library(pwr)
library(WebPower)
library(ggplot2)
library(ggfortify)
library(ggpubr)
library(SixSigma)
library(DoE.base)
library(FrF2)
library(equatiomatic)

#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
# Part 1

effectiveReleaseAngle = c(175,180,185)
firingAngle  = 90   #degrees
bungeePos    = 200  #mm
pinElevation = 200  #mm
cupElevation = 300  #mm
alpha        = 0.05 

#------------------------------------------------------------------------------
# Determine how many samples should be collected to detect a mean 
#   difference with a medium effect (i.e. 50% of the standard deviation) 
#   and a pattern of maximum variability with a probability of 75%.  
                                                                                                    
power1 = 0.75

#d is the range of means divided by sigma, k is the number of populations
d = mean(diff(effectiveReleaseAngle))
sd1 = sd(effectiveReleaseAngle)

# sigma = 5
# power = .75
# u = k - 1
# u = 2
# a = 0.05

kgroups = length(effectiveReleaseAngle)

#The effect is given by d/2 when k is even
#The effect is given by d*sqrt(k^2-1)/(2*k) when k is odd
fCal = d/sd1*sqrt(kgroups^2-1)/(2*kgroups)

sampleNumber <- pwr.anova.test(n = NULL,
                               k = kgroups,
                               f = fCal,
                               sig.level = alpha,
                               power = power1)

plot(sampleNumber)
#------------------------------------------------------------------------------
# Propose a layout using the number of samples from part (a) with 
#   randomized run order
n = ceiling(sampleNumber$n)

# set seed here for reproducibility
design <- design.crd(trt = effectiveReleaseAngle,
                     r = n,
                     seed = 1234)
design$book

#------------------------------------------------------------------------------
# Collect data and record observations on layout proposed in part (b)

recordedDistance <- c(270,275,281,272,283,269,281,
                      286,266,265,279,282,260,278,
                      281,264,270,267,277,275,267,
                      297,285,278,259,264,268,286,
                      295,263,288,268,250,273,265,
                      273,255,268,268,283,254,275)

#------------------------------------------------------------------------------
# Perform hypothesis test and check residuals.  Be sure to comment and 
#   take corrective action if necessary.
dat <- data.frame(angles   = design$book$effectiveReleaseAngle,
                  distance = recordedDistance)
dataov <- aov(distance~angles,data = dat)
summary(dataov)

datHSD <- TukeyHSD(dataov)
datHSD
plot(datHSD)
autoplot(dataov)
#------------------------------------------------------------------------------
#   If the null hypothesis is rejected, investigate pairwise comparisons. 


#------------------------------------------------------------------------------
#   State conclusions and make recommendation. 
                                                                                                   

#------------------------------------------------------------------------------
#------------------------------------------------------------------------------
# Part 2

# Determine effect of Pin Elevations and Release Angle, 
#   when Fire Angle at 100 Degrees
# 
# Bungee Position is 150m & Cup Elevation is 250mm
# 
# Pin Elevation:Fixed Effect,  100, 150, 200 mm.
# Release Angle:Random Effect, 110, 140, 170 degrees.

# model equation:
# distance~pinElevation+releaseAngle+pinElevation*releaseAngle

# init
pinElevation <- as.fixed(c(100,150,200))
releaseAngle <- as.random(c(110,140,170))

# 3x3 factorial with 3 replications
trts<-c(3,3)
design3x3 <- design.ab(trt=trts, r=3, design="crd",seed=1234)

# replace names & values
names(design3x3$book)[names(design3x3$book) == c("A","B")] <- c("pinElevation","releaseAngle")
design3x3$book$pinElevation <- pinElevation[match(design3x3$book$pinElevation,1:3)]
design3x3$book$releaseAngle <- releaseAngle[match(design3x3$book$releaseAngle,1:3)]

design3x3$book

distanceRecorded <- c(220,329,271,69,56,183,60,75,215,
                      181,53,340,182,217,209,272,48,64,
                      144,204,50,204,68,334,269,146,144)

dat2 <- data.frame(pinElevation = design3x3$book$pinElevation,
                   releaseAngle = design3x3$book$releaseAngle,
                   distance     = distanceRecorded)

dat2aov <- aov(distance~pinElevation+releaseAngle+pinElevation*releaseAngle,data = dat2)
gad(dat2aov)

autoplot(dat2aov)
#pinElevation vs releaseAngle
interaction.plot(x.factor = dat2$pinElevation, #x-axis variable
                 trace.factor = dat2$releaseAngle, #variable for lines
                 response = dat2$distance, #y-axis variable
                 fun = median, #metric to plot
                 ylab = "distance",
                 xlab = "pinElevation",
                 col = c("green", "blue", "purple"),
                 lty = 1, #line type
                 lwd = 2, #line width
                 trace.label = "releaseAngle")
dat2 %>% group_by(pinElevation, releaseAngle) %>%
  summarise(
    count = n(),
    mean = mean(distance, na.rm = TRUE),
    sd = sd(distance, na.rm = TRUE))

## Randomized Layout & Results

Fire_Angle      <- c(90 ,110) # Degrees
Bungee_Position <- c(130,150) # mm
Release_Angle   <- c(140,170) # Degrees
Pin_Elevation   <- c(125,175) # mm
Cup_Elevation   <- c(225,275) # mm

#(a & b)
trts<-c(2,2,2,2,2)
design2by5 <- design.ab(trt=trts, r=1, design="crd",seed=1234)
#design2by5

# replace names & values
colnames(design2by5$book)[-(1:2)] <- c("Fire_Angle",
                                       "Bungee_Position",
                                       "Release_Angle",
                                       "Pin_Elevation",
                                       "Cup_Elevation")
design2by5_table <- design2by5
design2by5_table$book$Fire_Angle      <- Fire_Angle[match(design2by5$book$Fire_Angle,1:2)]
design2by5_table$book$Bungee_Position <- Bungee_Position[match(design2by5$book$Bungee_Position,1:2)]
design2by5_table$book$Release_Angle   <- Release_Angle[match(design2by5$book$Release_Angle,1:2)]
design2by5_table$book$Pin_Elevation   <- Pin_Elevation[match(design2by5$book$Pin_Elevation,1:2)]
design2by5_table$book$Cup_Elevation   <- Cup_Elevation[match(design2by5$book$Cup_Elevation,1:2)]


# Results of the Trials
response <- c(237,260,343,189,135,154,121,153,312,152,
              171,114,169,114,145,135,149,197,98,134,157,
              125,262,180,124,143,101,183,238,199,104,140)

# Create the Table
design2by5table <- cbind(design2by5_table$book$Fire_Angle,
                         design2by5_table$book$Bungee_Position,
                         design2by5_table$book$Release_Angle,
                         design2by5_table$book$Pin_Elevation,
                         design2by5_table$book$Cup_Elevation,
                         response)

colnames(design2by5table) <- c("Fire Angle",
                              "Bungee Position",
                              "Release Angle",
                              "Pin Elevation",
                              "Cup Elevation",
                              "Response")

knitr::kable(design2by5table)

#(c)
io <- c(-1,1)
design2by5$book$Fire_Angle      <- io[match(design2by5$book$Fire_Angle,1:2)]
design2by5$book$Bungee_Position <- io[match(design2by5$book$Bungee_Position,1:2)]
design2by5$book$Release_Angle   <- io[match(design2by5$book$Release_Angle,1:2)]
design2by5$book$Pin_Elevation   <- io[match(design2by5$book$Pin_Elevation,1:2)]
design2by5$book$Cup_Elevation   <- io[match(design2by5$book$Cup_Elevation,1:2)]

design2by5$book <- cbind(design2by5$book,response)

datmodel <- lm(response~Fire_Angle*
                        Bungee_Position*
                        Release_Angle*
                        Pin_Elevation*
                        Cup_Elevation,
               data = design2by5$book)

modelFactorEff <- coef(datmodel)[-1]*2

halfnormal(datmodel)

halfEff <- halfnormal(datmodel)
# significant factors/interactions
knitr::kable(halfEff$signif, col.names = "Signficant Factors/Interactions")

# Final ANOVA Table
datmodelsig <- lm(response~
                    Fire_Angle:Release_Angle:Cup_Elevation + 
                    Fire_Angle:Bungee_Position + 
                    Fire_Angle:Release_Angle:Pin_Elevation + 
                    Release_Angle:Cup_Elevation + 
                    Release_Angle:Pin_Elevation + 
                    Fire_Angle:Cup_Elevation + 
                    Bungee_Position + 
                    Fire_Angle:Pin_Elevation + 
                    Pin_Elevation + 
                    Fire_Angle:Release_Angle + 
                    Cup_Elevation + 
                    Fire_Angle + 
                    Release_Angle,
                   data = design2by5$book)


dataovsig <- aov(datmodelsig)
summary(dataovsig)

## Final Model Equation
equatiomatic::extract_eq(datmodelsig, wrap = TRUE, use_coefs = TRUE)