IE5342 Final Project
Gabriel Garza, Ben Herndon, Jeffrey Goens, Scott Michaelis
11/24/2021
Introduction
This project is a three part experiment to evaulate the effect of several factors on the distance an online catapult called the Statapult can throw a virtual ball. A depiction of the Statapult is shown in Figure 1.
Figure 1 Statpult from the website https://sigmazone.com/catapult/ with descriptions added
The user can adjust the configuration of the Statapult by using their mouse to change the location of various parameters. Then the Statapult arm can be pulled back, again using the mouse, the the desired release angle. Once the ball is thrown, the website shows where the ball lands and a handy measurement appears which can be used to visually collect results. This is shown in Figure 2.
Figure 2 Measurement example from the website https://sigmazone.com/catapult/ after a ball is thrown
The Statapult has 5 adjustments (factors) that can be made to experiment on the distance that the ball can be thrown. In this project, each adjustment can be set within certain allowed ranges. These factors, with the maximum and minimum values are listed in Table 1.
| Factor Name | Minimum Setting | Maximum Setting |
|---|---|---|
| Fire Angle | 90 degrees | 140 degrees |
| Release Angle | Fire Angle | 180 degrees |
| Bungee Position | 100mm | 200 mm |
| Pin Elevation | 100 mm | 200 mm |
| Cup Elevation | 200 mm | 300 mm |
Part 1 of the project determins the effect of the Release Angle on the distance that the ball is thrown. Part 2 adds a second factor to also study the effect of Pin Elevation. Part 3 of the experiment looks at all 5 available factors on the distance that the ball is thrown.
To conduct the analysis required by this project, the statistics package R is used in Open Source RStudio. The text and formatting of this report are generated using the R Markdown feature in RStudio. The R commands and the output of the R scripts, in most cases, appears in this report as it is needed. This allows the reader to understand what commands are being used to ease in coorelating results with source material. A summary of all R-Code used is presented in the Appendix.
Part 1
The instructions for Part 1 of the project are as follows:
Perform a designed experiment on the effect of Release Angle on the distance in which the ball is thrown. Specifically, we would like to study whether the settings of 175, 180, and 185 degrees significantly differ in their mean distance. Since pulling the lever back takes additional work, we would like to investigate whether this makes a significant difference on the mean distance thrown. The other factors will be set to the following Fire Angle = 90deg, Bungee Position = 200mm, Pin Elevation = 200mm, and Cup Elevation = 300mm. To test this hypothesis, we wish to use a completely randomized design with an alpha around 0.05.
Project Part 1 Set Up
library(pwr)
library(agricolae)
library(tidyr)
library(stats)
library(GAD)## Loading required package: matrixStats## Loading required package: R.methodsS3## R.methodsS3 v1.8.1 (2020-08-26 16:20:06 UTC) successfully loaded. See ?R.methodsS3 for help.Part 1 - a)
Determine how many samples should be collected to detect a mean difference with a medium effect (i.e. 50% of the standard deviation) with a probability of 75%.*
library(pwr)
alphaLevel <- .05
confidence <- .75
d <- .5
numPop <- 3
fVal <- (d/2)*sqrt((numPop+1)/(3*(numPop-1)))
pwr.anova.test(k = numPop,n = NULL,f = fVal,sig.level = alphaLevel,power = confidence)##
## Balanced one-way analysis of variance power calculation
##
## k = 3
## n = 69.73415
## f = 0.2041241
## sig.level = 0.05
## power = 0.75
##
## NOTE: n is number in each groupThe number of samples needed is 70 per group.
NEED TO COMPLETE PART 1
Part 2
Instructions:
Perform a designed experiment to determine the effect of Pin Elevation and Release Angle on distance in which a ball is thrown when Fire Angle is 100 degrees, Bungee Position is 150mm, and Cup Elevation is 250mm. Settings of the Pin Elevation at 100, 150, and 200mm should be investigated as a fixed effect, as well as settings of the Release Angle corresponding to 110, 140, and 170 degrees as a random effect. The design should be replicated three times.
Part 2a
State model equation with the null and alternative hypotheses to be tested. In addition, state the level of significance that will be used in your analysis.
The linear effects equation for this analysis is \[y_{ijk}=\mu+\alpha_i+\beta_j+\alpha\beta_{ij} + \epsilon_{ijk}\] The Hypotheseis are as follow:
\[H_0: \alpha\beta_{ij} = 0\ for\ all\ ij\] \[H_1: \alpha\beta_{ij} \ne 0\ for\ at\ least\ one\ ij\] \[H_0: \alpha_i = 0\ for\ all\ ij\] \[H_1: \alpha_i \ne 0\ for\ at\ least\ one\ i\] \[H_0: \beta_j = 0\ for\ all\ ij\] \[H_1: \beta_j \ne 0\ for\ at\ least\ one\ j\] In this analysis, a standard \(\alpha = 0.05\) will be used.
Part 2b
Propose a layout with a randomized run order
An experiment with 2 Factors with 3 levels per factor and 3 replications will be conducted.
library(agricolae)
trts <- c(3,3)
seedNum <- 1234567
experiment <- design.ab(trt = trts, r=3,design="crd",seed = seedNum)The random data collection order is shown here below.
experiment$book## plots r A B
## 1 101 1 1 2
## 2 102 1 1 3
## 3 103 1 3 3
## 4 104 1 3 2
## 5 105 1 2 3
## 6 106 1 2 2
## 7 107 1 2 1
## 8 108 2 1 2
## 9 109 1 1 1
## 10 110 2 2 3
## 11 111 2 3 3
## 12 112 2 3 2
## 13 113 3 1 2
## 14 114 2 1 1
## 15 115 2 1 3
## 16 116 1 3 1
## 17 117 3 3 2
## 18 118 2 2 2
## 19 119 3 2 3
## 20 120 2 3 1
## 21 121 2 2 1
## 22 122 3 1 3
## 23 123 3 2 1
## 24 124 3 2 2
## 25 125 3 1 1
## 26 126 3 3 3
## 27 127 3 3 1Part 2c
Collect data and record observations on the layout proposed in part B
The data was collected using the online catapult per the data collection order shown above.
#read in the data
expData <- read.csv("C:/Users/fay/Documents/Scott Texas Tech/DoE/Project/Part2_Data.csv")
observation <- expData$Output
library(GAD)
pinEl <- as.fixed(expData$Pin.Elevation)
relAng <- as.random(expData$Release.Angle)The following table shows collected data.
| Replicate | Pin.Elevation | Release.Angle | Output |
|---|---|---|---|
| 1 | 100 | 140 | 124 |
| 1 | 100 | 170 | 178 |
| 1 | 200 | 170 | 273 |
| 1 | 200 | 140 | 186 |
| 1 | 150 | 170 | 227 |
| 1 | 150 | 140 | 156 |
| 1 | 150 | 110 | 53 |
| 2 | 100 | 140 | 123 |
| 1 | 100 | 110 | 39 |
| 2 | 150 | 170 | 229 |
| 2 | 200 | 170 | 277 |
| 2 | 200 | 140 | 188 |
| 3 | 100 | 140 | 124 |
| 2 | 100 | 110 | 38 |
| 2 | 100 | 170 | 183 |
| 1 | 200 | 110 | 60 |
| 3 | 200 | 140 | 186 |
| 2 | 150 | 140 | 156 |
| 3 | 150 | 170 | 228 |
| 2 | 200 | 110 | 63 |
| 2 | 150 | 110 | 52 |
| 3 | 100 | 170 | 180 |
| 3 | 150 | 110 | 53 |
| 3 | 150 | 140 | 155 |
| 3 | 100 | 110 | 38 |
| 3 | 200 | 170 | 280 |
| 3 | 200 | 110 | 63 |
Part 2d
Test the hypotheses and state conclusions, determining those effects that are significant. Show any plots that might be useful/necessary to show your findings. You may also show residual plots and make appropriate comments, but do not transform the data (i.e. use the raw data regardless of normality and variance constancy)
The Analysis of Variance follows:
aov1 <- aov(observation~pinEl+relAng+pinEl*relAng)
gad(aov1)## Analysis of Variance Table
##
## Response: observation
## Df Sum Sq Mean Sq F value Pr(>F)
## pinEl 2 16749 8374 8.4192 0.03685 *
## relAng 2 142985 71492 25398.5921 < 2e-16 ***
## pinEl:relAng 4 3979 995 353.3684 < 2e-16 ***
## Residual 18 51 3
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1qqnorm(aov1$residuals)
qqline(aov1$residuals)
boxplot(aov1$residuals~aov1$fitted.values)
interaction.plot(pinEl,relAng,observation)
interaction.plot(relAng,pinEl,observation)
The data looks to be normally distributed except for are few observations at the tails.
Would have some concerns about constant variance. However from the box plot, the residuals seem to be within whiskers listed.
Based on results of anova, the interaction of Pin Elevation and release angle is highly significant.
Part 3
Perform a designed experiment to determine the effect of the available factors of Fire Angle, Bungee Position, Release Angle, Pin Elevation, and Cup Elevation on distance in which a ball is thrown. Design this experiment as a single replicate of a 25 factorial design with the low and high level of the factors being as follows
Part 3a
Propose a data collection layout with a randomized run order
library(agricolae)
trts <- c(2,2,2,2,2)
reps <- 1
seedNum <- 1234567
experiment <- design.ab(trt = trts, r=reps,design="crd",seed = seedNum)
experiment$book## plots r A B C D E
## 1 101 1 1 1 2 1 1
## 2 102 1 1 2 1 1 1
## 3 103 1 2 2 1 1 1
## 4 104 1 2 1 2 2 1
## 5 105 1 2 2 2 1 2
## 6 106 1 2 2 1 1 2
## 7 107 1 2 1 1 1 1
## 8 108 1 1 2 2 2 1
## 9 109 1 1 2 1 1 2
## 10 110 1 1 1 2 1 2
## 11 111 1 1 1 1 1 1
## 12 112 1 2 1 1 1 2
## 13 113 1 2 2 2 2 2
## 14 114 1 2 1 1 2 2
## 15 115 1 1 1 1 2 2
## 16 116 1 2 1 2 2 2
## 17 117 1 2 1 2 1 1
## 18 118 1 1 2 1 2 1
## 19 119 1 1 1 1 1 2
## 20 120 1 2 1 1 2 1
## 21 121 1 2 2 2 2 1
## 22 122 1 2 2 1 2 1
## 23 123 1 2 2 1 2 2
## 24 124 1 1 1 2 2 1
## 25 125 1 1 2 1 2 2
## 26 126 1 1 2 2 1 1
## 27 127 1 1 1 1 2 1
## 28 128 1 2 1 2 1 2
## 29 129 1 1 1 2 2 2
## 30 130 1 2 2 2 1 1
## 31 131 1 1 2 2 1 2
## 32 132 1 1 2 2 2 2Part 3b
Collect the data and record observations
p3data<-read.csv("C:/Users/fay/Documents/Scott Texas Tech/DoE/Project/Part 3 csv.csv")
p3data## ï..Fire_Ang Bungee Release Pin_El Cup_El distance
## 1 -1 -1 1 -1 -1 123.0
## 2 -1 1 -1 -1 -1 101.0
## 3 1 1 -1 -1 -1 112.0
## 4 1 -1 1 1 -1 232.0
## 5 1 1 1 -1 1 261.0
## 6 1 1 -1 -1 1 148.5
## 7 1 -1 -1 -1 -1 100.0
## 8 -1 1 1 1 -1 151.0
## 9 -1 1 -1 -1 1 126.5
## 10 -1 -1 1 -1 1 155.0
## 11 -1 -1 -1 -1 -1 96.0
## 12 1 -1 -1 -1 1 138.0
## 13 1 1 1 1 1 342.0
## 14 1 -1 -1 1 1 173.5
## 15 -1 -1 -1 1 1 140.5
## 16 1 -1 1 1 1 310.0
## 17 1 -1 1 -1 -1 180.5
## 18 -1 1 -1 1 -1 118.0
## 19 -1 -1 -1 -1 1 120.5
## 20 1 -1 -1 1 -1 130.0
## 21 1 1 1 1 -1 261.0
## 22 1 1 -1 1 -1 142.0
## 23 1 1 -1 1 1 196.0
## 24 -1 -1 1 1 -1 143.5
## 25 -1 1 -1 1 1 152.5
## 26 -1 1 1 -1 -1 127.0
## 27 -1 -1 -1 1 -1 112.0
## 28 1 -1 1 -1 1 233.0
## 29 -1 -1 1 1 1 178.0
## 30 1 1 1 -1 -1 195.0
## 31 -1 1 1 -1 1 173.0
## 32 -1 1 1 1 1 196.0Part 3c
State model equation and determine what factors and interactions appear to be significant
library(DoE.base)## Loading required package: grid## Loading required package: conf.design## Registered S3 method overwritten by 'DoE.base':
## method from
## factorize.factor conf.design##
## Attaching package: 'DoE.base'## The following objects are masked from 'package:stats':
##
## aov, lm## The following object is masked from 'package:graphics':
##
## plot.design## The following object is masked from 'package:base':
##
## lengths#break things out to make the plot look better
distance<-(p3data$distance)
B<-as.factor(p3data$Bungee)
C<-as.factor(p3data$Release)
D<-as.factor(p3data$Pin_El)
E<-as.factor(p3data$Cup_El)
A<-as.factor(p3data$ï..Fire_Ang)
p3mod<-(lm(distance~A*B*C*D*E))
halfnormal(p3mod,ME.partial = TRUE)##
## Creation of A## Projected out: (Intercept)##
## Creation of B## Projected out: (Intercept),A##
## Creation of C## Projected out: (Intercept),A,B##
## Creation of D## Projected out: (Intercept),A,B,C##
## Creation of E## Projected out: (Intercept),A,B,C,D##
## Creation of AB## Projected out: (Intercept),A,B,C,D,E##
## Creation of AC## Projected out: (Intercept),A,B,C,D,E,AB##
## Creation of BC## Projected out: (Intercept),A,B,C,D,E,AB,AC##
## Creation of AD## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC##
## Creation of BD## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD##
## Creation of CD## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD##
## Creation of AE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD##
## Creation of BE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE##
## Creation of CE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE##
## Creation of DE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE##
## Creation of ABC## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE##
## Creation of ABD## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC##
## Creation of ACD## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD##
## Creation of BCD## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD##
## Creation of ABE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD##
## Creation of ACE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE##
## Creation of BCE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE##
## Creation of ADE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE##
## Creation of BDE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE,ADE##
## Creation of CDE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE,ADE,BDE##
## Creation of ABCD## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE,ADE,BDE,CDE##
## Creation of ABCE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE,ADE,BDE,CDE,ABCD##
## Creation of ABDE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE,ADE,BDE,CDE,ABCD,ABCE##
## Creation of ACDE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE,ADE,BDE,CDE,ABCD,ABCE,ABDE##
## Creation of BCDE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE,ADE,BDE,CDE,ABCD,ABCE,ABDE,ACDE##
## Creation of ABCDE## Projected out: (Intercept),A,B,C,D,E,AB,AC,BC,AD,BD,CD,AE,BE,CE,DE,ABC,ABD,ACD,BCD,ABE,ACE,BCE,ADE,BDE,CDE,ABCD,ABCE,ABDE,ACDE,BCDE##
## Significant effects (alpha=0.05, Lenth method):## [1] C1 A1 E1 A1:C1 D1 A1:D1 B1 A1:E1
##
## [9] C1:E1 C1:D1 A1:C1:D1
The following effects appear to be significant A, B, C, D, E (all of them) The following interactions appear to be significant A:C, A:D, A:E, C:E, C:D and A:C:D
Based on these estimates, the starting model function is
\[ y=A+B+C+D+E+A:C+A:D+A:E+C:E+C:D+A:C:D \]
The ANOVA will now be conducted. Insignificant factors will be eliminated, if any remain.
p3mod2 = lm(distance~A+B+C+D+E+A*C+A*D+A*D+A*E+C*E+C*D+A*C*D)
anova(p3mod2)## Analysis of Variance Table
##
## Response: distance
## Df Sum Sq Mean Sq F value Pr(>F)
## A 1 27671 27671 544.677 5.564e-16 ***
## B 1 1755 1755 34.551 9.477e-06 ***
## C 1 41616 41616 819.165 < 2.2e-16 ***
## D 1 10805 10805 212.674 4.041e-12 ***
## E 1 16200 16200 318.878 9.286e-14 ***
## A:C 1 11063 11063 217.768 3.251e-12 ***
## A:D 1 1938 1938 38.138 4.936e-06 ***
## A:E 1 1001 1001 19.709 0.0002519 ***
## C:E 1 703 703 13.840 0.0013520 **
## C:D 1 648 648 12.755 0.0019110 **
## A:C:D 1 458 458 9.006 0.0070601 **
## Residuals 20 1016 51
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1All factors in the model are significant at an \(\alpha = 0.05\) level.
The model will be checked for adaquacy.
plot(p3mod2)
Measurements 4, 28 and 30 residules are a little excessive. The data points were checked and were collected correctly. The model looks pretty linear and nothing too alarming in the residuals.
Project Conclusions
Appendix
All the R Code used to generate this report is shown here.
References and Acknowledgments
RStudio: Most calculations were done using R which is part of RStudio available at https://www.rstudio.com/. No specific versions were identified. RStudio was downloaded in September of 2021.
Statapult: All data was collected using the Statapult at the website https://sigmazone.com/catapult/. Figure 1 and Figure 2 were captured from this site as well.
General equations and techniques were provided by the Texas Tech IE5342 curriculum as taught by Dr. Timothy Matis. The text for this class was Design and Analysis of Experiments Eighth Edition, Doublas C Montgomery, Published by Wiley.