ME488 Catapult Design and Analysis
Johnny Froehlich, Mimi Shang, Pete Ahbram
June 8, 2016
Summary
This experiment was conducted by a group of students to practice design of experiments and statistical process control. The experiment was ran using a catapult with several controllable factors each with two levels. The primary objective was to determine the effect of launch angle, pull distance, spring constant, and air-flow factors on the distance the projectile launches. The secondary objective was to determine the settings required to reach a specified set of distances; Maximum distance and 50%, 25% and 10% of the maximum distance. The final regression model provided an adjusted R-squared value of 0.998, an F-statistic of 1114 and a p-value of 1.903E-6. These results, along with a full step-wise regression analysis and complete interaction effect evaluation indicated a descriptive and predictive model that agreed with our initial intuition and assumptions. The maximum range of the launcher was approximately 120 inches and requires settings; X1(LOW), X2(HIGH), X3(LOW), and X4(LOW).
Design Overview
Build Design
The projectile is fired using a spring-loaded tube launcher (see Figure 1). The design consist of an internal compression spring and holster mechanism. The user applies a load (P) to the spring by pulling back on the handle a distance (δ). Once released, the generated spring force (Fs) launches the projectile a distance (x).
Figure 1: Sketches of initial design ideas for tube launcher. (Left) Sketch of final design. (Right)
Description of Experiment: Response and Factors
Objective: To determine the effect of launch angle, pull distance, spring constant, and air-flow factors on on the distance the projectile launches.To determine the settings required to reach a specified distance. To determine the settings required to maximize distance.
Response variable: The distance (xi-x0) in feet covered by the projectile for a givin number of settings (= mean (over 16 repetitions) of the projectile distance).
Number of observations = 32 (Balanced 24, 25 factorial design with 2 replicates)
Variables:
Response Variable Y = Mean (over 32 reps) of projectile distance
Factor 1 = Angle (Angle of barrel at start position as measured from the ground). Levels: LOW (19.5 degrees) and HIGH (45 degrees)).
Factor 2 = Pull Distance (The distance in inches the launch handle is extended from its start position). Levels:: LOW (6 in) and HIGH (9 in)).
Factor 3 = Spring Constant. Levels: LOW (0.6875 lbs/in) and HIGH (0.75 lbs/in))
Factor 4 = Air Flow. Levels: LOW (0.5 (50%)) and HIGH (1 (100%)) ***
Experimental Design
The coded design matrix for all 16 runs is shown below. The actual randomized run order is given in the last column.
| Run | Angle | Pull.Distance | Spring | Air.Flow | Run.Order |
|---|---|---|---|---|---|
| 1 | -1 | 1 | 1 | -1 | 5 |
| 2 | -1 | -1 | 1 | -1 | 1 |
| 3 | -1 | -1 | -1 | -1 | 17 |
| 4 | 1 | -1 | -1 | -1 | 6 |
| 5 | -1 | 1 | -1 | -1 | 16 |
| 6 | 1 | 1 | -1 | -1 | 8 |
| 7 | 1 | -1 | 1 | -1 | 19 |
| 8 | 1 | -1 | 1 | -1 | 7 |
| 9 | 1 | 1 | -1 | 1 | 18 |
| 10 | -1 | 1 | 1 | 1 | 10 |
| 11 | 1 | -1 | 1 | 1 | 20 |
| 12 | -1 | -1 | -1 | 1 | 9 |
| 13 | 1 | 1 | 1 | 1 | 3 |
| 14 | -1 | 1 | -1 | 1 | 13 |
| 15 | -1 | -1 | 1 | 1 | 11 |
| 16 | 1 | -1 | -1 | 1 | 15 |
The complete design matrix, with measured travel distance responses and initial factor level settings, appears below. The actual randomized run order is given in the last column. (Serves as original data sheet)
| angle | pull | srate | air | travel | order |
|---|---|---|---|---|---|
| 19.5 | 9 | 0.75 | 0.5 | 73.00 | 5 |
| 19.5 | 6 | 0.75 | 0.5 | 41.50 | 1 |
| 19.5 | 6 | 0.6875 | 0.5 | 63.50 | 17 |
| 45 | 6 | 0.6875 | 0.5 | 57.25 | 6 |
| 19.5 | 9 | 0.6875 | 0.5 | 118.00 | 16 |
| 45 | 9 | 0.6875 | 0.5 | 100.00 | 8 |
| 45 | 6 | 0.75 | 0.5 | 26.50 | 19 |
| 45 | 9 | 0.75 | 0.5 | 64.00 | 7 |
| 45 | 9 | 0.6875 | 1 | 111.50 | 18 |
| 19.5 | 9 | 0.75 | 1 | 68.00 | 10 |
| 45 | 6 | 0.75 | 1 | 30.00 | 20 |
| 19.5 | 6 | 0.6875 | 1 | 67.00 | 9 |
| 45 | 9 | 0.75 | 1 | 75.50 | 3 |
| 19.5 | 9 | 0.6875 | 1 | 116.50 | 13 |
| 19.5 | 6 | 0.75 | 1 | 46.50 | 11 |
| 45 | 6 | 0.6875 | 1 | 62.00 | 15 |
Design Concerns
1. Friction: Measurable but not controllable. No anlalysis has been done with regards to barrel friction and its effects on launch distance/height. The design uses an internal spring mechnasim attached to a cup with an outside diameter that is less than the inner diameter of the tubing. All the internal parts do not make contact with the barrel walls when in motion.
2. Aerodynamics/Drag: Not Measurable and Not controllable. While the barrel exit has an adjustable hole pattern to controll exit speed, weather effects, and other external enviornmental effects cannot be controlled. The fractional factorial design will allow for a second run.
3. Part Failure
Other than the direct adjustments made to the apperatus, these concerns are mitigated through the application of fully randomized runs and a 2-replicate design. It will be shown in the analysis, that a Box Wilson transformation and summary were conducted to ensure that any fluctuations in variance with respect to travel distance could still provide residuals based on our assumed properties, and not those of external effects. Note however, that this is a saftey measure with respect to the data during analysis. This does not mean that Box Wilson tranformations can eliminate the faulty data taken during a random lightning strike.
Experimental Methods
The experiment was operated on level ground during a clear weather day. Prior to taking run data a distance of 20 feet was measured from the barrel end in half foot increments and marked using chalk. Marks were also added to the threaded pull rod in 1 inch increments over its entire length. The air flow was controlled using an adjustable slip-on rubber seal, where LOW/HIGH settings represented 8 holes open and 16 holes open respectivly. The process of collecting run data consisted of one member sitting behind the launcher and operating the pull distance, another member setting the correct angle and air flow levels, and another observing the flight path and recording the stopping distance.
Post Analysis
We started by plotting the response data several ways to see if any trends or anomalies appear that would not be accounted for by the standard linear response models. First, we looked at the distribution of the response variable regardless of factor levels by generating the following four plots. The first plot is a normal probability plot of the response variable. The red line is the theoretical normal distribution. The second plot is a box plot of the response variable. The third plot is a histogram of the response variable.The fourth plot is the response versus the run order.
Clearly there is “structure” that we hope to account for when we fit a response model. For example, the response variable is separated into two roughly equal-sized clumps in the histogram. The first clump is centered approximately around the value 60 inches while the second clump is centered approximately around the value 110 inches.
Note: The run-order distribution plot does not indicate a significant time effect.
Next, we look at box plots of the response for each factor.
Figure 2: Boxplots of travel response for each factor
Several factors, most notably “spring rate” followed by “pull distance” and possibly “angle”, appear to change the average response level.
For a 24 full factorial experiment we can fit a model containing a mean term, four main effect terms, six two-factor interaction terms, four three-factor interaction terms, and a four-factor interaction term (16 parameters). Initialy, we are assuming the four-factor interaction term is non-existent. While this term could be significant in the design (very rare), it is difficult to interpret from an engineering standpoint. Using these assumtions, we accumulate the sums of squares for each term and use them to estimate an error term. We begin with a theoretical model including the 15 (interaction terms) unknown constants, hoping to determine the significant effects for use in the final model.
The ANOVA table for the 15-parameter model up to third order interactions follows. (Intercept not shown)
Summary of Fit: - R-squared 0.9998 - R-adj 0.997 - Observations: 16
| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| angle | 1 | 282.6601562 | 282.6601562 | 115.7776 | 0.0589959 |
| srate | 1 | 4581.5976562 | 4581.5976562 | 1876.6224 | 0.0146931 |
| pull | 1 | 6899.3789062 | 6899.3789062 | 2825.9856 | 0.0119741 |
| air | 1 | 69.0976562 | 69.0976562 | 28.3024 | 0.1182851 |
| angle:srate | 1 | 0.0976562 | 0.0976562 | 0.0400 | 0.8743341 |
| angle:pull | 1 | 20.8164062 | 20.8164062 | 8.5264 | 0.2100508 |
| angle:air | 1 | 53.4726562 | 53.4726562 | 21.9024 | 0.1340145 |
| srate:pull | 1 | 226.8789063 | 226.8789063 | 92.9296 | 0.0658040 |
| srate:air | 1 | 0.6601563 | 0.6601563 | 0.2704 | 0.6947285 |
| pull:air | 1 | 0.0039063 | 0.0039063 | 0.0016 | 0.9745488 |
| angle:srate:pull | 1 | 108.9414062 | 108.9414062 | 44.6224 | 0.0945999 |
| angle:srate:air | 1 | 0.0351563 | 0.0351563 | 0.0144 | 0.9239692 |
| angle:pull:air | 1 | 55.3164063 | 55.3164063 | 22.6576 | 0.1318266 |
| srate:pull:air | 1 | 0.8789063 | 0.8789063 | 0.3600 | 0.6559583 |
| Residuals | 1 | 2.4414062 | 2.4414062 | NA | NA |
This fit has a large R2 (= 0.9998) and adjusted R2 (= 0.997). However, the large number of p-values (> 0.05) show the model as having many unnecessary terms.
Initial Regression Model Output based on Experimental Design: (Intercept (= 63.12) not included)
Model before elimination:
Step-wise Regression
Starting with the 15 terms, we preformed stepwise regression to eliminate unnecessary terms. Using a combination of stepwise regression and the elimination of remaining terms with a p-value larger than 0.05, we obtain a model with an intercept and 10 significant effect terms.
| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| angle | 1 | 282.6601562 | 282.6601562 | 228.2681388 | 0.0043522 |
| srate | 1 | 4581.5976562 | 4581.5976562 | 3699.9652997 | 0.0002702 |
| pull | 1 | 6899.3789062 | 6899.3789062 | 5571.7381703 | 0.0001794 |
| air | 1 | 69.0976562 | 69.0976562 | 55.8012618 | 0.0174530 |
| angle:srate | 1 | 0.0976562 | 0.0976562 | 0.0788644 | 0.8052277 |
| angle:pull | 1 | 20.8164062 | 20.8164062 | 16.8107256 | 0.0546547 |
| angle:air | 1 | 53.4726562 | 53.4726562 | 43.1829653 | 0.0223827 |
| srate:pull | 1 | 226.8789063 | 226.8789063 | 183.2208202 | 0.0054136 |
| srate:air | 1 | 0.6601563 | 0.6601563 | 0.5331230 | 0.5412400 |
| pull:air | 1 | 0.0039063 | 0.0039063 | 0.0031546 | 0.9603162 |
| angle:srate:pull | 1 | 108.9414062 | 108.9414062 | 87.9779180 | 0.0111763 |
| angle:pull:air | 1 | 55.3164063 | 55.3164063 | 44.6719243 | 0.0216608 |
| srate:pull:air | 1 | 0.8789063 | 0.8789063 | 0.7097792 | 0.4882066 |
| Residuals | 2 | 2.4765625 | 1.2382812 | NA | NA |
After the removal of the angle:srate, srate:air, pull:air, and srate:pull:air interactions, we obtain a model with an intercept and 9 significant effect terms.
| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| angle | 1 | 282.660156 | 282.660156 | 281.56031 | 0.0000739 |
| srate | 1 | 4581.597656 | 4581.597656 | 4563.77043 | 0.0000003 |
| pull | 1 | 6899.378906 | 6899.378906 | 6872.53307 | 0.0000001 |
| air | 1 | 69.097656 | 69.097656 | 68.82879 | 0.0011526 |
| angle:pull | 1 | 20.816406 | 20.816406 | 20.73541 | 0.0103891 |
| angle:air | 1 | 53.472656 | 53.472656 | 53.26459 | 0.0018741 |
| srate:pull | 1 | 226.878906 | 226.878906 | 225.99611 | 0.0001141 |
| angle:srate:pull | 2 | 109.039062 | 54.519531 | 54.30739 | 0.0012616 |
| angle:pull:air | 2 | 55.320312 | 27.660156 | 27.55253 | 0.0045801 |
| Residuals | 4 | 4.015625 | 1.003906 | NA | NA |
The following table shows the 95% confidence intervals for the final regression model.
| 2.5 % | 97.5 % | |
|---|---|---|
| (Intercept) | 60.715836 | 65.534164 |
| angle45 | -8.969572 | -2.155428 |
| srate0.75 | -24.031863 | -18.468137 |
| pull9 | 52.342928 | 59.157072 |
| air1 | 1.468137 | 7.031863 |
| angle45:pull9 | -18.130827 | -8.494173 |
| angle45:air1 | -4.059148 | 3.809148 |
| srate0.75:pull9 | -29.434148 | -21.565852 |
| angle45:srate0.75:pull6 | -14.059148 | -6.190852 |
| angle45:srate0.75:pull9 | 6.815852 | 14.684148 |
| angle19.5:pull9:air1 | -11.434148 | -3.565852 |
| angle45:pull9:air1 | 3.440852 | 11.309148 |
Normal Plot of Effects
Non-significant effects should follow an approximately normal distribution with the same location and scale. Significant effects will vary from this normal distribution. Therefore, we used the normal probability method as another way of determining significant effects. The effects that are shown to deviate from the straight line fit to the data are considered significant (somtimes subjective)
A normal probability plot of the effects is shown below. The effects we consider to be significant are labeled. In this case, we have obtained the same 9 terms from the normal probability plot as we did when applying step-wise regression.
Most of the effects group close to the center (zero) line and follow the fitted normal model straight line. The effects that appear to be above or below the line by more than a small amount are the same significant effects identified using the stepwise routine. Because the largest effects were highly significant, we could not remove them as a way to re-scale the y-axis. For this reason, it is difficult to see the significant effects twoards the center. Each label was checked and graphed individualy to confirm.
At this point, the model appears to account for most of the variability in the response. The final 9 term model achieves an adjusted R2 (=0.998), an F-statistic (= 1114) and a p-value (= 1.903e-06). All the main effects are significant, as are three 2-factor interactions and two 3-factor interactions.
Final regression model
Note: Second and third order interactions are subjectivly negligible. These effects are modeled, and may be referenced within the provided rmd file.
Analysis of Experiment
In order to obtain proper interaction plots, we first looked at wether the model could be transformed by the response variable and still obtain residuals with the assumed properties. We calculated an optimum Box-Cox transformation by finding the value of lambda that maximizes the negative log likelihood.
The optimum is found at lambda = 1.03. While this step was unnecessary, If we were to do a further analysis the boxcox tranformation would help us to resolve the problem of increasing variance with increasing travel distance. Since our catapult can barely function as it is, the effects are negligible. However, before any model is accepted, it is good form to evaluate the risiduals. This was a more streamed line way of doing so.
Plots of the main effects and the significant 2-way interactions are shown below.
The magnitudes of the effect estimates show that “Spring Rate” is by far the most important factor. “Angle” plays the next most critical role, followed by “Pull Distance”. Air Flow is shown to have the least amount of effect on travel distance. Note that large interactions can obscure main effects.
2-way interaction plot showing means for all combinations of levels for the two factors.
The labels located in the diagonal spaces of the plot grid have two purposes. First, the label indicates the factor associated with the x-axis for all plots in the same row. Second, the label indicates the factor defining the two lines for plots in the same column. The LOW levels of each factor are shown in blue, with coresponding HIGH level shown in red.
Conclusion
Factor Level Settings
Based on the full analysis, we can select factor settings that maximize projectile distance.
1. Maximum Projectile Distance Settings
- Angle (LOW)
- Pull Distance (HIGH)
- Spring Rate (LOW)
- Air Flow (LOW)
2. 50% Projectile Distance
- Angle (HIGH)
- Pull Distance (LOW)
- Spring Rate (LOW)
- Air Flow (LOW)
3. 25% Projectile Distance
- Angle (HIGH)
- Pull Distance (LOW)
- Spring Rate (HIGH)
- Air Flow (HIGH)
4. 10% Projectile Distance
- Angle (HIGH)
- Pull Distance (LOW)
- Spring Rate (HIGH)
- Air Flow (HIGH)
Review of Experiment
All-in-all the experiment ran smoother than others in the past. If we would have assumed that three-factor and higher interactions were negligible before experimenting, a 25-1, resolution V, half-fraction design might have been chosen with the additional factor being the wieght of the projectile. This design would have had three centerpoints and using response surface methods is an attractive idea. From our results, there does not appear to be a pattern to the residuals. When observing a single point, it is clear the model performs poorly in predicting a short distance. Because of the poor selection of springs, large distances where un-obtainable and the fact that the model predicts an impossible negative distance is an obvious shortcoming of the model and design. More attention may need to be given to distances below 40 inches. While we are confident in the results obtained, there is still room for a further analysis of the interaction terms. The model and fitting process, as illustrated in this analysis, is often an iterative process.