| trt | pre.leu | post.leu |
|---|---|---|
| Whey | 109 | 96.55 |
| Whey | 110.4 | 115.8 |
| Whey | 126.6 | 116.5 |
| Whey | 155.3 | 148.9 |
| Whey | 97.88 | 99.05 |
| Whey | 108.1 | 101.9 |
| Almond | 72.13 | 81.85 |
| Almond | 85.13 | 105.3 |
| Almond | 157 | 157 |
| Almond | 83.44 | 105.4 |
| Almond | 125.9 | 137.7 |
| Almond | 131.6 | 128.5 |
| Almond.F | 101.6 | 122.7 |
| Almond.F | 151.5 | 166.9 |
| Almond.F | 120.9 | 155.6 |
| Almond.F | 135.4 | 175.1 |
| Almond.F | 122.3 | 121.5 |
| Almond.F | 145.1 | 177.9 |
Protein quality study
Summary
This report illustrates recommended statistical analysis for client’s theses. As the data are currently incomplete, the suggested methods are illustrated using simulated data; clients are provided with copies of these simulated datasets along with this document to replicate in SPSS.
Background
Clients are analyzing data from a study in which participants were assigned a protein supplementation regimen and various response measures were recorded at baseline and again after a period of time on the regimen. The goal of the study is to compare changes from baseline among supplementation types.
Notes on study design: participants were recruited via flyer; all were female Cal Poly students. The protein supplementation types that define treatment groups exhibit factorial structure as combinations of protein type (Almond, Rice) and fortification (Fortified, Not Fortified). In addition, a control group (Whey protein) is included. Treatments were assigned on a rolling basis such that mean BMI did not differ among treatment groups. Thus treatment allocation was not completely randomized; this, together with volunteer participation, limits the scope of inference for the study. Care should be exercised not to interpret results in an over-broad manner.
Clients have divided the study data into two sets of treatment groups: Josie is analyzing data from the Almond, Fortified Almond, and Whey (control) groups; Mia is analyzing data from the Rice, Fortified Rice, and Whey (control) groups.
Recommendations
- Focus on analysis of the pairwise differences (post - pre) in response measurements as primary
- For each response, perform a standard one-way ANOVA with post-hoc inference of pairwise treatment contrasts using Tukey’s method or a similar adjustment
- If desired, perform parallel analyses using only the post value, depending on advisor recommendation and prior literature
- Interpret results in terms of associations rather than causal effects due to lack of randomization in treatment allocation
If clients consider publication in future, they may wish to do a single analysis of all treatments. This is similar to that illustrated below, except additional contrasts may be of interest to estimate factorial effects – that is, the marginal effects of protein type and fortification – which are not possible if the treatment groups are separated.
Example analyses with simulated data
Josie’s analysis
The simulation generated an effect of fortified almond protein on the change in leucine, but no effect for the other two treatments. This can be seen below.
Analysis should use pairwise differences in measurement levels (post - pre) as a response variable. Below this is stored as leu.ch. To compute these in SPSS, choose Transform > Compute Variable, input the desired column name (such as leu.ch below) in the Target variable field and the calculation (for instance leu.pre - leu.post below) in the Numeric Expression field using the column names listed at far left.
| trt | pre.leu | post.leu | leu.ch |
|---|---|---|---|
| Whey | 109 | 96.55 | -12.47 |
| Whey | 110.4 | 115.8 | 5.365 |
| Whey | 126.6 | 116.5 | -10.07 |
| Whey | 155.3 | 148.9 | -6.4 |
| Whey | 97.88 | 99.05 | 1.175 |
| Whey | 108.1 | 101.9 | -6.194 |
| Almond | 72.13 | 81.85 | 9.721 |
| Almond | 85.13 | 105.3 | 20.16 |
| Almond | 157 | 157 | -0.01627 |
| Almond | 83.44 | 105.4 | 22 |
| Almond | 125.9 | 137.7 | 11.85 |
| Almond | 131.6 | 128.5 | -3.173 |
| Almond.F | 101.6 | 122.7 | 21.18 |
| Almond.F | 151.5 | 166.9 | 15.37 |
| Almond.F | 120.9 | 155.6 | 34.72 |
| Almond.F | 135.4 | 175.1 | 39.71 |
| Almond.F | 122.3 | 121.5 | -0.7807 |
| Almond.F | 145.1 | 177.9 | 32.75 |
A standard one-way ANOVA on the pairwise differences by treatment group should be used to infer an effect of treatment and treatment contrasts. To perform this in SPSS:
- choose Analyze > Compare Means and Proportions > One-Way ANOVA
- select the response (change in measurement level) for the Dependent List field
- select the treatment for the Factor field (SPSS requires that the treatment be encoded numerically as, e.g., 1, 2, 3, 4)
- click the Post Hoc … button and add the Tukey option and select Continue
- click the Options … button and add Descriptive and Means plot and select Continue
- select Ok
This should generate an ANOVA table:
| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| trt | 2 | 2453 | 1227 | 9.74 | 0.001945 |
| Residuals | 15 | 1889 | 125.9 | NA | NA |
Interpretation:
The data provide evidence of an association between protein supplementation type and mean change in leucine (F = 9.47 on 2 and 15 degrees of freedom, p = 0.001945).
Following this will be a table of effect size measures – think of these as measures of association (owing to the nonrandom treatment allocation). I usually focus on the \(eta^2\) statistic (Eta-squared), but SPSS will display other measures.
# Effect Size for ANOVA
Parameter | Eta2 | 95% CI
-------------------------------
trt | 0.56 | [0.15, 0.75]Point estimate interpretation:
An estimated 56% of variation in change in leucine is associated with protein supplementation type.
Interval interpretation:
With 95% confidence, an estimated 15% to 75% of variation in change in leucine is associated with protein supplementation type.
Next SPSS will return post-hoc inference of treatment mean contrasts. This is shown below.
| contrast | estimate | std.error | df | statistic | adj.p.value |
|---|---|---|---|---|---|
| Whey - Almond | -14.85 | 6.479 | 15 | -2.293 | 0.08781 |
| Whey - Almond.F | -28.59 | 6.479 | 15 | -4.413 | 0.001376 |
| Almond - Almond.F | -13.73 | 6.479 | 15 | -2.12 | 0.1193 |
The adjusted p-value will be shown as Sig. (for “significance”). Test interpretations:
The mean change in leucine on the fortified almond supplementation differs significantly from the control (T = -4.413 on 15 degrees of freedom, p = 0.001376), but the mean change in leucine does not differ significantly between either the almond and fortified almond (T = -2.12 on 15 degrees of freedom, p = 0.1193) or the almond and control supplementation types (T = -2.293 on 15 degrees of freedom, p = 0.08781).
It is common to report only the significant contrasts and indicate that all treatment levels were compared. For those, one should also look at the confidence intervals to estimate the magnitude of difference between treatment groups.
| contrast | estimate | SE | df | lower.CL | upper.CL |
|---|---|---|---|---|---|
| Whey - Almond | -14.85 | 6.479 | 15 | -31.68 | 1.974 |
| Whey - Almond.F | -28.59 | 6.479 | 15 | -45.42 | -11.76 |
| Almond - Almond.F | -13.73 | 6.479 | 15 | -30.56 | 3.095 |
Example interpretation:
With 95% confidence, the mean change in leucine (post - pre) on the fortified almond supplementation is estimated to be between 11.76 and 45.42 units higher than the mean change on the whey supplementation (control).
Mia’s analysis
The analysis for Mia’s treatments is parallel to that shown above. Here are the simulated data:
| trt | pre.leu | post.leu |
|---|---|---|
| Whey | 109 | 96.55 |
| Whey | 110.4 | 115.8 |
| Whey | 126.6 | 116.5 |
| Whey | 155.3 | 148.9 |
| Whey | 97.88 | 99.05 |
| Whey | 108.1 | 101.9 |
| Rice | 133.7 | 120.8 |
| Rice | 72.78 | 76.01 |
| Rice | 132.3 | 146.4 |
| Rice | 95.96 | 102.8 |
| Rice | 99.05 | 113.6 |
| Rice | 73.22 | 66.88 |
| Rice.F | 97.09 | 113.3 |
| Rice.F | 146 | 160.5 |
| Rice.F | 119.4 | 159.8 |
| Rice.F | 88.5 | 114.7 |
| Rice.F | 92.14 | 108.7 |
| Rice.F | 145.7 | 153.8 |
Again, an effect of fortification is simulated with no other treatment effects.
The pairwise differences are shown below. Analysis is performed on the pairwise differences leu.ch. To compute these in SPSS, choose Transform > Compute Variable, input the desired column name (such as leu.ch below) in the Target variable field and the calculation (for instance leu.pre - leu.post below) in the Numeric Expression field using the column names listed at far left.
| trt | pre.leu | post.leu | leu.ch |
|---|---|---|---|
| Whey | 109 | 96.55 | -12.47 |
| Whey | 110.4 | 115.8 | 5.365 |
| Whey | 126.6 | 116.5 | -10.07 |
| Whey | 155.3 | 148.9 | -6.4 |
| Whey | 97.88 | 99.05 | 1.175 |
| Whey | 108.1 | 101.9 | -6.194 |
| Rice | 133.7 | 120.8 | -12.87 |
| Rice | 72.78 | 76.01 | 3.231 |
| Rice | 132.3 | 146.4 | 14.1 |
| Rice | 95.96 | 102.8 | 6.871 |
| Rice | 99.05 | 113.6 | 14.5 |
| Rice | 73.22 | 66.88 | -6.34 |
| Rice.F | 97.09 | 113.3 | 16.25 |
| Rice.F | 146 | 160.5 | 14.44 |
| Rice.F | 119.4 | 159.8 | 40.43 |
| Rice.F | 88.5 | 114.7 | 26.17 |
| Rice.F | 92.14 | 108.7 | 16.61 |
| Rice.F | 145.7 | 153.8 | 8.071 |
A standard one-way ANOVA on the pairwise differences by treatment group should be used to infer an effect of treatment and treatment contrasts. To perform this in SPSS:
- choose Analyze > Compare Means and Proportions > One-Way ANOVA
- select the response (change in measurement level) for the Dependent List field
- select the treatment for the Factor field (SPSS requires that the treatment be encoded numerically as, e.g., 1, 2, 3, 4)
- click the Post Hoc … button and add the Tukey option and select Continue
- click the Options … button and add Descriptive and Means plot and select Continue
- select Ok
This should generate an ANOVA table:
| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| trt | 2 | 1971 | 985.6 | 9.902 | 0.001813 |
| Residuals | 15 | 1493 | 99.53 | NA | NA |
Interpretation:
The data provide evidence of an association between protein supplementation type and mean change in leucine (F = 9.902 on 2 and 15 degrees of freedom, p = 0.001813).
Following this will be a table of effect size measures – think of these as measures of association (owing to the nonrandom treatment allocation). I usually focus on the \(eta^2\) statistic (Eta-squared), but SPSS will display other measures.
# Effect Size for ANOVA
Parameter | Eta2 | 95% CI
-------------------------------
trt | 0.57 | [0.16, 0.76]Point estimate interpretation:
An estimated 57% of variation in change in leucine is associated with protein supplementation type.
Interval interpretation:
With 95% confidence, an estimated 16% to 76% of variation in change in leucine is associated with protein supplementation type.
Next SPSS will return post-hoc inference of treatment mean contrasts. This is shown below.
| contrast | estimate | std.error | df | statistic | adj.p.value |
|---|---|---|---|---|---|
| Whey - Rice | -8.013 | 5.76 | 15 | -1.391 | 0.3701 |
| Whey - Rice.F | -25.09 | 5.76 | 15 | -4.356 | 0.001538 |
| Rice - Rice.F | -17.08 | 5.76 | 15 | -2.965 | 0.02461 |
The adjusted p-value will be shown as Sig. (for “significance”). Test interpretations:
The mean change in leucine on the fortified rice supplementation differs significantly from the control (T = -4.356 on 15 degrees of freedom, p = 0.001538) and the mean change in leucine on the fortified rice supplementation differs from the (non-fortified) rice supplementation (T = -2.965 on 15 degrees of freedom, p = 0.02461). However, the rice and control supplementation types do not differ significantly (T = -1.391 on 15 degrees of freedom, p = 0.3701).
It is common to report only the significant contrasts and indicate that all treatment levels were compared. For those, one should also look at the confidence intervals to estimate the magnitude of difference between treatment groups.
| contrast | estimate | SE | df | lower.CL | upper.CL |
|---|---|---|---|---|---|
| Whey - Rice | -8.013 | 5.76 | 15 | -22.97 | 6.948 |
| Whey - Rice.F | -25.09 | 5.76 | 15 | -40.05 | -10.13 |
| Rice - Rice.F | -17.08 | 5.76 | 15 | -32.04 | -2.119 |
Example interpretation:
With 95% confidence, the mean change in leucine (post - pre) on the fortified rice supplementation is estimated to be between 10.13 and 40.05 units higher than the mean change on the whey supplementation (control).