In Module 2, we are learning how to compare the means of two populations. Because we cannot know the value of the mean of a population, we need to base this comparison on sample means.
Some ideas we need to be clear about to understand how to compare the means of two populations (we discussed some of these ideas in the simulation example we did during the Review of Stat 1):
The sample mean is a variable (it changes from sample to sample); thus it is different from the population mean which is a constant.
If you take multiple samples from the same variable, the means of these samples will be different.
If 2) is true, then, this is also true: The sample means taken from two variables with the same population mean WILL BE DIFFERENT!
Corollary: We CANNOT claim that the means of two variables are different simply because sample means taken from these variables are different!
Two machines are used for filling plastic bottles. The quality engineering department suspects that there is a difference in the volume produced by these machines. The QE department wanted to make conclusions about this situation and for this purpose took a random sample from the output of these machines. The volume (in fluid ounces) obtained from each machine are stored in the following two vectors:
vol_mach1= c(15.98, 16.05, 16.07, 16.05, 16.08, 16.10, 15.96, 15.98, 16.10, 15.98)
vol_mach2= c(16.02, 15.95, 15.91, 16.02, 15.99, 16.01, 16.01, 16.02, 15.96, 16.03)How many elements does each sample contain?
length (vol_mach1)## [1] 10length (vol_mach2)## [1] 10Compute the mean for each sample in the following code cells. Round to two decimal places:
# Sample mean for machine 1
round(mean(vol_mach1),2)## [1] 16.04# Sample mean for machine 2
round(mean(vol_mach2),2)## [1] 15.99The average volume from the samples is different. However, that does not mean that if we compare the average from both populations (or even the average from really large samples (e.g., samples of more than 10,000 bottles)) we will observe such a difference.
We need a statistical procedure to tell us whether the observed sample difference is statistically significant. In other words, we need a procedure that tells us if the difference observed in the sample can be reasonably assumed to exist at the population level too (or in the long run).
Obtain a 95% CI for the difference between the mean volume for machine 1 and the mean volume for machine 2
# We set var.equal = TRUE because we assume that both populations have the same standard deviation.
# See slide 9 from the presentation.
t.test (vol_mach1, vol_mach2, var.equal = TRUE, conf.level = 0.95)$conf.int## [1] -0.001737528 0.087737528
## attr(,"conf.level")
## [1] 0.95This is the interval that we get:
(-0.0017, 0.0877)
What does the CI tell you about the difference between the mean volume for machine 1 and the mean volume for machine 2?
If you want a summary of what it tells you, this is the summary:
If the CI includes zero, we conclude that the two means are equal (the difference between the sample means is NOT statistically significant).
If the CI does not include zero, we conclude that the two means are different (the difference between the sample means is statistically significant).
What is the rationale behind this? READ THE RATIONALE INDEPENDENTLY IF INTERESTED!
A couple of points to consider when interpreting a CI:
Contain = The true difference between the two means is within the lower and upper limit of the interval.
If A = B, then A - B = 0
Therefore, if the CI that we compute DOES NOT include zero within its limits, we conclude that our assumption is not backed up by the data… And again, our assumption is that the means are equal (the difference between them is zero).
Note: “we conclude” does not mean “we know” or “we are sure of”… It is just a logical conclusion based on the analytic strategy we are following. But remember that in Statistics, there is always the possibility that our conclusion is incorrect.
In this example, the 95 % confidence interval includes zero. Therefore, we conclude that the difference between the mean volume for machine 1 and the mean volume for machine 2 is zero (at the population level). Based on the 95% CI, we conclude that the population means are the same. So, the difference between the samples means is NOT statistically significant, which means that we should NOT expect a difference between the means at the population level (or in the long run).
What does 95% confidence level mean?
It means that 95% of the confidence intervals (in plural) that we compute using this formula will contain the true difference between the two means.
Therefore, for example, if we take 1000 samples and compute a CI with each sample, we can expect that 950 of those intervals will contain the true difference between the two means. So, we expect 50 intervals to miss the target (= not to contain the true difference within its limits)
Obtain a 99% CI for the difference between the mean volume for machine 1 and the mean volume for machine 2. What’s the effect of increasing the confidence level on the interval ?
t.test (vol_mach1, vol_mach2, var.equal = TRUE, conf.level = 0.99)$conf.int## [1] -0.01829419 0.10429419
## attr(,"conf.level")
## [1] 0.99The 95% CI is (-0.0017, 0.0877)
The 99% CI is (-0.0183, 0.1043)
Increasing the confidence level from 95 to 99 has resulted in an increase of the interval width. The interval has become wider; therefore, we can be more confident that the true difference is inside the interval. However, a wider interval means it is less precise (your estimation has less precision). There is a trade off between confidence and precision.
Is there any way to tilt the trade off on our side? Can we keep a good confidence level and have a good precision?
Yes, we can keep a high confidence (95% or 99%) and still reduce the width of the interval (and make it more precise) by increase the sample sizes (in this example, this means taking more plastic bottles to analyze)