In this computer lab we will concentrate on interval estimation, in particular confidence intervals for normally-distributed data. We have covered the theory of this in the lectures. We will study confidence intervals for the population mean.

Aims and intended learning outcomes

The aim of this lab is to study confidence intervals for a population mean. By the end of this computer lab you should be able to:

The main new R functions we will use for are t.test(),qqnorm() and qqline().

Section A - One sample confidence interval

Exercise A1 - IQ in female psychiatric patients

Context: A sample of 39 female psychiatric patients had their IQ determined.

Question: Is the average IQ of female psychiatric patients significantly lower than that of the general population (100)? If so, by how much is it likely to be lower?

Data: The data are available from a worksheet iq.csv. This worksheet contains a single column of data containing the IQ scores of the female psychiatric patients.

It would be helpful to have more information on these women in order to determine the nature of the population they could be taken to represent; it may be reasonable to suppose that the females are representative of other similar psychiatric patients. We assume that the sample is representative of a wider population of similar female psychiatric patients.

Data Exploration:

First, download the data files required for this lab and then launch RStudio. After this, set the working directory to where the files are saved.

Next load the IQ data:

IQdata <- read.csv(file="IQ.csv")

Inspect it by looking in the Environment tab (top right) and clicking on IQdata. Notice it is stored as a single column called “iq”.

Obtain a histogram of the data:

hist(IQdata$iq)

R has made a choice of the number of bins which is very small, so we don’t get a very detailed picture of the sample. We can suggest a different number of bins:

hist(IQdata$iq, breaks=10)

(although R will not necessarily use exactly this value, just a value close to it). Use the main="" and xlab="" options (from Lab 1) to add an informative title and axis label.

hist(IQdata$iq, breaks = 10,xlab="IQ",main="Histogram of 39 IQ measurements")

From the histogram, it appears that the IQ of the female psychiatric patients is generally lower than 100. To include a vertical line at IQ of 100 use:

hist(IQdata$iq, breaks = 10,xlab="IQ",main="Histogram of 39 IQ measurements")
abline(v=100, col=2)

Now consider some summaries and a boxplot of the data:

summary(IQdata$iq)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   82.00   88.50   91.00   92.18   95.50  105.00
sd(IQdata$iq)
## [1] 5.538724
boxplot(IQdata$iq, main="Boxplot of IQ sample")

These allow us to produce an informal answer to the question of interest. It appears that the average IQ for female psychiatric patients is less than the general population average of 100 (as judged either by the mean or the median).

The approximate symmetry of the boxplot and the descriptive statistics suggest that the data comes from a Normal distribution with mean about 92.2 and standard deviation about 5.5.

However, we must carry out a formal analysis of the data before any conclusion as to the average IQ of female psychiatric patients in the wider population can be drawn.

Formal Analysis

This can be achieved by using the sample of data to produce a one sample Normal 95% confidence interval for the population mean. This provides an interval estimate of the range of plausible values of the population mean IQ of female psychiatric patients. If this 95% confidence interval contains the population average (i.e., 100), then it is plausible that there is no difference between average IQs of female psychiatric patients and the general population.

Since the population standard deviation is unknown we use the t distribution to produce the CI. In order to use this procedure we need to check some modelling assumptions. These are:

  1. the IQ scores are a random sample from the population;

  2. the IQ scores approximately follow a Normal probability model.

The assumption that the population can be described by a normal distribution has been informally assessed based on the boxplot/histogram. Formally there are various ways to check if this plausible. One is via a normal probability plot or a Q-Q plot. The commands are:

qqnorm(IQdata$iq) # to plot the points
qqline(IQdata$iq) # to draw the line

If the points falls close to the line and don’t appear to curve too much, this is good evidence of normally distributed data. This is more-or-less the case here (with just a little departure from the line in the right tail). So we can proceed with the assumption of normality.

The 95% CI is obtained from the t.test() function:

t.test(IQdata$iq)
## 
##  One Sample t-test
## 
## data:  IQdata$iq
## t = 103.93, df = 38, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  90.38404 93.97493
## sample estimates:
## mean of x 
##  92.17949

Much of the output is not important for the time being: just focus on the confidence interval output.

Conclusion

Since the 95% confidence interval for the mean IQ of the female psychiatric population ranges from 90.4 to 94.0 (rounding to 1.d.p.), it does not include the value 100 (the general population average). Therefore we may conclude that female psychiatric patients very likely have a different mean IQ from 100. Since the confidence interval includes only values less than 100, we conclude that their IQ is lower, on average, than the general population. It is lower by between 6 (100-94) and 10 (100-90) IQ points.

Exercise A2 - Alcohol Relapse

Context: In an investigation into the causes of relapse amongst alcoholics, representative samples of long-standing alcoholics were taken and had various aspects of their personal history and previous troubles with heavy drinking recorded. In particular, individuals were scored for ‘relapse precipitants’, which included a number of psychological measures such as ‘unpleasant mood states’ (UM).

Questions: Does the assumption of Normality for UM score seem reasonable? What is the population mean UM score of long-standing alcoholics? Provide an interval estimate of the population mean.

Data: This is held in a CSV file called LONGALCOHOL.csv. It contains the UM score in a column labelled UM.

Work through this context by yourself. Ensure you answer the questions by following the layout of Exercise A1.

Section B - Transformations for Non-Normality

When it is unreasonable to assume that the underlying sample has arisen from a Normal distribution, one can transform the scale of measurement of the data so that the transformed variable appears much more Normal.

When the data is heavily skewed to the right ( i.e. a number of ‘large’ values but few ‘small’ values of the variable of interest – which is quite common when one is measuring a variable that takes only positive values), two of the most common transformations used are:

In R the commands log() and sqrt() produce such transformations.

The results of any analysis (such as producing confidence intervals for the population mean) are obtained by carrying out the analysis on the transformed variable and then using the inverse transformation on the results back into the original scale of measurement for the `final answer’.

The inverse transformations are:

In R exp() and ..^2 are the commands for such inverse transformations.

Example B1 - Tensile Strength

Context: There are many factors that affect the strength and durability of fabrics including the weave and material used. One measure of strength is the tensile strength. This is the maximum amount of tensile stress that a material can take before failure, for example breaking or tearing. 30 polyester fibres.

Questions: Does the assumption of Normality for tensile strength seem reasonable? What is the population mean tensile strength of polyester firbres? Provide an interval estimate.

Data: This is held in a CSV file called tensile.csv.

Data Description

Load the tensile data:

tensile <- read.csv(file="tensile.csv")
tensile <- tensile$strength

Inspect it by looking in the “Workspace” tab and clicking on “tensile”: Obtain a histogram of the data:

hist(tensile, breaks=10, main="Histogram of Polyester Tensile Strength", xlab="Tensile Strength")

From the histogram, the distribution does not appear symmetric. The data is right skewed. Note however that from a boxplot this is much more difficult to assess.

boxplot(tensile, main="Boxplot of Polyester Tensile Strength",ylab="Tensile Strength")

summary(tensile)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.0230  0.1323  0.3360  0.3659  0.5265  0.9260
sd(tensile)
## [1] 0.2685371

However it is important to present and comment informally on the question of interest based on the plots so far and the summary statistics

Let us consider a log transformation of the data

ltensile <-log(tensile)
par(mfrow=c(1,2))
hist(ltensile, xlab="Log Tensile Strength", main="Histogram")
boxplot(ltensile, ylab="Log Tensile Strength", main="Boxplot")

Comment on normality of the log transformed tensile strength

Let’s instead consider the square root of each of the measurements.

stensile <-sqrt(tensile)
par(mfrow=c(1,2))
hist(stensile, xlab="Square Root(Tensile Strength)",
main="Histogram")
boxplot(stensile, ylab="Square Root(Tensile Strength)",
main="Boxplot")

here the distribution appears to be far more symmetric than before. Let’s check the summary statistics

summary(stensile)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##  0.1517  0.3635  0.5793  0.5585  0.7256  0.9623
sd(stensile)
## [1] 0.2361888

Formal Analysis

Again we use the t distribution to produce a one sample Normal 95% confidence interval for the population mean.

State and check the model assumptions:

  1. the square root of the tensile strengths are a random sample from the population;

  2. the square root of the tensile strengths approximately follow a Normal probability model.

We assume that the sample has been randomly drawn though this has not been mentioned in the context. Produce a Q-Q plot to check the normality assumption:

par(mfrow=c(1,1))
qqnorm(stensile)
qqline(stensile)

The points falls close to the line and don’t appear to curve too much, evidence of normally distributed data. So we can proceed with the assumption of normality. Given the other data looked out rightly asymmetric there was no need to test the normality assumption formally.

The 95% CI is obtained from the t.test() function:

t.test(stensile)
## 
##  One Sample t-test
## 
## data:  stensile
## t = 12.952, df = 29, p-value = 1.387e-13
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  0.4703224 0.6467112
## sample estimates:
## mean of x 
## 0.5585168

The 95% CI for the square root of the population mean tensile strength is 0.470 and 0.646. Apply the inverse transformation before concluding.

result <-t.test(stensile)
(result$conf.int)^2
## [1] 0.2212032 0.4182353
## attr(,"conf.level")
## [1] 0.95

Note : Compare this result to the result from applying t.test() function to the raw data

t.test(tensile)
## 
##  One Sample t-test
## 
## data:  tensile
## t = 7.4624, df = 29, p-value = 3.18e-08
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  0.2655933 0.4661401
## sample estimates:
## mean of x 
## 0.3658667

Conclusion

It is highly likely that the mean tensile strength lies between 0.26 and 0.47 in the wider population of similar polyester fibres.

Section C - Two-sample Confidence Interval

Problem C1 - Music and Mental Ability

Context : A sample of 30 university students was randomly allocated into one of two groups. The members of each group performed a series of mental tasks while one of two music types was playing in the background. The members of group A were listening to classical music, while those in group B were listening to rock music. The number of mental tasks correctly completed by each student was recorded.

Question : In the wider population of students, is there evidence of a difference in the average number of tasks correctly completed between those listening to classical music and those listening to rock music. Quantify the size of this difference.

Data : The data are available from the file called MUSIC.csv, where the columns are:

Tasks: number of tasks correctly completed by each student Group: group indicators – A (classical) or B (rock)

Aim

We wish to investigate whether the type of music played affects the number of tasks correctly completed. We assume that the sample is representative of a wider population of university students.

Data Description

Load the music data

music <- read.csv(file="MUSIC.csv")

A summary of the data is given by

summary(music)
##      tasks       group 
##  Min.   :17.00   A:15  
##  1st Qu.:20.00   B:15  
##  Median :21.00         
##  Mean   :22.07         
##  3rd Qu.:25.00         
##  Max.   :29.00

We can look at the data further by calculating summary statistics by group and plotting against groups

tapply(music$tasks, music$group, FUN=summary)
## $A
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   17.00   21.00   23.00   22.93   25.00   29.00 
## 
## $B
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##    17.0    19.0    21.0    21.2    22.5    27.0
boxplot(music$tasks~music$group, ylab="No of tasks",
xlab="Type of Music", main="Task Completions by Music Type",
names=c("Classical", "Rock"))

Comment on your initial impressions of the data in terms of location, spread and shape

Formal analysis

We will answer the question of interest by computing a 95% confidence interval for the difference between the mean number of tasks completed in the wider population of students listening to classical and the mean number of tasks completed in the wider population of students listening to rock. We do this in R using the function t.test() again.

First, we must discuss and check some modelling assumptions. These are:

  1. the two samples of tasks completed are independent;
  2. the students were randomly allocated to the music groups;
  3. the tasks completed by the classical and rock groups follow Normal probability models which have equal variances.

All of the observations are taken from different students and so we may assume that they are all independent and from the wider populations of students; so, assumptions (a) and (b) are OK. The assumption of Normality for each sample also seems reasonable because the shapes of the boxplots are approximately symmetric, though we can produce a Q-Q plot to check this further using the code:

classical <-music$tasks[music$group=="A"]
rock <- music$tasks[music$group=="B"]
par(mfrow=c(1,2))
qqnorm(classical, main="Q-Q plot: Classical")
qqline(classical)
qqnorm(rock, main="Q-Q plot: Rock")
qqline(rock)

For the classical music group the points lie quite close to the straight line but this does not appear to be the case for the rock music group. This may in part be due to the fairly small size (only 15).

Nonetheless we now produce a 95% confidence interval for the difference between the population means.

t.test(music$tasks~music$group,var.equal=T)
## 
##  Two Sample t-test
## 
## data:  music$tasks by music$group
## t = 1.4328, df = 28, p-value = 0.163
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.7447247  4.2113913
## sample estimates:
## mean in group A mean in group B 
##        22.93333        21.20000

Note that R takes the difference as the mean number of tasks for the population of students listening to classical music minus the mean number of tasks for the population of students listening to rock music simply because A (classical) comes before B (rock) in the alphabet. The 95% confidence interval is (-0.75, 4.21).

Interpret this interval

Conclusion

We conclude that the in the wider population of students it is highly likely the mean number of tasks completed whilst listening to classical music versus rock music is not different.

Exercise C2 - Alcohol relapse (continued)

Context : Continuing with the context in Exercise A2 the investigation into the causes of relapse among alcoholics, included representative samples of both ‘long-standing’ alcoholics and ‘recent’ alcoholics. Again we are focussing on the psychological measure ‘unpleasant mood states’ (UM).

Questions : Is the ‘UM’ score of long-standing alcoholics on different to that of recent alcoholics? If so, by how much is it likely to be different?

Data: This is held in a CSV file called alcohol.csv. It contains the UM score in a column labelled UM and the type of alcoholic in type.

Work through this question by yourself. Ensure you answer the questions by following the layout of Exercise C1.