Complete all Exercises, and submit answers to Questions on the Coursera platform.

Getting Started

Load packages

In this lab we will explore the data using the dplyr package and visualize it using the ggplot2 package for data visualization. The data can be found in the companion package for this course, statsr.

Let’s load the packages.

library(statsr)
library(dplyr)
library(ggplot2)

The data

In 2004, the state of North Carolina released a large data set containing information on births recorded in this state. This data set is useful to researchers studying the relation between habits and practices of expectant mothers and the birth of their children. We will work with a random sample of observations from this data set.

Load the nc data set into our workspace.

data(nc)

We have observations on 13 different variables, some categorical and some numerical. The meaning of each variable is as follows.

variable description
fage father’s age in years.
mage mother’s age in years.
mature maturity status of mother.
weeks length of pregnancy in weeks.
premie whether the birth was classified as premature (premie) or full-term.
visits number of hospital visits during pregnancy.
marital whether mother is married or not married at birth.
gained weight gained by mother during pregnancy in pounds.
weight weight of the baby at birth in pounds.
lowbirthweight whether baby was classified as low birthweight (low) or not (not low).
gender gender of the baby, female or male.
habit status of the mother as a nonsmoker or a smoker.
whitemom whether mom is white or not white.
  1. There are 1,000 cases in this data set, what do the cases represent?
    1. The hospitals where the births took place
    2. The fathers of the children
    3. The days of the births
    4. The births

Answer: The 1,000 cases in this data set represent births.

As a first step in the analysis, we should take a look at the variables in the dataset. This can be done using the str command:

str(nc)
## Classes 'tbl_df', 'tbl' and 'data.frame':    1000 obs. of  13 variables:
##  $ fage          : int  NA NA 19 21 NA NA 18 17 NA 20 ...
##  $ mage          : int  13 14 15 15 15 15 15 15 16 16 ...
##  $ mature        : Factor w/ 2 levels "mature mom","younger mom": 2 2 2 2 2 2 2 2 2 2 ...
##  $ weeks         : int  39 42 37 41 39 38 37 35 38 37 ...
##  $ premie        : Factor w/ 2 levels "full term","premie": 1 1 1 1 1 1 1 2 1 1 ...
##  $ visits        : int  10 15 11 6 9 19 12 5 9 13 ...
##  $ marital       : Factor w/ 2 levels "married","not married": 1 1 1 1 1 1 1 1 1 1 ...
##  $ gained        : int  38 20 38 34 27 22 76 15 NA 52 ...
##  $ weight        : num  7.63 7.88 6.63 8 6.38 5.38 8.44 4.69 8.81 6.94 ...
##  $ lowbirthweight: Factor w/ 2 levels "low","not low": 2 2 2 2 2 1 2 1 2 2 ...
##  $ gender        : Factor w/ 2 levels "female","male": 2 2 1 2 1 2 2 2 2 1 ...
##  $ habit         : Factor w/ 2 levels "nonsmoker","smoker": 1 1 1 1 1 1 1 1 1 1 ...
##  $ whitemom      : Factor w/ 2 levels "not white","white": 1 1 2 2 1 1 1 1 2 2 ...

As you review the variable summaries, consider which variables are categorical and which are numerical. For numerical variables, are there outliers? If you aren’t sure or want to take a closer look at the data, make a graph.

Exploratory data analysis

We will first start with analyzing the weight gained by mothers throughout the pregnancy: gained.

Using visualization and summary statistics, describe the distribution of weight gained by mothers during pregnancy. The summary function can also be useful.

summary(nc$gained)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##    0.00   20.00   30.00   30.33   38.00   85.00      27
  1. How many mothers are we missing weight gain data from?
    1. 0
    2. 13
    3. 27
    4. 31

Answer: As per the data, the number of missing observations for the weight gain variable is 27.

Next, consider the possible relationship between a mother’s smoking habit and the weight of her baby. Plotting the data is a useful first step because it helps us quickly visualize trends, identify strong associations, and develop research questions.

  1. Make side-by-side boxplots of habit and weight. Which of the following is false about the relationship between habit and weight?
    1. Median birth weight of babies born to non-smoker mothers is slightly higher than that of babies born to smoker mothers.
    2. Range of birth weights of babies born to non-smoker mothers is greater than that of babies born to smoker mothers.
    3. Both distributions are extremely right skewed.
    4. The IQRs of the distributions are roughly equal. 
# type your code for the Question 3 here, and Knit
boxplot(weight~habit,data=nc, main="Birth Data", xlab="Smoker or Non Smoker", ylab="Birth Weight of Babies")

Answer: A boxplot can give you information regarding the shape, variability, and center (or median) of a statistical data set. It is particularly useful for displaying skewed data.

A boxplot can show whether a data set is symmetric (roughly the same on each side when cut down the middle) or skewed (lopsided). A symmetric data set shows the median roughly in the middle of the box.

The median, part of the five-number summary, is shown by the line that cuts through the box in the boxplot.

Skewed data show a lopsided boxplot, where the median cuts the box into two unequal pieces. If the longer part of the box is to the right (or above) the median, the data is said to be skewed right. If the longer part is to the left (or below) the median, the data is skewed left.

As per the output produced by R, the distribution of the birth weight of babies (as grouped by mothers who do not smoke) remains fairly symmetric, while the distribution of the birth weight of babies (as grouped by mothers who smoke) is slightly skewed to the left. Hence, based on this information, option b is correct.

The box plots show how the medians of the two distributions compare, but we can also compare the means of the distributions using the following to first group the data by the habit variable, and then calculate the mean weight in these groups using the mean function.

nc %>%
  group_by(habit) %>%
  summarise(mean_weight = mean(weight))
## # A tibble: 3 x 2
##   habit     mean_weight
##   <fct>           <dbl>
## 1 nonsmoker        7.14
## 2 smoker           6.83
## 3 <NA>             3.63

There is an observed difference, but is this difference statistically significant? In order to answer this question we will conduct a hypothesis test.

Inference

Exercise: Are all conditions necessary for inference satisfied? Comment on each. You can compute the group sizes using the same by command above but replacing mean(weight) with n().

Answer: The conditions for making inferences regarding the means of two independent samples are:

  1. The within group independence assumption, which is achieved via random sampling assignment and the 10% condition if you are sampling without replacement. Note that the 10% condition in this case means that both {n_1} and {n_2} should be less than 10% of their respective populations. If both of these are met, then we can assume that the observations in our study are independent of each other with respect to the outcome variable that we are studying.

  2. Between groups independence assumption. Failure to meet this condition is not inherently a problem though, but it would mean that we need to use methods suited for dependent or in other words paired groups.

The population consists of birth records for the state of NC. It makes sense that the sample size of 1000 is less than 10% of the population. These samples are simple random samples so the independence assumption is reasonable. The distribution of differences shown in the boxplots are a bit skewed, but they seem reasonable for the size of the sample.

  1. What are the hypotheses for testing if the average weights of babies born to smoking and non-smoking mothers are different?
    1. \(H_0: \mu_{smoking} = \mu_{non-smoking}\); \(H_A: \mu_{smoking} > \mu_{non-smoking}\)
    2. \(H_0: \mu_{smoking} = \mu_{non-smoking}\); \(H_A: \mu_{smoking} \ne \mu_{non-smoking}\)
    3. \(H_0: \bar{x}_{smoking} = \bar{x}_{non-smoking}\); \(H_A: \bar{x}_{smoking} > \bar{x}_{non-smoking}\)
    4. \(H_0: \bar{x}_{smoking} = \bar{x}_{non-smoking}\); \(H_A: \bar{x}_{smoking} > \bar{x}_{non-smoking}\)
    5. \(H_0: \mu_{smoking} \ne \mu_{non-smoking}\); \(H_A: \mu_{smoking} = \mu_{non-smoking}\)

Next, we introduce a new function, inference, that we will use for conducting hypothesis tests and constructing confidence intervals.

Then, run the following:

inference(y = weight, x = habit, data = nc, statistic = "mean", type = "ht", null = 0, 
          alternative = "twosided", method = "theoretical")
## Response variable: numerical
## Explanatory variable: categorical (2 levels) 
## n_nonsmoker = 873, y_bar_nonsmoker = 7.1443, s_nonsmoker = 1.5187
## n_smoker = 126, y_bar_smoker = 6.8287, s_smoker = 1.3862
## H0: mu_nonsmoker =  mu_smoker
## HA: mu_nonsmoker != mu_smoker
## t = 2.359, df = 125
## p_value = 0.0199

Let’s pause for a moment to go through the arguments of this custom function.

The first argument is y, which is the response variable that we are interested in: weight. The second argument is the explanatory variable, x, which is the variable that splits the data into two groups, smokers and non-smokers: habit. The third argument, data, is the data frame these variables are stored in. Next is statistic, which is the sample statistic

we’re using, or similarly, the population parameter we’re estimating. In future labs, we can also work with “median” and “proportion”. Next we decide on the type of inference we want: a hypothesis test ("ht") or a confidence interval ("ci"). When performing a hypothesis test, we also need to supply the null value, which in this case is 0, since the null hypothesis sets the two population means equal to each other.

The alternative hypothesis can be "less", "greater", or "twosided". Lastly, the method of inference can be "theoretical" or "simulation" based.

For more information on the inference function see the help file with ?inference.

Exercise: What is the conclusion of the hypothesis test?

Answer: Judging by the magnitude of the p-value (considering the level of significance as 5%) and the absolute value of the test statistic t (which is greater than 2), we can reject the null hypothesis and conclude that the data does provide convincing evidence of there being a significant difference between the average birth weight of a baby born to a mom who doesn’t smoke vs. a mom who does smoke.

  1. Change the type argument to "ci" to construct and record a confidence interval for the difference between the weights of babies born to nonsmoking and smoking mothers, and interpret this interval in context of the data. Note that by default you’ll get a 95% confidence interval. If you want to change the confidence level, add a new argument (conf_level) which takes on a value between 0 and 1. Also note that when doing a confidence interval arguments like null and alternative are not useful, so make sure to remove them.
    1. We are 95% confident that babies born to nonsmoker mothers are on average 0.05 to 0.58 pounds lighter at birth than babies born to smoker mothers.
    2. We are 95% confident that the difference in average weights of babies whose moms are smokers and nonsmokers is between 0.05 to 0.58 pounds.
    3. We are 95% confident that the difference in average weights of babies in this sample whose moms are smokers and nonsmokers is between 0.05 to 0.58 pounds.
    4. We are 95% confident that babies born to nonsmoker mothers are on average 0.05 to 0.58 pounds heavier at birth than babies born to smoker mothers. 
# type your code for the Question 5 here, and Knit
inference(y = weight, x = habit, data = nc, statistic = "mean", type = "ci", method = "theoretical", order = c("smoker","nonsmoker"))
## Response variable: numerical, Explanatory variable: categorical (2 levels)
## n_smoker = 126, y_bar_smoker = 6.8287, s_smoker = 1.3862
## n_nonsmoker = 873, y_bar_nonsmoker = 7.1443, s_nonsmoker = 1.5187
## 95% CI (smoker - nonsmoker): (-0.5803 , -0.0508)

By default the function reports an interval for (\(\mu_{nonsmoker} - \mu_{smoker}\)) . We can easily change this order by using the order argument:

inference(y = weight, x = habit, data = nc, statistic = "mean", type = "ci", 
          method = "theoretical", order = c("smoker","nonsmoker"))
## Response variable: numerical, Explanatory variable: categorical (2 levels)
## n_smoker = 126, y_bar_smoker = 6.8287, s_smoker = 1.3862
## n_nonsmoker = 873, y_bar_nonsmoker = 7.1443, s_nonsmoker = 1.5187
## 95% CI (smoker - nonsmoker): (-0.5803 , -0.0508)

Answer: Based on the output from the inference function, we are 95% confident that babies born to mothers who smoke are on an average 0.05 to 0.58 pounds lighter than babies born to mothers who do not smoke.

  1. Calculate a 99% confidence interval for the average length of pregnancies (weeks). Note that since you’re doing inference on a single population parameter, there is no explanatory variable, so you can omit the x variable from the function. Which of the following is the correct interpretation of this interval?
    1. (38.1526 , 38.5168)
    2. (38.0892 , 38.5661)
    3. (6.9779 , 7.2241)
    4. (38.0952 , 38.5742) 
# type your code for Question 6 here, and Knit
inference(y=weeks, data=nc, type = "ci", statistic = "mean", success = NULL, order = NULL, method = "theoretical", null = 0, alternative = "twosided", conf_level = 0.99)
## Single numerical variable
## n = 998, y-bar = 38.3347, s = 2.9316
## 99% CI: (38.0952 , 38.5742)

Answer: Based on the output from R, we are 99% confident that the average length of pregnancies (in weeks) is between 38 weeks and 39 weeks.

Exercise: Calculate a new confidence interval for the same parameter at the 90% confidence level. Comment on the width of this interval versus the one obtained in the the previous exercise.

# type your code for the Exercise here, and Knit
inference(y=weeks, data=nc, type = "ci", statistic = "mean", success = NULL, order = NULL, method = "theoretical", null = 0, alternative = "twosided", conf_level = 0.90)
## Single numerical variable
## n = 998, y-bar = 38.3347, s = 2.9316
## 90% CI: (38.1819 , 38.4874)

Answer: Based on the output from R, we are 90% confident that the average length of pregnancies (in weeks) is between 38 weeks and 38.4 weeks.

For the previous question, we computed a 99% confidence interval. For this question, we computed a 90% confidence interval. Remember that the width of the 99% confidence is greater than that of the 90% confidence interval but only marginally in this case. Another way to think about this, is the width of the area that captures the middle 95% or 99% of the distribution. Obviously, the middle 99% will span a bigger area. The 99% confidence interval will be larger. As the confidence level increases, the width of the confidence interval increases. Increasing accuracy, implies increasing the confidence level. However, the precision goes down.

Exercise: Conduct a hypothesis test evaluating whether the average weight gained by younger mothers is different than the average weight gained by mature mothers.

#inference function code
inference(y = gained, x = mature, data = nc, statistic = "mean", type = "ht", method = "theoretical", null=0, alternative = "twosided")
## Response variable: numerical
## Explanatory variable: categorical (2 levels) 
## n_mature mom = 129, y_bar_mature mom = 28.7907, s_mature mom = 13.4824
## n_younger mom = 844, y_bar_younger mom = 30.5604, s_younger mom = 14.3469
## H0: mu_mature mom =  mu_younger mom
## HA: mu_mature mom != mu_younger mom
## t = -1.3765, df = 128
## p_value = 0.1711

Answer: Based upon the output from R, we fail to reject the null hypothesis that there is no signifcant difference between the average weight gained by a mature mom, relative to a younger mom. The data does not provide convincing evidence of there being a significant difference. As far as the magnitude of the p-value is concerned, there is a 17% probability that (given the null hypothesis is true), the difference in weight gain between a younger mom and a mature mom, is just due to chance.

  1. Now, a non-inference task: Determine the age cutoff for younger and mature mothers. Use a method of your choice, and explain how your method works.
# type your code for Question 7 here, and Knit
boxplot(mage~mature,data=nc, main="Maturity Status of the Mom", xlab="Mature or Younger", ylab="Age of the mother")

summarise(group_by(nc, mature), average.age = mean(mage), max.age = max(mage), min.age = min(mage))
## # A tibble: 2 x 4
##   mature      average.age max.age min.age
##   <fct>             <dbl>   <dbl>   <dbl>
## 1 mature mom         37.2      50      35
## 2 younger mom        25.4      34      13

Answer: I am making use of a data visualization tool known as a box plot and summary statistics. In the box plot above, the distribution of mature moms is highly left skewed. The majority of the distribution lies within the age of 35-37. The distribution of younger moms is slightly right skewed. The majority of the distribution lies within the age of around 30.

What is the maximum age of a younger mom and the minimum age of a mature mom, according to the data?
  1. The maximum age of younger moms is 32 and minimum age of mature moms is 33
  2. The maximum age of younger moms is 33 and minimum age of mature moms is 34
  3. The maximum age of younger moms is 35 and minimum age of mature moms is 36
  4. The maximum age of younger moms is 34 and minimum age of mature moms is 35
Answer: Based on the output produced by R for descriptive statistics, the maximum age for younger moms is 34 and the minimum age for mature moms is 35.

This is a product of OpenIntro that is released under a Creative Commons Attribution-ShareAlike 3.0 Unported. This lab was written for OpenIntro by Andrew Bray and Mine Çetinkaya-Rundel.