Chapter 4. Foundations for inference

Loading data

#Eliminating messages and warnings
suppressMessages(library(DATA606))

## 
## Welcome to CUNY DATA606 Statistics and Probability for Data Analytics 
## This package is designed to support this course. The text book used 
## is OpenIntro Statistics, 3rd Edition. You can read this by typing 
## vignette('os3') or visit www.OpenIntro.org. 
##  
## The getLabs() function will return a list of the labs available. 
##  
## The demo(package='DATA606') will list the demos that are available.

suppressWarnings(library(DATA606))
suppressMessages(library(dplyr))

## Warning: package 'dplyr' was built under R version 3.4.2

suppressWarnings(library(dplyr))
suppressMessages(library(ggplot2))
suppressWarnings(library(ggplot2))

library(DATA606)
library(dplyr)
library(ggplot2)

#Heights of Adults
exer4.4 <- read.csv('https://raw.githubusercontent.com/jbryer/DATA606Fall2017/master/Data/Data%20from%20openintro.org/Ch%204%20Exercise%20Data/bdims.csv')

#Thanksgiving spending. Part I
exer4.14 <- read.csv('https://raw.githubusercontent.com/jbryer/DATA606Fall2017/master/Data/Data%20from%20openintro.org/Ch%204%20Exercise%20Data/tgSpending.csv')

#Gifted children. Part I
exer4.24 <- read.csv('https://raw.githubusercontent.com/jbryer/DATA606Fall2017/master/Data/Data%20from%20openintro.org/Ch%204%20Exercise%20Data/gifted.csv')

#CLT
exer4.34 <- read.csv('https://raw.githubusercontent.com/jbryer/DATA606Fall2017/master/Data/Data%20from%20openintro.org/Ch%204%20Exercise%20Data/penniesAges.csv')

4.4 Heights of adults.

Researchers studying anthropometry collected body girth measurements and skeletal diameter measurements, as well as age, weight, height and gender, for 507 physically active individuals. The histogram below shows the sample distribution of heights in centimeters.

hist(exer4.4$hgt)

summary(exer4.4$hgt)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   147.2   163.8   170.3   171.1   177.8   198.1

What is the point estimate for the average height of active individuals? What about the median?

The point estimator is the mean or 171.1 cm. The median is the 170.3 cm.

What is the point estimate for the standard deviation of the heights of active individuals? What about the IQR?

sd(exer4.4$hgt)

## [1] 9.407205

The point estimate for the standard deviation is 9.4 cm.

The IQR is 14 cm.

Is a person who is 1m 80cm (180 cm) tall considered unusually tall? And is a person who is 1m 55cm (155cm) considered unusually short? Explain your reasoning.

xbar <- mean(exer4.4$hgt)
sd <- sd(exer4.4$hgt)
val <- 180
x = round(pnorm(val, xbar, sd, lower.tail = F) * 100 , 2)
y = round((val - xbar)/sd, 2)
x

## [1] 17.32

## [1] 0.94

There is an 17.32% chance that a randomly drawn individual is 180 or taller. This doesn’t seem unusual. That hight also only 0.94 standard deviations away. So while it falls outside of the IQR, it’s still pretty usual.

val <- 155
x = round(pnorm(val, xbar, sd, lower.tail = T) * 100 , 2)
y = round((val - xbar)/sd, 2)
x

## [1] 4.31

## [1] -1.72

There is a 4.31% chance that a person will be 155 cm tall. This falls below the typical 95% Confidence Intervals. Also this person would be -1.72 standard deviations away from the mean. One could make the argument that this person is unusual becuase unusual results are said to be over 95% CI.

The researchers take another random sample of physically active individuals. Would you expect the mean and the standard deviation of this new sample to be the ones given above? Explain your reasoning.

The mean and SD of a new sample would almost certainly not be the same. The chances of a mean being the same for two samples for a continous variable approach zero. (Area under a curver for a single point is zero.) Since there seems to be a precision amount in the calculations (e.g., the results are rounded to one decimal) we could calculate the actual chance of getting an observed mean that is the same, although the true mean would be different. To calculate this we also need to know the population mean. So really, the the chances are very low.

The sample means obtained are point estimates for the mean height of all active individuals, if the sample of individuals is equivalent to a simple random sample. What measure do we use to quantify the variability of such an estimate (Hint: recall that SD¯x = ! pn )? Compute this quantity using the data from the original sample under the condition that the data are a simple random sample.

dim(exer4.4)

## [1] 507  25

n <- 507
se <- sd/sqrt(n) 
se %% round(2)

## [1] 0.4177887

4.14 Thanksgiving spending, Part I.

The 2009 holiday retail season, which kicked off on November 27, 2009 (the day after Thanksgiving), had been marked by somewhat lower self-reported consumer spending than was seen during the comparable period in 2008. To get an estimate of consumer spending, 436 randomly sampled American adults were surveyed. Daily consumer spending for the six-day period after Thanksgiving, spanning the Black Friday weekend and Cyber Monday, averaged $84.71. A 95% confidence interval based on this sample is ($80.31, $89.11). Determine whether the following statements are true or false, and explain your reasoning.

hist(exer4.14$spending)

We are 95% confident that the average spending of these 436 American adults is between $80.31 and $89.11?

#Confidence intervals
sample_mean <- mean(exer4.14$spending)
se <- sd(exer4.14$spending) / sqrt(436)
lower <- sample_mean - 1.96 * se
upper <- sample_mean + 1.96 * se
c(lower, upper)

## [1] 80.30173 89.11180

#Getting the mean
mean(exer4.14$spending)

## [1] 84.70677

Answer: False : We are 95% certain the the TRUE mean of spending is between $80.31 and $89.11. The average spending of the sample is exactly $84.71

b.This confidence interval is not valid since the distribution of spending in the sample is right skewed.

Answer: True

    There are a few conditions that must be met for this interval to be valid
      What conditions must be met for this to be true? 
      The sample observations are indenpendent. : __True__
      The sample size is larger that 30 : __True__
      The population distribution is not strongly skewed : __False__

95% of random samples have a sample mean between $80.31 and $89.11.

Answer: False : It only provides a guide for how large we should make the confidence interval. We can’t say that 95% of samples will fall in this.

We are 95% confident that the average spending of all American adults is between $80.31 and $89.11.

Answer: True : yes, the population mean of 84.70677 falls in the 95% confidence interval [80.30173 89.11180]

A 90% confidence interval would be narrower than the 95% confidence interval since we don’t need to be as sure about our estimate.

Answer: __ True__ : We don’t need to be as certain so we can have a narrower CI.

In order to decrease the margin of error of a 95% confidence interval to a third of what it is now, we would need to use a sample 3 times larger.

Answer: False : We would need to use a sample 9 times larger because The standard error is se = sd/sqrt(n). If n is multiplied by 3, the standard error will be reduced by sqrt(3).
g. The margin of error is 4.4.

Answer: True

me = (upper <- sample_mean + 1.96 * se) -(mean(exer4.14$spending))
me

## [1] 4.405038

4.24 Gifted children, Part I.

Researchers investigating characteristics of gifted children collected data from schools in a large city on a random sample of (36) thirty-six children who were identified as gifted children soon after they reached the age of four. The following histogram shows the distribution of the ages (in months) at which these children first counted to 10 successfully. Also provided are some sample statistics.

hist(exer4.24$count)

summary(exer4.24$count)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   21.00   28.00   31.00   30.69   34.25   39.00

sd(exer4.24$count)

## [1] 4.314887

Are conditions for inference satisfied?

Answer : Yes. Becuse, the random sample is from less than 10% of all children, we assume The sample observations are indenpendent The sample size is larger that 30 The population distribution is not strongly skewed.

Suppose you read online that children first count to 10 successfully when they are 32 months old, on average. Perform a hypothesis test to evaluate if these data provide convincing evidence that the average age at which gifted children first count to 10 successfully is less than the general average of 32 months. Use a significance level of 0.10.

Ho = Average age gifted children count to 10 successfully when they are 32 months old Ha = Average age gifted children count to 10 successfully < 32 months Significance level of 0.10

statsGif <- exer4.24 %>% 
  select(count) %>% 
  summarise(meanGift = mean(count),
            sdGift = sd(count),
            nGift = n())

x <- pnorm(32, mean = statsGif$meanGift, sd = statsGif$sdGift, lower.tail = F)
t.test(exer4.24$count, alternative = "two.sided", mu = 32)

## 
##  One Sample t-test
## 
## data:  exer4.24$count
## t = -1.8154, df = 35, p-value = 0.07804
## alternative hypothesis: true mean is not equal to 32
## 95 percent confidence interval:
##  29.23450 32.15439
## sample estimates:
## mean of x 
##  30.69444

Interpret the p-value in context of the hypothesis test and the data.

Answer:The p-value of 0.078 is less than 0.1 significance level so we can reject the null hypothesis that the gifted students can counted to 10 to 32 and there is no difference between the groups.

Calculate a 90% confidence interval for the average age at which gifted children first count to 10 successfully.

t.test(exer4.24$count, alternative = "two.sided", conf.level = .9)

## 
##  One Sample t-test
## 
## data:  exer4.24$count
## t = 42.682, df = 35, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 0
## 90 percent confidence interval:
##  29.47939 31.90950
## sample estimates:
## mean of x 
##  30.69444

Do your results from the hypothesis test and the confidence interval agree? Explain.

Yes, They are the same, both results are matching because we are 90% confident that the count falls between the confidence interval 29.48 and 31.92 months.

4.26 Gifted children, Part II. Exercise 4.24 describes a study on gifted children. In this study, along with variables on the children, the researchers also collected data on the mother’s and father’s IQ of the 36 randomly sampled gifted children. The histogram below shows the distribution of mother’s IQ. Also provided are some sample statistics.

hist(exer4.24$motheriq)

summary(exer4.24$motheriq)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   101.0   113.8   118.0   118.2   122.2   131.0

Perform a hypothesis test to evaluate if these data provide convincing evidence that the average IQ of mothers of gifted children is different than the average IQ for the population at large, which is 100. Use a significance level of 0.10.

H0: the average IQ of mothers of gifted children is equal to 100 Ha: the average IQ of mothers of gifted children is different than 100

t.test(exer4.24$motheriq, alternative = "two.sided", mu = 100)

## 
##  One Sample t-test
## 
## data:  exer4.24$motheriq
## t = 16.756, df = 35, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 100
## 95 percent confidence interval:
##  115.9657 120.3676
## sample estimates:
## mean of x 
##  118.1667

The alternative hypothesis is telling us that the mother iq is different from the mean of 100.

Calculate a 90% confidence interval for the average IQ of mothers of gifted children.

t.test(exer4.24$motheriq, alternative = "two.sided", conf.level = .90)

## 
##  One Sample t-test
## 
## data:  exer4.24$motheriq
## t = 108.99, df = 35, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 0
## 90 percent confidence interval:
##  116.3349 119.9984
## sample estimates:
## mean of x 
##  118.1667

Do your results from the hypothesis test and the confidence interval agree? Explain.

Yes, We are 90% confident that the mothers iq fall in the confidence interval between 116.33 and 120 and the confidence interval doesn’t the population mean of 100.

4.34 CLT.

Define the term “sampling distribution” of the mean, and describe how the shape, center, and spread of the sampling distribution of the mean change as sample size increases.

The distribution of sample means, called the sampling distribution, represents the distribution of the point estimates based on sample of a fixed size from a certain population. When the sample distribution is unimodal and approx. symetric the sample meas should tend to fall around the population mean. The center shouldn’t be changing. the shape, the larger the sample the more lenient we can be with the sample’s skew The spread could be smaller or larger.

4.40 CFLBs. A manufacturer of compact fluorescent light bulbs advertises that the distribution of the lifespans of these light bulbs is nearly normal with a mean of 9,000 hours and a standard deviation of 1,000 hours.

What is the probability that a randomly chosen light bulb lasts more than 10,500 hours?

pnorm(10500, 9000, 1000, lower.tail = F)

## [1] 0.0668072

b.Describe the distribution of the mean lifespan of 15 light bulbs.

# the distribution of the mean is calculated as sd/sqrt(n)
sd = 1000
n = 15
b15 <- sd/sqrt(n)
b15

## [1] 258.1989

What is the probability that the mean lifespan of 15 randomly chosen light bulbs is more than 10,500 hours?

pnorm(10500, mean=9000, sd=b15, lower.tail = FALSE)

## [1] 3.133452e-09

Sketch the two distributions (population and sampling) on the same scale.

SN <- seq(9000 - (4 * sd), 9000 + (4 * sd), length=15)
SR <- seq(9000 - (4 * b15), 9000 + (4 * b15), length=15)
nd1 <- dnorm(SN,9000,1000)
nd2 <- dnorm(SR,9000,b15)

plot(SN, nd1, type="l",col="blue",
     xlab=" Population vs Sampling", 
     main=" Compact Fluorescent Light Bulbs",
     ylim=c(0,0.002))
lines(SR, nd2, col="red")

Could you estimate the probabilities from parts (a) and (c) if the lifespans of light bulbs had a skewed distribution?

It would be difficult because the sample size is too small.

4.48 Same observation, different sample size

Suppose you conduct a hypothesis test based on a sample where the sample size is n = 50, and arrive at a p-value of 0.08. You then refer back to your notes and discover that you made a careless mistake, the sample size should have been n = 500. Will your p-value increase, decrease, or stay the same? Explain.

The p-value will decrease because:

The Z-value is calculated base on

Z = (xbar - null)/ SE. the larger sample has a smaller Standar Error. If the SE decreases the Z value increases and the p-value also will decreases.

HW 4

Durley Torres-Marin

October 15, 2017

Chapter 4. Foundations for inference

Loading data