1 Pre-Requistes : Setup R environment for DATA 606.

See https://data606.net/assignments/homework/ for more information. Chapter 4 Foundations for Inference Practice: 4.3, 4.13, 4.23, 4.25, 4.39, 4.47 Graded: 4.4, 4.14, 4.24, 4.26, 4.34, 4.40, 4.48

1.1 Install R packages as per DATA606Spring2019-master/R/Setup.R

Refer to “Getting Started with R” in https://data606.net/post/

2 Exercises

2.1 Question#4.4 Heights of adults.

Researchers studying anthropometry collected body girth measurements and skeletal diameter measurements, as well as age, weight, height and gender, for 507 physically active individuals. The histogram below shows the sample distribution of heights in centimeters.

hist(bdims$hgt)

2.1.1 (a) What is the point estimate for the average height of active individuals? What about the median?

(See the next page for parts (b)-(e).)

paste("Average height of active individuals:",mean(bdims$hgt),"centimeters.")
## [1] "Average height of active individuals: 171.143786982249 centimeters."
paste("Median height of active individuals:",median(bdims$hgt),"centimeters.")
## [1] "Median height of active individuals: 170.3 centimeters."

2.1.2 (b) What is the point estimate for the standard deviation of the heights of active individuals? What about the IQR?

summary(bdims$hgt)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   147.2   163.8   170.3   171.1   177.8   198.1
paste("Standard deviation of the heights of active individuals:",sd(bdims$hgt),"centimeters.")
## [1] "Standard deviation of the heights of active individuals: 9.40720520351794 centimeters."
paste("InterQuartile Range (IQR) of the heights of active individuals:",summary(bdims$hgt)[2],"-",quantile(bdims$hgt, 0.75),"centimeters.")
## [1] "InterQuartile Range (IQR) of the heights of active individuals: 163.8 - 177.8 centimeters."

2.1.3 (c) Is a person who is 1m 80cm (180 cm) tall considered unusually tall? And is a person who is 1m 55cm (155cm) considered unusually short?

Explain your reasoning.

Z <- (x-u)/sd
Z180 <- (180 - 171.1) / 9.4; Z180
## [1] 0.9468085
Z155 <- (155 - 171.1) / 9.4; Z155
## [1] -1.712766

According 68-95-99.7 rule, 95% percentile z score is ±1.96. Also Z score range are within 2 stardard deviations from mean is (152.3, 189.9). Since both observations fall within 2 standard deviations, both heights of 180cm and 155cm are not considered as unusual.

2.1.4 (d) The researchers take another random sample of physically active individuals. Would you expect the mean and the standard deviation of this new sample to be the ones given above?

Explain your reasoning.

The new sample is a different sample population with different estimates and will have different values than the given sample. In such a case, the new values will have a slightly different mean and standard deviation values but close enough to the given.

2.1.5 (e) The sample means obtained are point estimates for the mean height of all active individuals, if the sample of individuals is equivalent to a simple random sample.

What measure do we use to quantify the variability of such an estimate (Hint: recall that SD¯x = ! pn )? Compute this quantity using the data from the original sample under the condition that the data are a simple random sample.

n <- 507
SE <- sigma/sqrt(n)
paste("Compute Standard Error for sample mean:",sd(bdims$hgt)/sqrt(507))
## [1] "Compute Standard Error for sample mean: 0.417788650505626"

2.2 Question#4.14 Thanksgiving spending, Part I.

The 2009 holiday retail season, which kicked off on November 27, 2009 (the day after Thanksgiving), had been marked by somewhat lower self-reported consumer spending than was seen during the comparable period in 2008. To get an estimate of consumer spending, 436 randomly sampled American adults were surveyed. Daily consumer spending for the six-day period after Thanksgiving, spanning the Black Friday weekend and Cyber Monday, averaged $84.71. A 95% confidence interval based on this sample is ($80.31, $89.11). Determine whether the following statements are true or false, and explain your reasoning.

hist(tgSpending$spending)

2.2.1 (a) We are 95% confident that the average spending of these 436 American adults is between $80.31 and $89.11.

#Confidence intervals
sample_mean <- mean(tgSpending$spending)
se <- sd(tgSpending$spending) / sqrt(436)
lower <- sample_mean - 1.96 * se
upper <- sample_mean + 1.96 * se
c(lower, upper)
## [1] 80.30173 89.11180

FALSE - as per above we are 95% confident that the average spending of ALL (not just limited to 436) American adults is between $80.31 and $89.11.

2.2.2 (b) This confidence interval is not valid since the distribution of spending in the sample is right skewed.

There are a few conditions that must be met for this interval to be valid What conditions must be met for this to be true? - The sample observations are indenpendent. : True - The sample size is larger that 30 : True - The population distribution is not strongly skewed : True

FALSE - As per the histogram although graph is right skewed but it is not strongly right skewed and the distribtion is normal for the mean and already n (436) > 30

2.2.3 (c) 95% of random samples have a sample mean between $80.31 and $89.11.

FALSE - It only provides a guide for how large we should make the confidence interval. We can’t say that 95% of samples will fall in this interval. Confidence interval is used to determine population parameter and has no relation with samples and sample means.

2.2.4 (d) We are 95% confident that the average spending of all American adults is between $80.31 and $89.11.

paste0("The population mean of ", mean(tgSpending$spending),"falls in the 95% confidence interval [80.31 89.11]")
## [1] "The population mean of 84.7067651338864falls in the 95% confidence interval [80.31 89.11]"

TRUE - It’s estimated by the point estimate and the confidence interval.

2.2.5 (e) A 90% confidence interval would be narrower than the 95% confidence interval since we don’t need to be as sure about our estimate.

Confidence level depends on three components. - Point estimate - z-score and - standard error.

In the case of 95%, z-score is 1.98, that means point estimate is within ±1.98 times standard error. For 90% z-score point estimate would be 1.65 which is in the range of ±1.30 times standard error.

TRUE - Hence when confidence level is reduced, there is a chance population parameter may not be captured. This will include less accurate population mean than before however we don’t need to be as certain so we can have a narrower CI.

2.2.6 (f) In order to decrease the margin of error of a 95% confidence interval to a third of what it is now, we would need to use a sample 3 times larger.

paste0("The standaed error : ", sd(tgSpending$spending) / sqrt(436))
## [1] "The standaed error : 2.24746812614075"

FALSE - If n is multiplied by 3, the standard error will be reduced by sqrt(3) instead sample size needs to be increased by 9 times ( since sqrt(9)=3 )

2.2.7 (g) The margin of error is 4.4.

paste0("TRUE - The margin of error : ",  (upper <- sample_mean + 1.96 * se) -(mean(tgSpending$spending)))
## [1] "TRUE - The margin of error : 4.40503752723588"
paste0("Another way to calculate the margin of error : ",  (upper - lower)/2)
## [1] "Another way to calculate the margin of error : 4.40503752723588"

2.3 Question#4.24 Gifted children, Part I.

Researchers investigating characteristics of gifted children collected data from schools in a large city on a random sample of thirty-six children who were identified as gifted children soon after they reached the age of four. The following histogram shows the distribution of the ages (in months) at which these children first counted to 10 successfully. Also provided are some sample statistics.

hist(gifted$count)

2.3.1 (a) Are conditions for inference satisfied?

Use this information to determine if the heights approximately follow the 68-95-99.7% Rule.

TRUE - Because, - the random sample of 36 must be much less than 10% of all children in a large city, we may assume the sample observations are independent - the sample size is larger that 30 - the population distribution is not strongly skewed.

2.3.2 (b) Suppose you read online that children first count to 10 successfully when they are 32 months old, on average.

Perform a hypothesis test to evaluate if these data provide convincing evidence that the average age at which gifted children fist count to 10 successfully is less than the general average of 32 months. Use a significance level of 0.10. Explain your reasoning using the graphs provided below.

Assuming, Ho = Average age gifted children count to 10 successfully when they are 32 months old Ha = Average age gifted children count to 10 successfully < 32 months Significance level = 0.10

x <- 32
n <- 36
mean <- mean(gifted$count)
sd <- sd(gifted$count)
SE <- sd/sqrt(n)
z = (mean - x)/SE
p = pnorm(z)
paste0 ("Since, ",p,"< 0.10, we can reject the null hypothesis Ho in favor of alternative Ha")
## [1] "Since, 0.0347296895177584< 0.10, we can reject the null hypothesis Ho in favor of alternative Ha"
normalPlot(mean = 0, sd = 1, bounds = c(-Inf,z), tails = FALSE)

2.3.3 (c) Interpret the p-value in context of the hypothesis test and the data.

If the null hypothesis is true, then the probability of observing a sample mean lower than 30.69 for a sample of 36 children is only 0.0344 (p-value). The p-value of 0.034 is less than 0.1 significance level so we can reject the null hypothesis that the gifted students can counted to 10 to 32 and there is no difference between the groups. As can be seen from the plot above, p-value falls in the left tail area of the normal curve. p-value is less than significance level(??) = 0.0344 < 0.10.

2.3.4 (d) Calculate a 90% confidence interval for the average age at which gifted children first count to 10 successfully.

lower_interval <- mean - (qnorm(0.90) * SE)
upper_interval <- mean + (qnorm(0.90) * SE)
c(lower_interval,upper_interval)
## [1] 29.77282 31.61607

2.3.5 (e) Do your results from the hypothesis test and the confidence interval agree?

Explain.

Yes, the hypothesis testing and confidence interval agree as we see the range of confidence interval is less than 32. The upper bound for the 90% confidence interval of the average age at which gifted children first count to 10 successfully is 31.61 months, which is still less than 32 months. 32 months is the average age for the general population. Both support the alternative hypothesis in providing convincing evidence that the average age at which gifted children first count to 10 successfully can be less than the general average of 32 months

2.4 Question#4.26 Gifted children, Part II.

Exercise 4.24 describes a study on gifted children. In this study, along with variables on the children, the researchers also collected data on the mother’s and father’s IQ of the 36 randomly sampled gifted children. The histogram below shows the distribution of mother’s IQ. Also provided are some sample statistics.

hist(gifted$motheriq)

2.4.1 (a) Perform a hypothesis test to evaluate if these data provide convincing evidence that the average IQ of mothers of gifted children is different than the average IQ for the population at large, which is 100. Use a significance level of 0.10.

Ho: Null hypothesis - The average IQ of mothers of gifted children is the same as that of the general population, ??g=100 months. Ha: Alternative hypothesis - The average IQ of mothers of gifted children is different than that of the general population, ??g???100 32 months.

x <- 100
n <- 36
mean <- mean(gifted$motheriq); mean
## [1] 118.1667
sd <- sd(gifted$motheriq); sd
## [1] 6.504943
SE <- sd/sqrt(n); SE
## [1] 1.084157
z <- (mean - x)/SE; z
## [1] 16.75649
p <-   2 * (pnorm(z, 0, 1, lower.tail = FALSE)); p
## [1] 5.077477e-63
normalPlot(mean = 0, sd = 1, bounds = c(-z,z), tails = TRUE)

A P value is less than our significance level of .1, so we can reject the null hypothesis in favor of the alternative hypothesis. There is statistical evidence that the IQ of the mothers of gifted children is different than that of the general population.

2.4.2 (b) Calculate a 90% confidence interval for the average IQ of mothers of gifted children.

lower_value <- mean - (qnorm(0.90) * SE)
upper_value <- mean + (qnorm(0.90) * SE)
c(lower_value,upper_value)
## [1] 116.7773 119.5561

2.4.3 (c) Do your results from the hypothesis test and the confidence interval agree?

Explain.

TRUE - We are 90% confident that the average IQ of mothers of gifted children is between 116.4125 and 119.9875. This is significantly above population average of 100.

2.5 Question#4.34 CLT.

Define the term “sampling distribution” of the mean, and describe how the shape, center, and spread of the sampling distribution of the mean change as sample size increases.

A sampling distribution is a probability distribution of a statistic obtained through a large number of samples drawn from a specific population. The sampling distribution of the mean takes the mean of each sample. These means are used as the data, rather than individual scores.

Ideally, a sample to be considered random it should be less than 10% of actual individual scores and at least 30 such samples form good sampling distribution. If the sample size(sample of means) increases,

  • The mean of the sample means will be the mean of the population.
  • As the sample size gets larger, the shape becomes on a more ‘normal’ and has a smaller spread. We can be more lenient with the sample’s skewness.
  • The variance of the sample means will be the variance of the population divided by the sample size.
  • The standard deviation of the sample means (known as the standard error of the mean) will be smaller than the population mean and will be equal to the standard deviation of the population divided by the square root of the sample size.

Along with the above, sample means will fit into normal bell curve, 68-95-99.7 rule.

2.6 Question#4.40 CFLBs.

A manufacturer of compact fluorescent light bulbs advertises that the distribution of the lifespans of these light bulbs is nearly normal with a mean of 9,000 hours and a standard deviation of 1,000 hours.

2.6.1 (a) What is the probability that a randomly chosen light bulb lasts more than 10,500 hours?

pnorm(10500, 9000, 1000, lower.tail = F)
## [1] 0.0668072

2.6.2 (b) Describe the distribution of the mean lifespan of 15 light bulbs.

sd = 1000
n = 15
b15 <- sd/sqrt(n); b15
## [1] 258.1989

2.6.3 (c) What is the probability that the mean lifespan of 15 randomly chosen light bulbs is more than 10,500 hours?

pnorm(10500, mean=9000, sd=b15, lower.tail = FALSE)
## [1] 3.133452e-09

2.6.4 (d) Sketch the two distributions (population and sampling) on the same scale.

SN <- seq(9000 - (4 * sd), 9000 + (4 * sd), length=15)
SR <- seq(9000 - (4 * b15), 9000 + (4 * b15), length=15)
nd1 <- dnorm(SN,9000,1000)
nd2 <- dnorm(SR,9000,b15)

plot(SN, nd1, type="l",col="blue",
     xlab=" Population vs Sampling", 
     main=" Compact Fluorescent Light Bulbs",
     ylim=c(0,0.002))
lines(SR, nd2, col="red")

z <- (10500-9000)/1000
normalPlot(mean = 0, sd = 1, bounds = c(-Inf,z), tails = FALSE)

2.6.5 (e) Could you estimate the probabilities from parts (a) and (c) if the lifespans of light bulbs had a skewed distribution?

It would be difficult because - the sample size is too small (n<30) - without normal population distribution we would not be able to resolve. Sample could not generate normal distribution if population distribution is not normal.

2.7 Question#4.48 Same observation, different sample size.

Suppose you conduct a hypothesis test based on a sample where the sample size is n = 50, and arrive at a p-value of 0.08. You then refer back to your notes and discover that you made a careless mistake, the sample size should have been n = 500. Will your p-value increase, decrease, or stay the same? Explain.

This can be explained using the example. Lets assume (SD=10), for as sample size (n) = 50

z-score(50) = 1.41421356237309, then z-score will be used to calculate p-value, and p-value effects hypothesis testing.

z-score(500) = 0.447213595499958, in this case z-score decresed, this will impact p-value, and hypothesis testing

As the sample size(n) increases, p-value decreases. - If it’s a negative z-score, then this would cause the z-score to decrease which would make the p-value would decrease as well (left tail). - If it’s a positive z-score, then this would cause the z-score to increase which would make the p-value would decrease (right tail).