Assignment 1
Saayed Alam
September 7, 2018
1.8 A survey was conducted to study the smoking habits of UK residents. Below is a data matrix displaying a portion of the data collected in this survey. Note that “£” stands for British Pounds Sterling, “cig” stands for cigarettes, and “N/A” refers to a missing component of the data.
(a) What does each row of the data matrix represent?
(b) How many participants were included in the survey?
(c) Indicate whether each variable in the study is numerical or categorical. If numerical, identify as continuous or discrete. If categorical, indicate if the variable is ordinal.
library(tidyverse)
url <- ("https://raw.githubusercontent.com/jbryer/DATA606Fall2016/master/Data/Data%20from%20openintro.org/Ch%201%20Exercise%20Data/smoking.csv")
smoke_df <- read.csv(url)
#a Each row represents an observation of an invididual's smoking habit, marital status and income.
head(smoke_df)#b 1691 participants were included in the survey.
nrow(smoke_df)## [1] 1691#c There are 12 variables and they are :
#- gender: categorical
#- age: numerical - continous
#- marital status: categorical
#- highest qualification: ordinal - categorical
#- nationality: categorical
#- ethnicity: categorical
#- gross income: ordinal - categorical
#- region: categorical
#- smoke: categorical
#- amount weekends: numerical - discrete
#- amount weekdays: numerical - discrete
#- type: categorical
summary(smoke_df)## gender age maritalStatus highestQualification
## Female:965 Min. :16.00 Divorced :161 No Qualification :586
## Male :726 1st Qu.:34.00 Married :812 GCSE/O Level :308
## Median :48.00 Separated: 68 Degree :262
## Mean :49.84 Single :427 Other/Sub Degree :127
## 3rd Qu.:65.50 Widowed :223 Higher/Sub Degree:125
## Max. :97.00 A Levels :105
## (Other) :178
## nationality ethnicity grossIncome
## English :833 Asian : 41 5,200 to 10,400 :396
## British :538 Black : 34 10,400 to 15,600:268
## Scottish:142 Chinese: 27 2,600 to 5,200 :257
## Other : 71 Mixed : 14 15,600 to 20,800:188
## Welsh : 66 Refused: 13 20,800 to 28,600:155
## Irish : 23 Unknown: 2 Under 2,600 :133
## (Other) : 18 White :1560 (Other) :294
## region smoke amtWeekends amtWeekdays
## London :182 No :1270 Min. : 0.00 Min. : 0.00
## Midlands & East Anglia:443 Yes: 421 1st Qu.:10.00 1st Qu.: 7.00
## Scotland :148 Median :15.00 Median :12.00
## South East :252 Mean :16.41 Mean :13.75
## South West :157 3rd Qu.:20.00 3rd Qu.:20.00
## The North :426 Max. :60.00 Max. :55.00
## Wales : 83 NA's :1270 NA's :1270
## type
## :1270
## Both/Mainly Hand-Rolled: 10
## Both/Mainly Packets : 42
## Hand-Rolled : 72
## Packets : 297
##
## 1.10 Exercise 1.5 introduces a study where researchers studying the relationship between honesty, age, and self-control conducted an experiment on 160 children between the ages of 5 and 15. The researchers asked each child to toss a fair coin in private and to record the outcome (white or black) on a paper sheet, and said they would only reward children who report white. Half the students were explicitly told not to cheat and the others were not given any explicit instructions. Differences were observed in the cheating rates in the instruction and no instruction groups, as well as some differences across children’s characteristics within each group. (a) Identify the population of interest and the sample in this study.
(b) Comment on whether or not the results of the study can be generalized to the population, and if the findings of the study can be used to establish causal relationships.
#a The population of interest is all children between age 5 and 15. And the sample in the study is 160 children between the ages of 5 and 15.
#b No, the results of the study cannot be generalized to the population because this is an observational study. The students were not randomly assigned or randomly sampled. They were merely given a set of instructions and observed. Therefore, the results cannot be used to establish causal relationships.1.28 Below are excerpts from two articles published in the NY Times:
(a) An article titled Risks: Smokers Found More Prone to Dementia states the following: “Researchers analyzed data from 23,123 health plan members who participated in a voluntary exam and health behavior survey from 1978 to 1985, when they were 50-60 years old. 23 years later, about 25% of the group had dementia, including 1,136 with Alzheimer’s disease and 416 with vascular dementia. After adjusting for other factors, the researchers concluded that pack-a- day smokers were 37% more likely than nonsmokers to develop dementia, and the risks went up with increased smoking; 44% for one to two packs a day; and twice the risk for more than two packs.” Based on this study, can we conclude that smoking causes dementia later in life? Explain your reasoning.
(b) Another article titled The School Bully Is Sleepy states the following: “The University of Michigan study, collected survey data from parents on each child’s sleep habits and asked both parents and teachers to assess behavioral concerns. About a third of the students studied were identified by parents or teachers as having problems with disruptive behavior or bullying. The researchers found that children who had behavioral issues and those who were identified as bullies were twice as likely to have shown symptoms of sleep disorders.” A friend of yours who read the article says, “The study shows that sleep disorders lead to bullying in school children.” Is this statement justified? If not, how best can you describe the conclusion that can be drawn from this study?
#a Based on this study, we cannot conclude that smoking causes dementia later in life because this is an observational study. There might be correlation between smoking and dementia but correlation does not imply causation because there are outside factors we do not know about. And more importantly the study did not apply random assignment and random sampling.
#b This statement is not justified because this is an observational study. There might be correlation between sleep disorders and bullying but correlation does not imply causation because there there may be some other confounding variables such as child abuse which is causing the correlation. 1.36 A researcher is interested in the effects of exercise on mental health and he proposes the following study: Use stratified random sampling to ensure representative proportions of 18-30, 31-40 and 41-55 year olds from the population. Next, randomly assign half the subjects from each age group to exercise twice a week, and instruct the rest not to exercise. Conduct a mental health exam at the beginning and at the end of the study, and compare the results.
(a) What type of study is this?
(b) What are the treatment and control groups in this study?
(c) Does this study make use of blocking? If so, what is the blocking variable?
(d) Does this study make use of blinding?
(e) Comment on whether or not the results of the study can be used to establish a causal relationship between exercise and mental health, and indicate whether or not the conclusions can be generalized to the population at large.
(f) Suppose you are given the task of determining if this proposed study should get funding. Would you have any reservations about the study proposal?
#a This is an experimental study.
#b In this study, the treatment group is half of the subjects assigned to exercise twice a week. And the control group is the rest of the subjects instructed not to excercise.
#c Yes, this study makes use of blocking. The blocking variables are 18-30, 31-40, and 41-55 year olds from the population.
#d No, this study does not make use of blinding.
#e Yes, the results of the study can be used to extablish a causal relationship between exercise and mental health because both sampling and assignment were random. Hence, the conclusions can be generalized to the population at large.
#f I wouldn't have any reservation. I myself excercise daily. I would love to see the effect it has on my mental health. Moreoever, I would recommend adding a new attribute such as another age group 13-18 to see how it effects the study. 1.48 Below are the final exam scores of twenty introductory statistics students: 57, 66, 69, 71, 72, 73, 74, 77, 78, 78, 79, 79, 81, 81, 82, 83, 83, 88, 89, 94 Create a box plot of the distribution of these scores.
scores <- c(57, 66, 69, 71, 72, 73, 74, 77, 78, 78, 79, 79, 81, 81, 82, 83, 83, 88, 89, 94)
boxplot(scores)
1.50 Describe the distribution in the histograms below and match them to the box plots.
#a Symmetrical and Unimodal distribution. The histogram matches the box plot figure 2.
#b Uniform distribution. The histogram matches the boxplot figure 3.
#c Right skewed unimodal distribution. The histogram matches the boxplot figure 1. 1.56 For each of the following, state whether you expect the distribution to be symmetric, right skewed, or left skewed. Also specify whether the mean or median would best represent a typical observation in the data, and whether the variability of observations would be best represented using the standard deviation or IQR. Explain your reasoning. (a) Housing prices in a country where 25% of the houses cost below $350,000, 50% of the houses cost below $450,000, 75% of the houses cost below $1,000,000 and there are a meaningful number of houses that cost more than $6,000,000. (b) Housing prices in a country where 25% of the houses cost below $300,000, 50% of the houses cost below $600,000, 75% of the houses cost below $900,000 and very few houses that cost more than $1,200,000. (c) Number of alcoholic drinks consumed by college students in a given week. Assume that most of these students don’t drink since they are under 21 years old, and only a few drink excessively. (d) Annual salaries of the employees at a Fortune 500 company where only a few high level executives earn much higher salaries than the all other employees.
#a The distribution is left skewed. Median would best represent the distribution. IQR would best represent the variability.
#b The distribution is symmetrical. Mean would best represent the distribution. Standard deviation would best represent the variability.
#c The distribution is left skewed. Median would best represent the distribution. IQR would best represent the variability.
#d The distribution is right skewed. Mean would best represent the distribution. IQR would best represent the variability. 1.70 The Stanford University Heart Transplant Study was conducted to determine whether an experimental heart transplant program increased lifespan. Each patient entering the program was designated an official heart transplant candidate, meaning that he was gravely ill and would most likely benefit from a new heart. Some patients got a transplant and some did not. The variable transplant indicates which group the patients were in; patients in the treatment group got a transplant and those in the control group did not. Another variable called survived was used to indicate whether or not the patient was alive at the end of the study.
(a) Based on the mosaic plot, is survival independent of whether or not the patient got a transplant? Explain your reasoning. (b) What do the box plots below suggest about the efficacy (effectiveness) of the heart transplant treatment. (c) What proportion of patients in the treatment group and what proportion of patients in the control group died? (d) One approach for investigating whether or not the treatment is e???ective is to use a randomization technique. i. What are the claims being tested? ii. The paragraph below describes the set up for such approach, if we were to do it without using statistical software. Fill in the blanks with a number or phrase, whichever is appropriate. *We write alive on cards representing patients who were alive at the end of the study, and dead on cards representing patients who were not. Then, we shu???e these cards and split them into two groups: one group of size representing treatment, and another group of size representing control. We calculate the di???erence between the proportion of dead cards in the treatment and control groups (treatment - control) and record this value. We repeat this 100 times to build a distribution centered at . Lastly, we calculate the fraction of simulations where the simulated di???erences in proportions are . If this fraction is low, we conclude that it is unlikely to have observed such an outcome by chance and that the null hypothesis should be rejected in favor of the alternative. iii. What do the simulation results shown below suggest about the effectiveness of the transplant program?
library(openintro)
data("heartTr")
#a Based on the mosaic plot, survival is dependent on the patient getting a transplant because a significant number of patients died in the control group where they received no treatment.
mosaicplot(table(heartTr$transplant, heartTr$survived))
#b The box plots suggest the efficacy of heart transplant treatment is very high. A significal number of patients lived longer than the control group. The difference is so compelling because the third quartile of control group is lower than the first quartile of the treatment group.
boxplot(heartTr$survtime ~ heartTr$transplant)
#c
transplant_survival <- table(heartTr$survived, heartTr$transplant)
# 88% of patients in control group died
transplant_survival[2] / (transplant_survival[1] + transplant_survival[2])## [1] 0.8823529# 65% of patients in treatment group died
transplant_survival[4] / (transplant_survival[3] + transplant_survival[4]) ## [1] 0.6521739#d.i The claim being test is whether or not a heart transplant will increase a patient's lifespan.
#d.ii We write alive on [28] cards representing patients who were alive at the end of the study, and dead on [75] cards representing patients who were not. Then, we shuffle these cards and split them into two groups: one group of size [69] representing treatment, and another group of size [34] representing control. We calculate the difference between the proportion of dead cards in the treatment and control groups (treatment - control) and record this value. We repeat this 100 times to build a distribution centered at [approximately zero]. Lastly, we calculate the fraction of simulations where the simulated differences in proportions are [at least as extreme or greater than (-23.02%)]. If this fraction is low, we conclude that it is unlikely to have observed such an outcome by chance and that the null hypothesis should be rejected in favor of the alternative.
sum(heartTr$survived=="alive") ## [1] 28sum(heartTr$survived!="alive") ## [1] 75sum(heartTr$transplant=="treatment")## [1] 69sum(heartTr$transplant!="treatment")## [1] 34(transplant_survival[4] / (transplant_survival[3] + transplant_survival[4])) - (transplant_survival[2] / (transplant_survival[1] + transplant_survival[2]))## [1] -0.230179#e The simulation results show that the transplant does effect and improve survival. We conclude that the study results do provide strong evidence against the NULL hypothesis. That is, we do have sufficiently strong evidence to conclude the heart transplant was a success.