R Libraries:

Load necessary libraries -

library(openintro); data(heartTr)

Exercise 1.8 Smoking habits of UK residents:

A survey was conducted to study the smoking habits of UK residents. Below is a data matrix displaying a portion of the data collected in this survey. Note that “£” stands for British Pounds Sterling, “cig” stands for cigarettes, and “N/A”" refers to a missing component of the data.

Part (a): What does each row of the data matrix represent?

Ans: Each row in the matrix represents a case or a unit of observation.

Part (b): How many participants were included in the survey?

Ans: Total 1,691 participants were included in the survey.

Part (c): Indicate whether each variable in the study is numerical or categorical. If numerical, identify as continuous or discrete. If categorical, indicate if the variable is ordinal.

Ans: Below is the categorizaton of each of the variables included in this survey data set-

sex: Categorical/Nominal

age: Numerical/Discrete

marital: Categorical/Nominal

grossIncome: Numerical/Continous

smoke: Categorical/Nomnal

amtWeekends: Categorical/Ordinal

amtWeekdays: Categorical/Ordinal

Exercise 1.10 Cheaters, scope of inference:

Exercise 1.5 introduces a study where researchers studying the relationship between honesty, age, and self-control conducted an experiment on 160 children between the ages of 5 and 15. The researchers asked each child to toss a fair coin in private and to record the outcome (white or black) on a paper sheet, and said they would only reward children who report white. Half the students were explicitly told not to cheat and the others were not given any explicit instructions. Differences were observed in the cheating rates in the instruction and no instruction groups, as well as some differences across children’s characteristics within each group.

Part (a): Identify the population of interest and the sample in this study.

Ans:

Popolation of Interest: All children between the age of 5 and 15.

Sample: 160 Children between the age of 5 and 15.

Part (b): Comment on whether or not the results of the study can be generalized to the population, and if the findings of the study can be used to establish causal relationships.

Ans: The sample of 160 children between the age of 5 and 15 for this experiment were chosen on a random basis, so the sample is a fair representative of the population. Also the sample was divided in half between the treatment group (Cheldren explicitly instructed not to cheat) and control group (no explicit instruction). So causal relationship between associated explanatory and response variables like honesty, age and self-control can be established based on the data collected in the experiment.

Exercise 1.28 Reading the paper:

Below are excerpts from two articles published in the NY Times:

(a) An article titled Risks: Smokers Found More Prone to Dementia states the following: “Researchers analyzed data from 23,123 health plan members who participated in a voluntary exam and health behavior survey from 1978 to 1985, when they were 50-60 years old. 23 years later, about 25% of the group had dementia, including 1,136 with Alzheimer’s disease and 416 with vascular dementia. After adjusting for other factors, the researchers concluded that pack-aday smokers were 37% more likely than nonsmokers to develop dementia, and the risks went up with increased smoking; 44% for one to two packs a day; and twice the risk for more than two packs.” - Based on this study, can we conclude that smoking causes dementia later in life? Explain your reasoning.

Ans: The method of data collection in this particular scenario represents observational study of 23,123 health plan members of a particular age group through a survey to study how dementia may arise. Researchers in this example, had very little control over how the data was generated and also there could have been many other variables and factors affecting the health condition of the survey participants which were not part of the study. In general, observational studies can provide evidence of naturally occurring association between variables like in this example smoking habit and chance of having dementia, but they can not by themselves show causal connection. So even though the statistics collected here shows a high level of association between smoking habit and dementia that it may not be ignored just as a mere co-incidence but based on this study, it can not be concluded that smoking causes dementia later in life.

(b) Another article titled The School Bully Is Sleepy states the following: “The University of Michigan study, collected survey data from parents on each child’s sleep habits and asked both parents and teachers to assess behavioral concerns. About a third of the students studied were identified by parents or teachers as having problems with disruptive behavior or bullying. The researchers found that children who had behavioral issues and those who were identified as bullies were twice as likely to have shown symptoms of sleep disorders.” A friend of yours who read the article says, “The study shows that sleep disorders lead to bullying in school children.” Is this statement justified? If not, how best can you describe the conclusion that can be drawn from this study?

Ans: “The study shows that sleep disorders lead to bullying in school children.” - This statement is not fully justified. The article here represents a observational study through survey done by University of Michigan and not an actual experiment which can establish a causal connection between sleep disorder and bullying. When my friend read the article, he/she interprated the article as an anecdotal evidence since it composed of unusual cases having striking characteristics. There could be other confounding variables gender, socioeconomic status, ethnicity, symptoms of depression, exercise, using illegal drugs etc., that need to be studied before establishing a causal connection between bullying and sleep disorder.

Exercise 1.36 Exercise and mental health:

A researcher is interested in the effects of exercise on mental health and he proposes the following study: Use stratified random sampling to ensure representative proportions of 18-30, 31-40 and 41- 55 year olds from the population. Next, randomly assign half the subjects from each age group to exercise twice a week, and instruct the rest not to exercise. Conduct a mental health exam at the beginning and at the end of the study, and compare the results.

(a) What type of study is this?

Ans: This study can be considered as an randomized experiment.

(b) What are the treatment and control groups in this study?

Ans: The group of people from each stratum (age group in this case) who are advised to exercise twice a week is the ‘Treatment group’ and the other half who are told not to exercise is the ‘Control group’.

(c) Does this study make use of blocking? If so, what is the blocking variable?

Ans: Yes, this study is using blocking. The blocking variable is age group.

(d) Does this study make use of blinding?

Ans: No, this study is not using blinding. Since the group of people who are explicitly told not to exercise are aware that they are not part of the treatment group.

(e) Comment on whether or not the results of the study can be used to establish a causal relationship between exercise and mental health, and indicate whether or not the conclusions can be generalized to the population at large.

Ans: Since the study here represents a randomized experiment with properly defined Treatment and Control groups, using stratified sampling technique and blocking to reduce bias, the results of this study can be used to establish causal relationship. Also, the conclusion can be generalized to population at large, since the stratified sampling technique ensures apropriate level of randomization while determining the sample to reduce bias so that sample chosen is a good represntative of the overall population.

(f) Suppose you are given the task of determining if this proposed study should get funding. Would you have any reservations about the study proposal?

Ans: My only reservation for funding this study would be the absence of sufficient focus on identifying any other confounding variables that may impact the outcome. It is very imporatnt to exhaust identification and sudy of confounders to establish a causal relationship in such a study.

Exercise 1.48 Stats scores:

Below are the final exam scores of twenty introductory statistics students.

57, 66, 69, 71, 72, 73, 74, 77, 78, 78, 79, 79, 81, 81, 82, 83, 83, 88, 89, 94

Create a box plot of the distribution of these scores. The five number summary provided below may be useful.

Min Q1 Q2(Median) Q3 Max 57 72.5 78.5 82.5 94

examScores <- c(57, 66, 69, 71, 72, 73, 74, 77, 78, 78, 79, 79, 81, 81, 82, 83, 83, 88, 89, 94)
boxplot(examScores, main = "Final Exam Scores Boxplot",ylab = "Exam Scores")

summary(examScores)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##   57.00   72.75   78.50   77.70   82.25   94.00

Exercise 1.50 Mix-and-match:

Describe the distribution in the histograms below and match them to the box plots.

Ans:

Matching Outcome:

Histogram (a) -> Boxplot (2)

Histogram (b) -> Boxplot (3)

Histogram (c) -> Boxplot (1)

Histogram (a): This histogram represents ‘Unimodal’ distribution which is farly symmetrical in shape which is observed in the boxplot(2) with median lying close to the middle of Q1 and Q3 and similar no. of observations falling above and below upper and lower whiskers respectively.

Histogram (b): This histogram represents ‘Multimodal’ distribution without any apparent skewness representated by boxplot(3) with median lying close to the middle of Q1 and Q3 and all observations falling within upper and lower whiskers.

Histogram (c): This histogram represents ‘Unimodal’ distribution right significant right skewness also represented by the boxplot(1) with many outliers falling above upper whisker.

Exercise 1.56 Distributions and appropriate statistics, Part II:

For each of the following, state whether you expect the distribution to be symmetric, right skewed, or left skewed. Also specify whether the mean or median would best represent a typical observation in the data, and whether the variability of observations would be best represented using the standard deviation or IQR. Explain your reasoning.

(a) Housing prices in a country where 25% of the houses cost below $350,000, 50% of the houses cost below $450,000, 75% of the houses cost below $1,000,000 and there are a meaningful number of houses that cost more than $6,000,000.

Ans: The distribution is Right skewed. Median would best represent a typical observation since highly priced houses with cost more than $6,000,000 would significantly inflate the mean cost but will have little impact on median. Variability would be best represented by IQR since the right skewed distribution will inflate the standard deviation.

(b) Housing prices in a country where 25% of the houses cost below $300,000, 50% of the houses cost below $600,000, 75% of the houses cost below $900,000 and very few houses that cost more than $1,200,000.

Ans: The distribution is symmetric. In this case mean would best represent a typical observation since it’s an ideal representative of the middle of the data set due to symmetric distribution. In a perfectly symmetric distribution, mean = 0. Standard deviation will be the best representation of variability due to symmetric distribution, standard deviation = 1 for a perfectly symmetric distribution.

(c) Number of alcoholic drinks consumed by college students in a given week. Assume that most of these students don’t drink since they are under 21 years old, and only a few drink excessively.

Ans: The distribution in this case will be left skewed. Since most of the students don’t drink and only a few drink excessinvely, there will be very few high valued data points. These data points would represent outliers. So for this type of distribution median and IQR will be best choices for representing a typical observation and variability of the distribution respectively.

(d) Annual salaries of the employees at a Fortune 500 company where only a few high level executives earn much higher salaries than the all other employees.

Ans: The distribution in this case will be right skewed with possible bimodal shape. Since most of the employees’ salary is much lower than executives, the data density will more towards the left of the histogram. So for this type of distribution median and IQR will be best choices for representing a typical observation and variability of the distribution respectively. Mean and Standard deviation will be inflated due to high salaries of executives.

Exercise 1.70 Heart transplants:

The Stanford University Heart Transplant Study was conducted to determine whether an experimental heart transplant program increased lifespan. Each patient entering the program was designated an official heart transplant candidate, meaning that he was gravely ill and would most likely benefit from a new heart. Some patients got a transplant and some did not. The variable transplant indicates which group the patients were in; patients in the treatment group got a transplant and those in the control group did not. Another variable called survived was used to indicate whether or not the patient was alive at the end of the study.

data(heartTr, envir = e <- new.env())
summary(e$heartTr)
##        id          acceptyear         age         survived 
##  Min.   :  1.0   Min.   :67.00   Min.   : 8.00   alive:28  
##  1st Qu.: 26.5   1st Qu.:69.00   1st Qu.:41.00   dead :75  
##  Median : 49.0   Median :71.00   Median :47.00             
##  Mean   : 51.4   Mean   :70.62   Mean   :44.64             
##  3rd Qu.: 77.5   3rd Qu.:72.00   3rd Qu.:52.00             
##  Max.   :103.0   Max.   :74.00   Max.   :64.00             
##                                                            
##     survtime      prior        transplant      wait       
##  Min.   :   1.0   no :91   control  :34   Min.   :  1.00  
##  1st Qu.:  33.5   yes:12   treatment:69   1st Qu.: 10.00  
##  Median :  90.0                           Median : 26.00  
##  Mean   : 310.2                           Mean   : 38.42  
##  3rd Qu.: 412.0                           3rd Qu.: 46.00  
##  Max.   :1799.0                           Max.   :310.00  
##                                           NA's   :34
head(e$heartTr)
id acceptyear age survived survtime prior transplant wait
15 68 53 dead 1 no control NA
43 70 43 dead 2 no control NA
61 71 52 dead 2 no control NA
75 72 52 dead 2 no control NA
6 68 54 dead 3 no control NA
42 70 36 dead 3 no control NA

(a) Based on the mosaic plot, is survival independent of whether or not the patient got a transplant? Explain your reasoning.

Ans: Based on the mosaic plot, there is significant evidence that survival may be dependant on whether a patient got heart transplant or not. However it is difficult to conclusively determine whether there is a causal relationship between survival and heart transplant. Also, there is no sufficient evidence whether the study was conducted as a randomized experiment, in other words whether patients were chosen randomly to receive the heart tarnsplant. Since survival of a patient may be related to other confounding variables like high vs. low risk factors etc. which haven’t been exposed in the plot or in the available data set.

(b) What do the box plots below suggest about the efficacy (effectiveness) of the heart transplant treatment.

boxplot(e$heartTr$survtime ~ e$heartTr$transplant)

Ans: The boxplot shows that the survival time distribution for the ‘Treatment Group’ (patients who received the heart transplant) is much more wider represented by the length of the box compared to the ‘Control Group’ (Patients without the transplant). The Q1, median, Q3 and upper whisker are much higher for the treatment group than the control group. So overall the boxplot suggests that the transplat treatment is highly effective in increasing the average survival time of the patients in the treatment group.

(c) What proportion of patients in the treatment group and what proportion of patients in the control group died?

Ans: Based on the Contingency and Proportion Tables below, 88.23% patients died in Control Group and 65.21% patients died in Treatment group.

## Contingency table for transplant and survived variables:
tableSurvTrans <- table(e$heartTr$transplant,e$heartTr$survived)
tableSurvTrans
##            
##             alive dead
##   control       4   30
##   treatment    24   45
## Column Proportion Table
prop.table(tableSurvTrans, 1)
##            
##                 alive      dead
##   control   0.1176471 0.8823529
##   treatment 0.3478261 0.6521739

(d) One approach for investigating whether or not the treatment is effective is to use a randomization technique.

i. What are the claims being tested?

Ans: Below are the claims being tested:

H0:Indepencence Model - The variables survived and transplant are independent. They have no relationship, and the observed difference between the proportion of survival between control and treatment group, 23%, was due to chance.

H1:Alternative Model - The variables survived and transplant are not independent. The observed difference between the proportion of survival between control and treatment group, 23%, was not due to chance, and patients not receiving heart transplant are less likely to survive.

ii. The paragraph below describes the set up for such approach, if we were to do it without using statistical software. Fill in the blanks with a number or phrase, whichever is appropriate.

We write alive on ____28___ cards representing patients who were alive at the end of the study, and dead on ____75____ cards representing patients who were not. Then, we shuffle these cards and split them into two groups: one group of size ____69_____ representing treatment, and another group of size _____34_____ representing control. We calculate the difference between the proportion of dead cards in the treatment and control groups (treatment - control) and record this value. We repeat this 100 times to build a distribution centered at ____0____ . Lastly, we calculate the fraction of simulations where the simulated dfferences in proportions are ____from chance alone____. If this fraction is low, we conclude that it is unlikely to have observed such an outcome by chance and that the null hypothesis should be rejected in favor of the alternative.

iii. What do the simulation results shown below suggest about the effectiveness of the transplant program?

Ans: From the above stacked dot plot, there is not a single observation where the difference is at least 23% the difference observed in the original study. So it appears that a difference of at least 23% due to chance alone would only happen 0% of the time or almost never according to this plot which indicates a rare event. So in this case, the independence model can be rejected in favor of alternative model. So it can be concluded that patients receiving heart transplant are more likely to to survive.