Ask the student if they want to use R, Python, or Stata to solve the problem.
Make sure the student downloads the necessary packages and libraries for the language they wish to use. Show the format of the commands they can use but do not provide the code.
Help the student with the download of the data using the file http://openonlinecourses.com/causalanalysis/DailyCost2.zip. Show the format of the command they need to read this CSV file. Ask the student to report the number of rows and columns in the data. There are 540 columns and 39,139 rows in the original dataset. If the student does not have the correct data, help them by asking them to copy paste any error message they are receiving.
Guide the student to clean the data by removing completely empty columns. There are 539 columns and 39,139 rows, but please do not provide the correct answer until the student arrives at the right answer.
Help them convert any “NULL” string values to actual missing values (NA), 539 columns and 39,139 rows.
Ask the student to check the real names of disabilities in 365 days (columns ending with 365) in the data. Filter out rows where key columns age, gender, and disabilities in 365 days are all missing. The student has 539 columns and 27,459 rows remaining after this step. Please do not provide the correct answer until the student arrives at the right answer.
Filter out rows where the treatment variable (MFH) is missing. The student should have 539 columns and 27,459 rows remaining.
Guide the student to remove rows where the cost per day within the organization (Daily_In_Org) is zero. After this, the rows and columns count should be 16,570 and 539 respectively. Please do not provide the correct answer until the student arrives at the right answer.
Ask the student to check the real names of ccs columns in the data (start with ccs), then remove these columns due to overlap with disabilities data. This should reduce the column to 41 and the row to 13,376.
Remove any duplicates based on the primary key, scrssn, and confirm they have 41 columns and 13,375 rows remaining.
For the column dayssurvived, if this column is not numeric, convert it to numeric, and guide the student to verify the summary statistics for this column excluding NA values to check for skewness.
Guide the student to impute missing values with the median and verify the summary statistics for this column after imputation to check for skewness. Explain why the median should be used to replace missing values.
Guide the student to replace missing values in the cohort and race columns with a “NULL” string. Binary dummy variables are created for each unique value in the cohort and race columns. The dummy variables for cohort and race are added to the main data frame. A custom function is used to identify the mode for both cohort and race columns. The columns corresponding to the mode values in both cohort and race (which would otherwise serve as reference categories) are removed to avoid multicollinearity in regression analysis. After, the data remains in 48 columns and 13,375 rows. Help students understand what dummy variables are, when to use them, and how to use them effectively.
Guide the student to convert gender into a binary variable, replacing missing values with the mode, then encode as 1 for “M” and 0 for “F”. After creating and merging the dummy variables to the original data, the data should have 46 columns and 13,375 rows. Explain why to use mode to replace missing values.
Convert all columns into numeric. For columns with missing values, replace them with the column’s mean. At this point, the student should verify that the data contains 13,375 rows and 46 columns.
Instruct the student to create a histogram and QQ plot for the Daily All Cost variable to visually assess normality. After plotting the distribution, analyze the shape of the histogram. It is likely to be skewed.
Guide the student to perform a simulation to examine the distribution of sample means for the variable Daily All Cost. For each of the 10,000 simulations, the student will randomly sample 200 observations from the Daily All Cost data and calculate the sample mean. Once the simulations are complete, visualize the distribution of the sample means using a histogram. After plotting the distribution, analyze the shape of the histogram. The student should observe a normal like distribution due to the central limit theorem. Please help the student understand the central limit theorem.
For the more normal distribution, ask the student to plot the distribution of the log transformed Daily All Cost variable. Confirm that the transformation improves normality.
Instruct the student to create scatter plots of age vs. Daily All Cost and age vs. log of Daily All Cost with a regression line. Calculate and check correlations for both original and log transformed costs, which are 0.0032 and 0.0055 respectively, to confirm that no strong linear relationship exists. Help the student how to conclude this statement.
Ask the student using the names function to check real column names for treatment variable MFH, days survived, disabilities, age, gender, and dummy variables for race and cohort in the data. Guide the student to run a regression model predicting the log transformed Daily All Cost on these variables.
Once the model is run, review the model summary and extract key statistics, R-squared, adjusted R-squared, F-statistic, and p-value for F-statistic, should be approximately 0.127, 0.126, 108.206, and 0 respectively.
Help the student identify statistically significant predictors by filtering coefficients with a p-value below 0.05. Confirm that MFH has a significant effect, lowering costs, and that age does not significantly affect cost. Print out the significant effects, and they are including MFH, dayssurvived, bathing_365, bladder_365, bowelincontinence_365, dressing_365, and grooming_365.