For this project, we analyze a cross-sectional dataset containing information about medical appointments.
The goal of our analysis is to investigate factors that are associated with how long patients wait for their medical appointments.
Our main research question is:
What factors are associated with the number of days a patient waits between scheduling and their medical appointment?
Our dependent variable, or Y variable, is WaitingDays.
We create this variable by calculating the number of days between the date an appointment was scheduled and the date the appointment occurred.
We then use multiple linear regression to examine whether characteristics such as age, gender, health conditions, SMS reminders, and appointment attendance are associated with waiting time.
Setup
# Load packages used in the projectlibrary(tidyverse)
── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.2.1 ✔ readr 2.2.0
✔ forcats 1.0.1 ✔ stringr 1.6.0
✔ ggplot2 4.0.3 ✔ tibble 3.3.1
✔ lubridate 1.9.5 ✔ tidyr 1.3.2
✔ purrr 1.2.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
library(knitr)
Import the Data
# Import the original datasetappointments <-read.csv("KaggleV2-May-2016.csv")# Display the first six rowshead(appointments)
PatientId AppointmentID Gender ScheduledDay AppointmentDay
1 2.987250e+13 5642903 F 2016-04-29T18:38:08Z 2016-04-29T00:00:00Z
2 5.589978e+14 5642503 M 2016-04-29T16:08:27Z 2016-04-29T00:00:00Z
3 4.262962e+12 5642549 F 2016-04-29T16:19:04Z 2016-04-29T00:00:00Z
4 8.679512e+11 5642828 F 2016-04-29T17:29:31Z 2016-04-29T00:00:00Z
5 8.841186e+12 5642494 F 2016-04-29T16:07:23Z 2016-04-29T00:00:00Z
6 9.598513e+13 5626772 F 2016-04-27T08:36:51Z 2016-04-29T00:00:00Z
Age Neighbourhood Scholarship Hipertension Diabetes Alcoholism Handcap
1 62 JARDIM DA PENHA 0 1 0 0 0
2 56 JARDIM DA PENHA 0 0 0 0 0
3 62 MATA DA PRAIA 0 0 0 0 0
4 8 PONTAL DE CAMBURI 0 0 0 0 0
5 56 JARDIM DA PENHA 0 1 1 0 0
6 76 REPÚBLICA 0 1 0 0 0
SMS_received No.show
1 0 No
2 0 No
3 0 No
4 0 No
5 0 No
6 0 No
The original dataset contains both the scheduling date and the appointment date.
We first convert these variables into dates.
# Convert the original date variables into datesappointments <- appointments |>mutate(ScheduledDate =as.Date(ScheduledDay),AppointmentDate =as.Date(AppointmentDay) )
Now we calculate the number of days between the two dates.
We now visualize our quantitative and qualitative variables.
Histograms are used for quantitative variables and bar charts are used for qualitative variables.
Age
ggplot( appointments,aes(x = Age)) +geom_histogram(binwidth =5,color ="black" ) +labs(title ="Distribution of Patient Age",x ="Age",y ="Number of Appointments" ) +theme_minimal()
The dataset contains patients from a wide range of ages.
Waiting Days
ggplot( appointments,aes(x = WaitingDays)) +geom_histogram(binwidth =5,color ="black" ) +labs(title ="Distribution of Appointment Waiting Times",x ="Waiting Time (Days)",y ="Number of Appointments" ) +theme_minimal()
Waiting time is particularly important because it is our dependent variable.
Scheduled Hour
ggplot( appointments,aes(x = ScheduledHour)) +geom_histogram(binwidth =1,color ="black" ) +labs(title ="Time of Day Appointments Were Scheduled",x ="Hour of Day",y ="Number of Appointments" ) +theme_minimal()
Appointments Per Patient
ggplot( appointments,aes(x = PatientAppointments)) +geom_histogram(binwidth =1,color ="black" ) +labs(title ="Appointments Per Patient",x ="Number of Appointments",y ="Number of Observations" ) +theme_minimal()
Gender
ggplot( appointments,aes(x = Gender)) +geom_bar() +labs(title ="Appointments by Gender",x ="Gender",y ="Number of Appointments" ) +theme_minimal()
There are several important parts of the regression output.
Coefficients
A regression coefficient estimates the relationship between an explanatory variable and waiting time while holding the other variables constant.
For example, the coefficient for Age tells us the predicted change in waiting days associated with a one-year increase in age, holding the other variables constant.
For qualitative variables, coefficients represent differences compared with a reference category.
P-Values
We use a significance level of:
0.05
A p-value below 0.05 provides evidence of a statistically significant relationship between that explanatory variable and waiting time.
Because this dataset is very large, we should also pay attention to the size of the coefficients. A very small relationship could be statistically significant simply because we have many observations.
R-Squared
R-squared measures the proportion of variation in waiting time explained by the explanatory variables in our regression model.
A larger R-squared means that our model explains more of the differences in appointment waiting times.
Regression Diagnostics
Linear regression makes several assumptions.
We examine diagnostic plots to determine whether our model appears reasonable.
Residuals vs. Fitted Values
# Residuals versus fitted valuesplot( waiting_model,which =1)
Ideally, the residuals should be scattered around zero without a strong pattern.
Normal Q-Q Plot
# Normal Q-Q plotplot( waiting_model,which =2)
The Q-Q plot helps us examine whether the regression residuals are approximately normally distributed.
Predicted Waiting Time
We can use the regression model to generate a predicted waiting time for each appointment.
# Add model predictions to the datasetappointments <- appointments |>mutate(PredictedWaitingDays =predict(waiting_model) )
This graph compares the waiting times predicted by our model with the actual waiting times observed in the dataset.
Key Findings
Our analysis investigates the factors associated with the amount of time between scheduling and a medical appointment.
When interpreting our regression results, we focus on:
Which explanatory variables have p-values below 0.05.
Whether each relationship is positive or negative.
The size of the regression coefficients.
The R-squared value of the model.
Whether the regression diagnostic plots suggest that the linear regression model is reasonable.
These results allow us to identify which patient and appointment characteristics have the strongest associations with waiting time.
Limitations
There are several limitations to this analysis.
First, this is observational data. Therefore, our regression results identify associations but cannot prove that one variable causes another.
Second, there may be important factors affecting appointment waiting time that are not included in the dataset.
Third, some patients appear in the dataset multiple times, meaning that the observations may not be completely independent.
Finally, the dataset is very large. This means that even small relationships may have very small p-values. Therefore, both statistical significance and the size of the estimated relationships should be considered.
Conclusion
The purpose of this project was to investigate factors associated with medical appointment waiting times.
We created our dependent variable, WaitingDays, by calculating the number of days between the date an appointment was scheduled and the date the appointment occurred.
We then used patient age, gender, scholarship status, health conditions, SMS reminder status, appointment attendance, and appointment frequency as explanatory variables.
Through data cleaning, summary statistics, visualization, and multiple linear regression, we can determine which characteristics have the strongest relationships with appointment waiting time.
Overall, this project demonstrates how cross-sectional data and multiple linear regression can be used to analyze real-world healthcare scheduling patterns.
