DRAFT: Ortho Sports Med Mystery Caller Study
Tyler M. Muffly, MD
17 August, 2024
Setup
Parameterized directories
Bespoke functions
Read in data
Quality Check the Data
Are there any physicians included more than twice?
| id_number | physician_information | N |
|---|---|---|
| NA | NA | NA |
| ———: | :——————— | –: |
Variables of those physicians included more than twice?
| id_number | physician_information | reason_for_exclusions | insurance | business_days_until_appointment |
|---|---|---|---|---|
| NA | NA | NA | NA | NA |
| ———: | :——————— | :——————— | :——— | ——————————-: |
Filter data so there is only one NPI per insurance type
Find NPI numbers called more than once
| physician_information | calls_count |
|---|---|
| NA | NA |
| :——————— | ———–: |
Do they have exclusion and have a business_days_until_appointment >0?
Do they have business_days_until_appointment >0 but are an excluded category?
Do they have NA for business_days_until_appointment but are “Included” in the Reasons for exclusion category?
Check data assumptions
Check for normality
## $median
## [1] 12
##
## $iqr
## [1] 15## $median
## [1] 12
##
## $iqr
## [1] 15Non-parametric testing
Kruskal-Wallis Test: Non-parametric alternative to ANOVA used when the normality assumption is violated. Dunn’s Test: Post-hoc test for pairwise comparisons following a significant Kruskal-Wallis test, with Bonferroni correction for multiple comparisons.
Demographics of the Sample
## Our sample included 1043 physicians from 49 states, including the District of Columbia, excluding Delaware and South Dakota . We made calls to 523 physician offices that accepted Blue Cross/Blue Shield. two hundred sixty-six physician offices accepted Medicaid, giving a 51.2 % Medicaid acceptance rate for Orthopedic Sports Medicine practices.The median age of the dataset was 55(IQR 25th percentile 48 to 75th percentile 62).
The most common gender in the dataset was male (9444%). The most common training was MD (9789%). The most common specialty was Sports Medicine Orthopaedics (10000%).
## In our dataset, the most common gender was male, representing 94.4% of the total. The predominant specialty observed was Sports Medicine Orthopaedics, accounting for 100.0% of the entries. Additionally, the most prevalent professional qualification was MD, which constituted 97.9% of the dataset.| Median_business_days_until_appointment | Q1 | Q3 |
|---|---|---|
| 12 | 6 | 21 |
The median wait time across all insurance was 12 business days, with an interquartile range (IQR) of 6 to 21.
Scenarios
| scenario | Median_business_days_until_appointment | Q1 | Q3 |
|---|---|---|---|
| HIP scenario | 12 | 6 | 22 |
| KNEE scenario | 12 | 6 | 21 |
| SHOULDER scenario | 11 | 6 | 20 |
The median wait time across all joint scenarios (hips, shoulder, knee) was 12 business days, with an interquartile range (IQR) of 6 to 22.
| insurance | Median_business_days_until_appointment | Q1 | Q3 |
|---|---|---|---|
| Blue Cross/Blue Shield | 12 | 6 | 20 |
| Medicaid | 13 | 7 | 23 |
Exclusions
## Of the total 1043 phones calls made, 874 (84%) successfully reached a representative, while 169 calls (16%) did not yield a connection even after two attempts. For the unsuccessful connections, 91 (54%) were redirected to voicemail, 55 (32%) listed an incorrect telephone number, and 23 (14%) reached a busy signal. For successful connections, the reasons for exclusion were 27 (3%) requiring a prior referral,43 (5%) reported that they were not currently accepting new patients and, 41 physician offices (5%) put the caller on hold for more than five minutes.## There were 1043 calls, with sports medicine orthopedists specializing in 346 hips, 367 shoulders, and 330 knees.Correlation
Visualizing the Data
Graph each variable
Business days by Subspecialty
NOTE: positively skewed distribution
sqrt(Business days) by insurance with transform
NOTE: as we have counts data of response variable, sqrt
transformation is suitable
Day of the week by insurance
Central Appointment Line by insurance
Physician Gender by insurance
Physician MD vs. DO by insurance
Table 1
Demographics of all physicians called
| Overall (N=524) | |
|---|---|
| Age (years) | |
| - Less than 50 years old | 155 (30.2%) |
| - 50 to 55 years old | 96 (18.7%) |
| - 56 to 60 years old | 101 (19.6%) |
| - 61 to 65 years old | 77 (15.0%) |
| - Greater than 65 years old | 85 (16.5%) |
| Gender | |
| - Female | 29 (5.5%) |
| - Male | 495 (94.5%) |
| Medical School Training | |
| - Osteopathic training | 11 (2.1%) |
| - Allopathic training | 513 (97.9%) |
| Academic Affiliation | |
| - Academic | 79 (15.1%) |
| - Not Academic | 445 (84.9%) |
| Rurality | |
| - Metropolitan area | 484 (92.4%) |
| - Rural area | 40 (7.6%) |
| Number of Phone Transfers | |
| - No transfers | 202 (38.6%) |
| - One transfer | 229 (43.8%) |
| - Two transfers | 75 (14.3%) |
| - More than two transfers | 17 (3.3%) |
| Insurance | |
| - Blue Cross/Blue Shield | 152 (29.0%) |
| - Medicaid | 372 (71.0%) |
| US Census Bureau Subdivision | |
| - East North Central | 62 (11.9%) |
| - East South Central | 39 (7.5%) |
| - Middle Atlantic | 75 (14.4%) |
| - Mountain | 36 (6.9%) |
| - New England | 34 (6.5%) |
| - Pacific | 81 (15.6%) |
| - South Atlantic | 105 (20.2%) |
| - West North Central | 27 (5.2%) |
| - West South Central | 61 (11.7%) |
| Orthopedic Scenario | |
| - Hip | 174 (33.2%) |
| - Knee | 167 (31.9%) |
| - Shoulder | 183 (34.9%) |
| Central Scheduling | |
| - No | 340 (64.9%) |
| - Yes, central scheduling number | 184 (35.1%) |
| Call time (minutes) | |
| - n | 464 |
| - Median (Q1, Q3) | 2.4 (1.3, 4.0) |
| Hold time (minutes) | |
| - n | 383 |
| - Median (Q1, Q3) | 0.5 (0.0, 1.7) |
| Day of the week Called | |
| - Monday | 168 (32.1%) |
| - Tuesday | 112 (21.4%) |
| - Wednesday | 159 (30.3%) |
| - Thursday | 52 (9.9%) |
| - Friday | 33 (6.3%) |
Table 1.5 - Split across Insurances
| Blue Cross/Blue Shield (N=523) | Medicaid (N=520) | Total (N=1043) | p value | |
|---|---|---|---|---|
| Age (years) | 1.00 | |||
| - Less than 50 years old | 161 (30.8%) | 160 (30.8%) | 321 (30.8%) | |
| - 50 to 55 years old | 96 (18.4%) | 94 (18.1%) | 190 (18.2%) | |
| - 56 to 60 years old | 99 (18.9%) | 101 (19.4%) | 200 (19.2%) | |
| - 61 to 65 years old | 79 (15.1%) | 79 (15.2%) | 158 (15.1%) | |
| - Greater than 65 years old | 88 (16.8%) | 86 (16.5%) | 174 (16.7%) | |
| Orthopedist Gender | 0.98 | |||
| - Female | 29 (5.5%) | 29 (5.6%) | 58 (5.6%) | |
| - Male | 494 (94.5%) | 491 (94.4%) | 985 (94.4%) | |
| Medical School Training | 0.99 | |||
| - Osteopathic training | 11 (2.1%) | 11 (2.1%) | 22 (2.1%) | |
| - Allopathic training | 512 (97.9%) | 509 (97.9%) | 1021 (97.9%) | |
| Academic Affiliation | 0.90 | |||
| - Academic | 78 (14.9%) | 79 (15.2%) | 157 (15.1%) | |
| - Not Academic | 445 (85.1%) | 441 (84.8%) | 886 (84.9%) | |
| US Census Bureau Subdivision | 1.00 | |||
| - East North Central | 62 (11.9%) | 62 (12.0%) | 124 (12.0%) | |
| - East South Central | 39 (7.5%) | 39 (7.6%) | 78 (7.5%) | |
| - Middle Atlantic | 75 (14.5%) | 74 (14.3%) | 149 (14.4%) | |
| - Mountain | 36 (6.9%) | 35 (6.8%) | 71 (6.9%) | |
| - New England | 34 (6.6%) | 35 (6.8%) | 69 (6.7%) | |
| - Pacific | 80 (15.4%) | 81 (15.7%) | 161 (15.6%) | |
| - South Atlantic | 105 (20.2%) | 103 (20.0%) | 208 (20.1%) | |
| - West North Central | 27 (5.2%) | 27 (5.2%) | 54 (5.2%) | |
| - West South Central | 61 (11.8%) | 60 (11.6%) | 121 (11.7%) | |
| Rurality | 0.98 | |||
| - Metropolitan area | 482 (92.3%) | 479 (92.3%) | 961 (92.3%) | |
| - Rural area | 40 (7.7%) | 40 (7.7%) | 80 (7.7%) | |
| Number of Phone Transfers | 0.78 | |||
| - No transfers | 194 (37.2%) | 192 (37.0%) | 386 (37.1%) | |
| - One transfer | 240 (46.1%) | 228 (43.9%) | 468 (45.0%) | |
| - Two transfers | 68 (13.1%) | 78 (15.0%) | 146 (14.0%) | |
| - More than two transfers | 19 (3.6%) | 21 (4.0%) | 40 (3.8%) | |
| Orthopedic Scenario | 1.00 | |||
| - Hip | 173 (33.1%) | 173 (33.3%) | 346 (33.2%) | |
| - Knee | 166 (31.7%) | 164 (31.5%) | 330 (31.6%) | |
| - Shoulder | 184 (35.2%) | 183 (35.2%) | 367 (35.2%) | |
| Central Scheduling | 0.95 | |||
| - No | 346 (66.2%) | 345 (66.3%) | 691 (66.3%) | |
| - Yes, central scheduling number | 177 (33.8%) | 175 (33.7%) | 352 (33.7%) | |
| Call time (minutes) | 0.28 | |||
| - n | 455 | 459 | 914 | |
| - Median (Q1, Q3) | 2.6 (1.5, 4.0) | 2.4 (1.2, 4.2) | 2.5 (1.3, 4.0) | |
| Hold time (minutes) | 0.24 | |||
| - n | 359 | 370 | 729 | |
| - Median (Q1, Q3) | 0.6 (0.0, 1.6) | 0.5 (0.0, 2.1) | 0.6 (0.0, 2.0) | |
| Day of the week Called | < 0.01 | |||
| - Monday | 16 (3.1%) | 162 (31.2%) | 178 (17.1%) | |
| - Tuesday | 106 (20.3%) | 77 (14.8%) | 183 (17.5%) | |
| - Wednesday | 176 (33.7%) | 123 (23.7%) | 299 (28.7%) | |
| - Thursday | 133 (25.4%) | 98 (18.8%) | 231 (22.1%) | |
| - Friday | 92 (17.6%) | 60 (11.5%) | 152 (14.6%) |
Table 2 - Included vs. Not Included
The table could help assess potential selection bias. By comparing the characteristics of those included versus those excluded, researchers can evaluate whether the exclusion of certain physicians (e.g., those not accepting Medicaid) might have skewed the results.
| Included in the Analysis (N=348) | Not Included (N=176) | Total (N=524) | p value | |
|---|---|---|---|---|
| Age (years) | 0.05 | |||
|
113 (32.5%) | 47 (26.7%) | 160 (30.5%) | |
|
71 (20.4%) | 25 (14.2%) | 96 (18.3%) | |
|
58 (16.7%) | 43 (24.4%) | 101 (19.3%) | |
|
54 (15.5%) | 25 (14.2%) | 79 (15.1%) | |
|
52 (14.9%) | 36 (20.5%) | 88 (16.8%) | |
| Orthopedist Gender | 0.27 | |||
|
22 (6.3%) | 7 (4.0%) | 29 (5.5%) | |
|
326 (93.7%) | 169 (96.0%) | 495 (94.5%) | |
| Medical School Training | 0.27 | |||
|
9 (2.6%) | 2 (1.1%) | 11 (2.1%) | |
|
339 (97.4%) | 174 (98.9%) | 513 (97.9%) | |
| Academic Affiliation | < 0.01 | |||
|
63 (18.1%) | 16 (9.1%) | 79 (15.1%) | |
|
285 (81.9%) | 160 (90.9%) | 445 (84.9%) | |
| US Census Bureau Subdivision | 0.03 | |||
|
48 (13.8%) | 14 (8.1%) | 62 (11.9%) | |
|
31 (8.9%) | 8 (4.7%) | 39 (7.5%) | |
|
45 (12.9%) | 30 (17.4%) | 75 (14.4%) | |
|
25 (7.2%) | 11 (6.4%) | 36 (6.9%) | |
| 25 (7.2%) | 9 (5.2%) | 34 (6.5%) | ||
|
51 (14.7%) | 30 (17.4%) | 81 (15.6%) | |
|
74 (21.3%) | 31 (18.0%) | 105 (20.2%) | |
|
18 (5.2%) | 9 (5.2%) | 27 (5.2%) | |
|
31 (8.9%) | 30 (17.4%) | 61 (11.7%) | |
| Rurality | 0.39 | |||
|
318 (91.6%) | 165 (93.8%) | 483 (92.4%) | |
|
29 (8.4%) | 11 (6.2%) | 40 (7.6%) | |
| Number of Phone Transfers | 0.02 | |||
|
119 (34.2%) | 83 (47.4%) | 202 (38.6%) | |
|
161 (46.3%) | 68 (38.9%) | 229 (43.8%) | |
|
57 (16.4%) | 18 (10.3%) | 75 (14.3%) | |
|
11 (3.2%) | 6 (3.4%) | 17 (3.3%) | |
| Insurance | < 0.01 | |||
|
152 (43.7%) | 0 (0.0%) | 152 (29.0%) | |
|
196 (56.3%) | 176 (100.0%) | 372 (71.0%) | |
| Orthopedic Scenario | 0.68 | |||
|
120 (34.5%) | 54 (30.7%) | 174 (33.2%) | |
|
109 (31.3%) | 58 (33.0%) | 167 (31.9%) | |
|
119 (34.2%) | 64 (36.4%) | 183 (34.9%) | |
| Central Scheduling | 0.04 | |||
|
215 (61.8%) | 125 (71.0%) | 340 (64.9%) | |
|
133 (38.2%) | 51 (29.0%) | 184 (35.1%) | |
| Call time (minutes) | < 0.01 | |||
|
312 | 152 | 464 | |
|
3.0 (1.7, 4.1) | 1.5 (1.0, 2.8) | 2.4 (1.3, 4.0) | |
| Hold time (minutes) | 0.59 | |||
|
259 | 124 | 383 | |
|
0.5 (0.0, 1.6) | 0.5 (0.0, 2.1) | 0.5 (0.0, 1.7) | |
| Day of the week Called | 0.69 | |||
|
99 (28.4%) | 69 (39.2%) | 168 (32.1%) | |
|
83 (23.9%) | 29 (16.5%) | 112 (21.4%) | |
|
114 (32.8%) | 45 (25.6%) | 159 (30.3%) | |
|
37 (10.6%) | 15 (8.5%) | 52 (9.9%) | |
|
15 (4.3%) | 18 (10.2%) | 33 (6.3%) |
Wait Time Figures
Waiting time in Days (Log Scale) for Blue Cross/Blue Shield versus Medicaid. The code you provided will create a scatter plot with points representing the relationship between the insurance variable (x-axis) and the days variable (y-axis). Additionally, it includes a line plot that connects points with the same npi value.
Line Plot
Here we show a scatterplot that compares the Private and Medicaid times. Notice that the graph is in logarithmic scale. Points above the diagonal line are providers for whom the Medicaid waiting time was longer than the private insurance waiting time.
We also see a strong linear association, indicating that providers with longer waiting time for private insurance tend to also have longer waiting times for Medicaid.
Scatter Plot
Density Plot
Poisson Model
The models need to be able to deal with NA in the
business_days_until_appointment outcome variable (456) and
also non-parametric data.
glmulti - Not working Scale the Continuous Variables
Visualize the weight of each model in the confidence set
Plot variable importance from the best model
business_days_until_appointment is transformed with a
square root function so that 0 is not infinity from
log(business_days_until_appointment).
Full Model
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance + age + gender +
## Provider.Credential.Text + academic + census_division + scenario +
## call_date_wday + central_number + number_of_transfers + call_time_minutes +
## hold_time_minutes + cbsatype10 + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 1363
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.16450 -0.42581 -0.07484 0.36506 3.06385
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.958 1.399
## Residual 1.500 1.225
## Number of obs: 344, groups: last, 242
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) 3.708717 1.231873 281.596126
## insuranceMedicaid 0.508749 0.187322 159.880431
## age -0.008028 0.013292 265.621309
## genderMale -0.410651 0.603733 267.018821
## Provider.Credential.TextMD 0.820682 0.711933 235.232253
## academicNot Academic -0.766408 0.325022 296.628838
## census_divisionEast South Central -0.948684 0.435440 238.561500
## census_divisionMiddle Atlantic 1.193933 0.490880 254.453705
## census_divisionMountain 0.355633 0.518432 249.061673
## census_divisionPacific 1.004186 0.422844 257.314345
## census_divisionSouth Atlantic -0.204612 0.365103 250.295108
## census_divisionWest North Central -0.248164 0.554524 214.937747
## census_divisionWest South Central -0.634149 0.445023 249.979574
## scenarioKNEE scenario 0.014862 0.285729 272.630226
## scenarioSHOULDER scenario 0.040034 0.293793 238.284374
## call_date_wday.L -0.208487 0.278514 268.075577
## call_date_wday.Q 0.808709 0.242061 231.163978
## call_date_wday.C -0.198789 0.226131 217.159075
## call_date_wday^4 0.244227 0.200439 243.493855
## central_numberYes 0.195503 0.202006 270.061421
## number_of_transfersOne transfer 0.244171 0.233139 261.626404
## number_of_transfersTwo transfers -0.207271 0.330547 269.859074
## number_of_transfersMore than two transfers 0.387221 0.487188 251.713832
## call_time_minutes 0.067395 0.105031 272.582465
## hold_time_minutes -0.037020 0.096827 243.696961
## cbsatype10Micro 0.720558 0.422705 214.240259
## t value Pr(>|t|)
## (Intercept) 3.011 0.002843 **
## insuranceMedicaid 2.716 0.007337 **
## age -0.604 0.546363
## genderMale -0.680 0.496975
## Provider.Credential.TextMD 1.153 0.250182
## academicNot Academic -2.358 0.019022 *
## census_divisionEast South Central -2.179 0.030335 *
## census_divisionMiddle Atlantic 2.432 0.015696 *
## census_divisionMountain 0.686 0.493366
## census_divisionPacific 2.375 0.018290 *
## census_divisionSouth Atlantic -0.560 0.575692
## census_divisionWest North Central -0.448 0.654946
## census_divisionWest South Central -1.425 0.155411
## scenarioKNEE scenario 0.052 0.958555
## scenarioSHOULDER scenario 0.136 0.891726
## call_date_wday.L -0.749 0.454775
## call_date_wday.Q 3.341 0.000974 ***
## call_date_wday.C -0.879 0.380324
## call_date_wday^4 1.218 0.224230
## central_numberYes 0.968 0.334007
## number_of_transfersOne transfer 1.047 0.295918
## number_of_transfersTwo transfers -0.627 0.531154
## number_of_transfersMore than two transfers 0.795 0.427473
## call_time_minutes 0.642 0.521630
## hold_time_minutes -0.382 0.702549
## cbsatype10Micro 1.705 0.089712 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1What variables are significant?
Single predictor models
This analysis explores the significance of various predictors on the
outcome variable business_days_until_appointment,
accounting for the random effects associated with physicians. The goal
is to identify which variables significantly influence the time to
appointment while controlling for variability across individual
physicians.
The step-by-step approach demonstrates how individual predictors are assessed for their significance in influencing the response variable while accounting for the random effects associated with repeated measures on physicians. Significant variables will be used in the final multivariate model to better understand their impact on appointment wait times.
Academic: Significant
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ academic + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2365.6
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3095 -0.4597 -0.0287 0.3725 4.1209
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.786 1.336
## Residual 1.838 1.356
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 4.115 0.219 472.293 18.788 <0.0000000000000002
## academicNot Academic -0.545 0.235 497.655 -2.319 0.0208
##
## (Intercept) ***
## academicNot Academic *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## acdmcNtAcdm -0.912Insurance: insurance has significant effect on response, specifically, medicaid insurance positively influence appt days as compared to BCBS insurance.
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2360.5
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3869 -0.4264 -0.0350 0.3509 3.9416
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.834 1.354
## Residual 1.777 1.333
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 3.50952 0.09861 514.64428 35.589 < 0.0000000000000002
## insuranceMedicaid 0.43207 0.12547 291.00043 3.444 0.000659
##
## (Intercept) ***
## insuranceMedicaid ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## insurncMdcd -0.416Scenario: scenario is non-significant factor, this variable will not be used
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ scenario + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2372
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3401 -0.4508 -0.0386 0.3637 4.0157
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.803 1.343
## Residual 1.856 1.362
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 3.5573 0.1535 449.3025 23.178
## scenarioKNEE scenario 0.1799 0.2135 491.9757 0.843
## scenarioSHOULDER scenario 0.1021 0.2084 474.1964 0.490
## Pr(>|t|)
## (Intercept) <0.0000000000000002 ***
## scenarioKNEE scenario 0.400
## scenarioSHOULDER scenario 0.624
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) sKNEEs
## scnrKNEEscn -0.691
## scSHOULDERs -0.715 0.509Central number: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ central_number + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2370.7
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3544 -0.4345 -0.0325 0.3674 4.0214
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.808 1.344
## Residual 1.846 1.359
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 3.5946 0.1044 497.0332 34.419 <0.0000000000000002 ***
## central_numberYes 0.1588 0.1486 489.0360 1.069 0.286
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## cntrl_nmbrY -0.507ntransf: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ number_of_transfers + (1 |
## last)
## Data: df3_filtered
##
## REML criterion at convergence: 2370.2
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3057 -0.4353 -0.0473 0.3602 4.0674
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.762 1.327
## Residual 1.877 1.370
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) 3.56696 0.13771 570.67232
## number_of_transfersOne transfer 0.15664 0.16644 541.70772
## number_of_transfersTwo transfers -0.06717 0.23735 546.92502
## number_of_transfersMore than two transfers 0.47488 0.38002 498.98013
## t value Pr(>|t|)
## (Intercept) 25.902 <0.0000000000000002 ***
## number_of_transfersOne transfer 0.941 0.347
## number_of_transfersTwo transfers -0.283 0.777
## number_of_transfersMore than two transfers 1.250 0.212
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) nm__Ot nm__Tt
## nmbr_f_trOt -0.717
## nmbr_f_trTt -0.515 0.429
## nmbr_f_Mttt -0.308 0.251 0.240##
## No transfers One transfer Two transfers
## 195 286 82
## More than two transfers
## 24## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ number_of_transfers + (1 |
## last)
## Data: reg_dat1
##
## REML criterion at convergence: 2372.1
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3258 -0.4536 -0.0542 0.3536 4.0637
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.789 1.338
## Residual 1.864 1.365
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) 3.56478 0.13786 571.92765
## number_of_transfersOne transfer 0.15938 0.16639 540.81333
## number_of_transfersMore than one transfer 0.05151 0.22069 558.80838
## t value Pr(>|t|)
## (Intercept) 25.858 <0.0000000000000002 ***
## number_of_transfersOne transfer 0.958 0.339
## number_of_transfersMore than one transfer 0.233 0.816
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) nm__Ot
## nmbr_f_trOt -0.716
## nmbr_f_Mtot -0.548 0.455call_time_minutes: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ call_time_minutes + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2301.5
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.2002 -0.4533 -0.0495 0.3595 4.0312
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.728 1.315
## Residual 1.962 1.401
## Number of obs: 566, groups: last, 378
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 3.61895 0.17943 562.67782 20.169 <0.0000000000000002
## call_time_minutes 0.01199 0.05394 522.53002 0.222 0.824
##
## (Intercept) ***
## call_time_minutes
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## cll_tm_mnts -0.861hold_time_minutes: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ hold_time_minutes + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 1825.4
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.2592 -0.4359 -0.0353 0.3639 3.9355
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.863 1.365
## Residual 1.919 1.385
## Number of obs: 446, groups: last, 312
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 3.53547 0.12305 406.81347 28.731 <0.0000000000000002
## hold_time_minutes 0.05021 0.06076 385.26435 0.826 0.409
##
## (Intercept) ***
## hold_time_minutes
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## hld_tm_mnts -0.547day_of_the_week: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ day_of_the_week + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2366.5
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.4053 -0.4654 -0.0537 0.3603 4.3012
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.796 1.340
## Residual 1.821 1.349
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 3.680019 0.092932 406.298507 39.599 <0.0000000000000002
## day_of_the_week.L 0.009987 0.183706 462.662847 0.054 0.9567
## day_of_the_week.Q 0.300627 0.176430 509.249872 1.704 0.0890
## day_of_the_week.C -0.313229 0.159312 480.315297 -1.966 0.0499
## day_of_the_week^4 -0.192978 0.143921 487.515261 -1.341 0.1806
##
## (Intercept) ***
## day_of_the_week.L
## day_of_the_week.Q .
## day_of_the_week.C *
## day_of_the_week^4
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) d___.L d___.Q d___.C
## dy_f_th_w.L 0.058
## dy_f_th_w.Q 0.254 0.179
## dy_f_th_w.C 0.096 0.168 0.076
## dy_f_th_w^4 0.011 0.084 0.152 -0.071age: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ age + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2326.8
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3113 -0.4413 -0.0439 0.3629 4.1363
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.746 1.322
## Residual 1.838 1.356
## Number of obs: 577, groups: last, 381
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 4.032600 0.563972 457.659757 7.150 0.00000000000345 ***
## age -0.007058 0.010170 463.587739 -0.694 0.488
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## age -0.987gender: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ gender + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2369.6
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3254 -0.4431 -0.0442 0.3508 4.0847
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.811 1.346
## Residual 1.849 1.360
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 3.8454 0.4048 471.6254 9.500 <0.0000000000000002 ***
## genderMale -0.2033 0.4133 476.8983 -0.492 0.623
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## genderMale -0.975census_division: Middle Atlantic, New England, and Pacific are all SIGNFICANT
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ census_division + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2342.9
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.5219 -0.4276 -0.0487 0.3680 4.1947
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.679 1.296
## Residual 1.812 1.346
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 3.42537 0.22578 386.43219 15.171
## census_divisionEast South Central -0.38043 0.36072 430.72156 -1.055
## census_divisionMiddle Atlantic 0.78097 0.33225 421.20311 2.351
## census_divisionMountain 0.12763 0.40744 415.26896 0.313
## census_divisionNew England 0.99910 0.40612 400.85407 2.460
## census_divisionPacific 0.90268 0.31307 439.04343 2.883
## census_divisionSouth Atlantic -0.13694 0.28825 440.14790 -0.475
## census_divisionWest North Central 0.03899 0.41120 387.12426 0.095
## census_divisionWest South Central -0.14390 0.35066 424.65809 -0.410
## Pr(>|t|)
## (Intercept) < 0.0000000000000002 ***
## census_divisionEast South Central 0.29218
## census_divisionMiddle Atlantic 0.01921 *
## census_divisionMountain 0.75425
## census_divisionNew England 0.01431 *
## census_divisionPacific 0.00413 **
## census_divisionSouth Atlantic 0.63497
## census_divisionWest North Central 0.92450
## census_divisionWest South Central 0.68173
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) cn_ESC cns_MA cnss_M cns_NE cnss_P cns_SA cn_WNC
## cnss_dvsESC -0.625
## cnss_dvsnMA -0.679 0.448
## cnss_dvsnMn -0.553 0.347 0.376
## cnss_dvsnNE -0.556 0.348 0.377 0.308
## cnss_dvsnPc -0.713 0.446 0.485 0.407 0.416
## cnss_dvsnSA -0.768 0.492 0.529 0.440 0.427 0.560
## cnss_dvsWNC -0.549 0.354 0.381 0.304 0.305 0.392 0.425
## cnss_dvsWSC -0.644 0.417 0.438 0.356 0.358 0.459 0.495 0.354rurality: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ cbsatype10 + (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 1872.1
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.2017 -0.4304 -0.0404 0.3251 3.6542
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 2.289 1.513
## Residual 1.694 1.301
## Number of obs: 459, groups: last, 301
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 3.5877 0.1128 295.9699 31.792 <0.0000000000000002 ***
## cbsatype10Micro 0.4859 0.3727 303.5686 1.304 0.193
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## cbstyp10Mcr -0.291Provider.Credential.Text: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ Provider.Credential.Text +
## (1 | last)
## Data: df3_filtered
##
## REML criterion at convergence: 2367.2
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3198 -0.4561 -0.0358 0.3584 4.0836
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.788 1.337
## Residual 1.855 1.362
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 2.8528 0.6029 420.0240 4.732 0.00000304 ***
## Provider.Credential.TextMD 0.8168 0.6097 419.1709 1.340 0.181
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## Prvdr.C.TMD -0.989- Significant Predictor:
- Insurance Type: Medicaid insurance was significantly associated with longer waiting times compared to Blue Cross/Blue Shield insurance.
- Non-Significant Predictors:
- Scenario: Specific scenarios like knee or shoulder appointments did not significantly impact waiting times.
- Central Number Usage: Whether a central scheduling number was used did not significantly affect waiting times.
- Number of Phone Transfers: The number of phone transfers was not a significant predictor of waiting times.
- Call Time: The duration of the call did not significantly influence the waiting times.
- Hold Time: The amount of time on hold during a call was not a significant factor.
- Day of the Week: The day of the week on which the call was made did not significantly impact waiting times.
- Physician Age: The age of the physician did not have a significant effect on waiting times.
- Physician Gender: There was no significant difference in waiting times based on the physician’s gender.
- Regional Variables: US Census Bureau subdivisions did not significantly impact waiting times.
- Physician Credentials: Whether the physician had an MD or another credential was not a significant factor.
- Conclusion:
- The analysis emphasizes the significant role of insurance type in determining appointment wait times, with Medicaid patients experiencing longer delays compared to those with Blue Cross/Blue Shield.
possible interactions
In this analysis, we explored possible interactions between significant variables (insurance, ability to contact the office, number of transfers, and whether the physician accepts Medicaid) and other predictors. Each interaction was modeled using a linear mixed-effects model to see if the interaction significantly influenced the number of business days until an appointment.
## [1] "physician_information" "age"
## [3] "gender" "academic"
## [5] "address" "census_division"
## [7] "insurance" "scenario"
## [9] "call_date_wday" "central_number"
## [11] "number_of_transfers" "call_time_minutes"
## [13] "reason_for_exclusions" "hold_time_minutes"
## [15] "notes" "day_of_the_week"
## [17] "business_days_until_appointment" "insurance_type"
## [19] "record_id" "ID"
## [21] "last" "phone"
## [23] "first" "middle"
## [25] "state" "does_the_physician_accept_medicaid"
## [27] "Subspecialty" "Provider.Credential.Text"
## [29] "zip" "cbsatype10"insurance*scenario: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance * scenario + (1 |
## last)
## Data: reg_dat1
##
## REML criterion at convergence: 2363.3
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.4078 -0.4250 -0.0473 0.3361 3.8684
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.823 1.350
## Residual 1.797 1.341
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) 3.45548 0.17095 565.69778
## insuranceMedicaid 0.29844 0.22184 298.24394
## scenarioKNEE scenario 0.13632 0.23736 576.31692
## scenarioSHOULDER scenario 0.03342 0.23072 572.44736
## insuranceMedicaid:scenarioKNEE scenario 0.15496 0.31887 297.38784
## insuranceMedicaid:scenarioSHOULDER scenario 0.23848 0.30493 289.44201
## t value Pr(>|t|)
## (Intercept) 20.213 <0.0000000000000002 ***
## insuranceMedicaid 1.345 0.180
## scenarioKNEE scenario 0.574 0.566
## scenarioSHOULDER scenario 0.145 0.885
## insuranceMedicaid:scenarioKNEE scenario 0.486 0.627
## insuranceMedicaid:scenarioSHOULDER scenario 0.782 0.435
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) insrnM sKNEEs sSHOUs iM:KNs
## insurncMdcd -0.449
## scnrKNEEscn -0.698 0.322
## scSHOULDERs -0.721 0.327 0.514
## insrM:KNEEs 0.316 -0.698 -0.446 -0.228
## iM:SHOULDEs 0.330 -0.730 -0.235 -0.438 0.509insurance*age: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance * age + (1 | last)
## Data: reg_dat1
##
## REML criterion at convergence: 2322.9
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.4084 -0.4438 -0.0314 0.3501 3.9742
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.781 1.335
## Residual 1.761 1.327
## Number of obs: 577, groups: last, 381
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 4.06482 0.62514 557.45233 6.502 0.000000000176 ***
## insuranceMedicaid -0.48319 0.83447 302.67534 -0.579 0.563
## age -0.01022 0.01122 557.83520 -0.911 0.363
## insuranceMedicaid:age 0.01696 0.01529 307.12168 1.109 0.268
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) insrnM age
## insurncMdcd -0.436
## age -0.987 0.432
## insrncMdcd: 0.423 -0.989 -0.428insurance*central_number: SIGNIFICANT INTERACTION
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance * central_number +
## (1 | last)
## Data: reg_dat1
##
## REML criterion at convergence: 2350.2
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.5032 -0.4336 -0.0431 0.3637 3.7654
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.782 1.335
## Residual 1.756 1.325
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 3.60729 0.11759 568.21823 30.676
## insuranceMedicaid 0.04918 0.16555 354.73450 0.297
## central_numberYes -0.27221 0.18789 535.37470 -1.449
## insuranceMedicaid:central_numberYes 1.03184 0.29711 450.25482 3.473
## Pr(>|t|)
## (Intercept) < 0.0000000000000002 ***
## insuranceMedicaid 0.766606
## central_numberYes 0.147998
## insuranceMedicaid:central_numberYes 0.000565 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) insrnM cntr_Y
## insurncMdcd -0.483
## cntrl_nmbrY -0.558 0.392
## insrncMd:_Y 0.358 -0.658 -0.632insurance*number_of_transfers: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance * number_of_transfers +
## (1 | last)
## Data: reg_dat1
##
## REML criterion at convergence: 2363.4
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3268 -0.4438 -0.0407 0.3448 3.9395
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.819 1.349
## Residual 1.801 1.342
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate
## (Intercept) 3.42965
## insuranceMedicaid 0.45984
## number_of_transfersOne transfer 0.16944
## number_of_transfersMore than one transfer -0.01724
## insuranceMedicaid:number_of_transfersOne transfer -0.08732
## insuranceMedicaid:number_of_transfersMore than one transfer 0.06781
## Std. Error
## (Intercept) 0.15993
## insuranceMedicaid 0.24192
## number_of_transfersOne transfer 0.20279
## number_of_transfersMore than one transfer 0.26908
## insuranceMedicaid:number_of_transfersOne transfer 0.32077
## insuranceMedicaid:number_of_transfersMore than one transfer 0.40041
## df t value
## (Intercept) 580.72669 21.445
## insuranceMedicaid 381.66279 1.901
## number_of_transfersOne transfer 544.43580 0.836
## number_of_transfersMore than one transfer 521.32502 -0.064
## insuranceMedicaid:number_of_transfersOne transfer 412.10769 -0.272
## insuranceMedicaid:number_of_transfersMore than one transfer 385.36697 0.169
## Pr(>|t|)
## (Intercept) <0.0000000000000002
## insuranceMedicaid 0.0581
## number_of_transfersOne transfer 0.4038
## number_of_transfersMore than one transfer 0.9489
## insuranceMedicaid:number_of_transfersOne transfer 0.7856
## insuranceMedicaid:number_of_transfersMore than one transfer 0.8656
##
## (Intercept) ***
## insuranceMedicaid .
## number_of_transfersOne transfer
## number_of_transfersMore than one transfer
## insuranceMedicaid:number_of_transfersOne transfer
## insuranceMedicaid:number_of_transfersMore than one transfer
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) insrnM nm__Ot n__tot iM:__t
## insurncMdcd -0.514
## nmbr_f_trOt -0.747 0.453
## nmbr_f_Mtot -0.559 0.338 0.456
## insrnM:__Ot 0.443 -0.806 -0.582 -0.269
## insM:__Mtot 0.335 -0.639 -0.283 -0.581 0.494insurance*call_time_minutes: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance * call_time_minutes +
## (1 | last)
## Data: reg_dat1
##
## REML criterion at convergence: 2293.5
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3597 -0.4479 -0.0530 0.3478 3.8833
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.766 1.329
## Residual 1.879 1.371
## Number of obs: 566, groups: last, 378
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 3.61966 0.21209 549.69728 17.066
## insuranceMedicaid 0.30813 0.33648 375.36605 0.916
## call_time_minutes -0.04400 0.06832 517.53694 -0.644
## insuranceMedicaid:call_time_minutes 0.05615 0.10224 392.71636 0.549
## Pr(>|t|)
## (Intercept) <0.0000000000000002 ***
## insuranceMedicaid 0.360
## call_time_minutes 0.520
## insuranceMedicaid:call_time_minutes 0.583
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) insrnM cll_t_
## insurncMdcd -0.529
## cll_tm_mnts -0.880 0.508
## insrncMd:__ 0.540 -0.918 -0.611insurance*hold_time_minutes: NS
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance * hold_time_minutes +
## (1 | last)
## Data: reg_dat1
##
## REML criterion at convergence: 1817.1
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3187 -0.4332 -0.0370 0.3538 3.7935
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.861 1.364
## Residual 1.845 1.358
## Number of obs: 446, groups: last, 312
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 3.417851 0.139040 440.958062 24.582
## insuranceMedicaid 0.409815 0.204098 264.396333 2.008
## hold_time_minutes -0.001015 0.076891 374.305058 -0.013
## insuranceMedicaid:hold_time_minutes 0.094485 0.114274 302.590845 0.827
## Pr(>|t|)
## (Intercept) <0.0000000000000002 ***
## insuranceMedicaid 0.0457 *
## hold_time_minutes 0.9895
## insuranceMedicaid:hold_time_minutes 0.4090
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) insrnM hld_t_
## insurncMdcd -0.481
## hld_tm_mnts -0.581 0.393
## insrncMd:__ 0.357 -0.678 -0.626insurance*day_of_the_week: SIGNIFICANT
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: business_days_until_appointment ~ insurance * day_of_the_week +
## (1 | last)
## Data: reg_dat1
##
## REML criterion at convergence: 2354.7
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.3637 -0.4391 -0.0441 0.3211 4.0859
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 1.860 1.364
## Residual 1.735 1.317
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 3.52857 0.12405 573.25952 28.444
## insuranceMedicaid 0.30486 0.16006 336.28186 1.905
## day_of_the_week.L 0.32939 0.32381 483.82407 1.017
## day_of_the_week.Q 0.21005 0.28941 500.17361 0.726
## day_of_the_week.C 0.06665 0.22345 503.48043 0.298
## day_of_the_week^4 -0.20933 0.17573 510.19051 -1.191
## insuranceMedicaid:day_of_the_week.L -0.38047 0.45206 430.15826 -0.842
## insuranceMedicaid:day_of_the_week.Q -0.41058 0.40233 399.20548 -1.021
## insuranceMedicaid:day_of_the_week.C -0.80719 0.36815 426.91536 -2.193
## insuranceMedicaid:day_of_the_week^4 -0.05277 0.31936 438.14907 -0.165
## Pr(>|t|)
## (Intercept) <0.0000000000000002 ***
## insuranceMedicaid 0.0577 .
## day_of_the_week.L 0.3096
## day_of_the_week.Q 0.4683
## day_of_the_week.C 0.7656
## day_of_the_week^4 0.2341
## insuranceMedicaid:day_of_the_week.L 0.4005
## insuranceMedicaid:day_of_the_week.Q 0.3081
## insuranceMedicaid:day_of_the_week.C 0.0289 *
## insuranceMedicaid:day_of_the_week^4 0.8688
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) insrnM d___.L d___.Q d___.C d___^4 iM:___.L iM:___.Q
## insurncMdcd -0.537
## dy_f_th_w.L -0.433 0.331
## dy_f_th_w.Q 0.584 -0.446 -0.485
## dy_f_th_w.C -0.297 0.233 0.508 -0.370
## dy_f_th_w^4 0.164 -0.110 -0.168 0.279 -0.165
## insrM:___.L 0.304 -0.013 -0.706 0.348 -0.361 0.116
## insrM:___.Q -0.392 0.429 0.328 -0.668 0.261 -0.195 0.060
## insrM:___.C 0.189 0.102 -0.312 0.232 -0.605 0.095 0.289 -0.044
## insrM:___^4 -0.096 0.027 0.111 -0.148 0.072 -0.561 0.045 0.197
## iM:___.C
## insurncMdcd
## dy_f_th_w.L
## dy_f_th_w.Q
## dy_f_th_w.C
## dy_f_th_w^4
## insrM:___.L
## insrM:___.Q
## insrM:___.C
## insrM:___^4 -0.103Poisson Model poisson
Given that the “business_days_until_appointment” variable represents the count of days until a new patient appointment and is a count variable, the Poisson regression model is appropriate for your data. It will model the relationship between the predictor variables and the count of days until a new patient appointment.
In the Poisson regression model, random effects are used to account for variability that is not explained by the fixed effects alone. The random effects for “last” in this model capture the variability in the number of business days until an appointment that is attributed to differences between physicians. By including last name as a random effect, the model acknowledges that observations within the same last name are likely to be more similar to each other than to observations from different last names. This clustering effect is accounted for by allowing the intercept to vary across last name. Random effects help to improve model fit by accounting for unexplained variability that is due to the hierarchical structure of the data (i.e., appointments are nested within physicians). This results in more accurate estimates of the fixed effects and a better understanding of the variability in appointment wait times.
Model Formula with only significant variables
$$ \[\begin{align*} P(\text{{Business Days until New Patient Appointment}} = x) &= \frac{e^{-\lambda} \cdot \lambda^x}{x!} \\ \sqrt{\lambda} &= \beta_0 \\ & + \beta_1 \cdot \text{{Patient Insurance}} \\ & + \beta_2 \cdot \text{{US Census Bureau Subdivision}} \\ & + \beta_3 \cdot \text{{Physician Academic Affiliation}} \\ & + ( 1 | \text{{Physician Name}}) \end{align*}\] $$
where:
Fixed effects include…
Random effects account for variability between physicians, modeled as a random intercept.
The random effect for physician suggests that there is substantial variability in appointment wait times between physician. Physicians with a higher random intercept will tend to have longer wait times compared to Physicians with a lower random intercept.
Model with only significant variables
## [1] "physician_information" "age"
## [3] "gender" "academic"
## [5] "address" "census_division"
## [7] "insurance" "scenario"
## [9] "call_date_wday" "central_number"
## [11] "number_of_transfers" "call_time_minutes"
## [13] "reason_for_exclusions" "hold_time_minutes"
## [15] "notes" "day_of_the_week"
## [17] "business_days_until_appointment" "insurance_type"
## [19] "record_id" "ID"
## [21] "last" "phone"
## [23] "first" "middle"
## [25] "state" "does_the_physician_accept_medicaid"
## [27] "Subspecialty" "Provider.Credential.Text"
## [29] "zip" "cbsatype10"## Generalized linear mixed model fit by maximum likelihood (Adaptive
## Gauss-Hermite Quadrature, nAGQ = 0) [glmerMod]
## Family: poisson ( log )
## Formula:
## business_days_until_appointment ~ academic + insurance + census_division +
## (1 | last)
## Data: df3
##
## AIC BIC logLik deviance df.resid
## 5066.4 5118.9 -2521.2 5042.4 575
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -6.7996 -0.5642 -0.0303 0.3005 10.3526
##
## Random effects:
## Groups Name Variance Std.Dev.
## last (Intercept) 0.8144 0.9025
## Number of obs: 587, groups: last, 387
##
## Fixed effects:
## Estimate Std. Error z value
## (Intercept) 2.88418 0.11959 24.117
## academicNot Academic -0.30480 0.09199 -3.313
## insuranceMedicaid 0.17974 0.02443 7.357
## census_divisionEast South Central -0.61473 0.14569 -4.219
## census_divisionMiddle Atlantic 0.01451 0.14053 0.103
## census_divisionMountain -0.30351 0.18888 -1.607
## census_divisionNew England 0.29847 0.17157 1.740
## census_divisionPacific 0.14616 0.12418 1.177
## census_divisionSouth Atlantic -0.84743 0.10258 -8.261
## census_divisionWest North Central 0.23039 0.18042 1.277
## census_divisionWest South Central -0.22729 0.16291 -1.395
## Pr(>|z|)
## (Intercept) < 0.0000000000000002 ***
## academicNot Academic 0.000922 ***
## insuranceMedicaid 0.000000000000187 ***
## census_divisionEast South Central 0.000024487785132 ***
## census_divisionMiddle Atlantic 0.917773
## census_divisionMountain 0.108083
## census_divisionNew England 0.081919 .
## census_divisionPacific 0.239211
## census_divisionSouth Atlantic < 0.0000000000000002 ***
## census_divisionWest North Central 0.201625
## census_divisionWest South Central 0.162962
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) acdmNA insrnM cn_ESC cns_MA cnss_M cns_NE cnss_P cns_SA
## acdmcNtAcdm -0.579
## insurncMdcd -0.165 0.044
## cnss_dvsESC -0.549 0.081 0.069
## cnss_dvsnMA -0.501 -0.041 0.046 0.528
## cnss_dvsnMn -0.225 -0.278 0.051 0.278 0.323
## cnss_dvsnNE -0.462 0.079 0.002 0.328 0.334 0.254
## cnss_dvsnPc -0.453 -0.137 0.084 0.403 0.436 0.448 0.504
## cnss_dvsnSA -0.495 -0.100 0.119 0.498 0.517 0.393 0.359 0.521
## cnss_dvsWNC -0.356 -0.115 0.066 0.413 0.456 0.281 0.259 0.359 0.400
## cnss_dvsWSC -0.433 -0.075 0.063 0.468 0.405 0.300 0.296 0.394 0.420
## cn_WNC
## acdmcNtAcdm
## insurncMdcd
## cnss_dvsESC
## cnss_dvsnMA
## cnss_dvsnMn
## cnss_dvsnNE
## cnss_dvsnPc
## cnss_dvsnSA
## cnss_dvsWNC
## cnss_dvsWSC 0.331`
| GVIF | Df | GVIF^(1/(2*Df)) | |
|---|---|---|---|
| academic | 1.180096 | 1 | 1.086322 |
| insurance | 1.025897 | 1 | 1.012865 |
| census_division | 1.201476 | 8 | 1.011538 |
Table of Model Coefficients
|
business days until appointment |
|||
|---|---|---|---|
| Predictors | Incidence Rate Ratios | CI | p |
| (Intercept) | 17.89 | 14.15 – 22.61 | <0.001 |
| academic [Not Academic] | 0.74 | 0.62 – 0.88 | 0.001 |
| insurance [Medicaid] | 1.20 | 1.14 – 1.26 | <0.001 |
|
census division [East South Central] |
0.54 | 0.41 – 0.72 | <0.001 |
|
census division [Middle Atlantic] |
1.01 | 0.77 – 1.34 | 0.918 |
|
census division [Mountain] |
0.74 | 0.51 – 1.07 | 0.108 |
|
census division [New England] |
1.35 | 0.96 – 1.89 | 0.082 |
| census division [Pacific] | 1.16 | 0.91 – 1.48 | 0.239 |
|
census division [South Atlantic] |
0.43 | 0.35 – 0.52 | <0.001 |
|
census division [West North Central] |
1.26 | 0.88 – 1.79 | 0.202 |
|
census division [West South Central] |
0.80 | 0.58 – 1.10 | 0.163 |
| Random Effects | |||
| σ2 | 0.08 | ||
| τ00 last | 0.81 | ||
| ICC | 0.91 | ||
| N last | 387 | ||
| Observations | 587 | ||
| Marginal R2 / Conditional R2 | 0.166 / 0.925 | ||
Model Performance
## We fitted a poisson mixed model (estimated using ML and BOBYQA optimizer) to
## predict business_days_until_appointment with academic, insurance and
## census_division (formula: business_days_until_appointment ~ academic +
## insurance + census_division). The model included last as random effect
## (formula: ~1 | last). The model's total explanatory power is substantial
## (conditional R2 = 0.93) and the part related to the fixed effects alone
## (marginal R2) is of 0.17. The model's intercept, corresponding to academic =
## Academic, insurance = Blue Cross/Blue Shield and census_division = East North
## Central, is at 2.88 (95% CI [2.65, 3.12], p < .001). Within this model:
##
## - The effect of academic [Not Academic] is statistically significant and
## negative (beta = -0.30, 95% CI [-0.49, -0.12], p < .001; Std. beta = -0.30, 95%
## CI [-0.49, -0.12])
## - The effect of insurance [Medicaid] is statistically significant and positive
## (beta = 0.18, 95% CI [0.13, 0.23], p < .001; Std. beta = 0.18, 95% CI [0.13,
## 0.23])
## - The effect of census division [East South Central] is statistically
## significant and negative (beta = -0.61, 95% CI [-0.90, -0.33], p < .001; Std.
## beta = -0.61, 95% CI [-0.90, -0.33])
## - The effect of census division [Middle Atlantic] is statistically
## non-significant and positive (beta = 0.01, 95% CI [-0.26, 0.29], p = 0.918;
## Std. beta = 0.01, 95% CI [-0.26, 0.29])
## - The effect of census division [Mountain] is statistically non-significant and
## negative (beta = -0.30, 95% CI [-0.67, 0.07], p = 0.108; Std. beta = -0.30, 95%
## CI [-0.67, 0.07])
## - The effect of census division [New England] is statistically non-significant
## and positive (beta = 0.30, 95% CI [-0.04, 0.63], p = 0.082; Std. beta = 0.30,
## 95% CI [-0.04, 0.63])
## - The effect of census division [Pacific] is statistically non-significant and
## positive (beta = 0.15, 95% CI [-0.10, 0.39], p = 0.239; Std. beta = 0.15, 95%
## CI [-0.10, 0.39])
## - The effect of census division [South Atlantic] is statistically significant
## and negative (beta = -0.85, 95% CI [-1.05, -0.65], p < .001; Std. beta = -0.85,
## 95% CI [-1.05, -0.65])
## - The effect of census division [West North Central] is statistically
## non-significant and positive (beta = 0.23, 95% CI [-0.12, 0.58], p = 0.202;
## Std. beta = 0.23, 95% CI [-0.12, 0.58])
## - The effect of census division [West South Central] is statistically
## non-significant and negative (beta = -0.23, 95% CI [-0.55, 0.09], p = 0.163;
## Std. beta = -0.23, 95% CI [-0.55, 0.09])
##
## Standardized parameters were obtained by fitting the model on a standardized
## version of the dataset. 95% Confidence Intervals (CIs) and p-values were
## computed using a Wald z-distribution approximation.## The marginal R² value of the model is 0.166 and the conditional R² value is 0.925## The marginal R² value of the model is 0.166 and the conditional R² value is 0.925 .## The marginal R² represents the proportion of variance explained by the fixed effects ( (Intercept), academicNot Academic, insuranceMedicaid, census_divisionEast South Central, census_divisionMiddle Atlantic, census_divisionMountain, census_divisionNew England, census_divisionPacific, census_divisionSouth Atlantic, census_divisionWest North Central, census_divisionWest South Central ) alone ( 16.65 %). The conditional R² represents the proportion of variance explained by both the fixed effects and the random effects ( last ) combined ( 92.51 %). This indicates how much of the variability in the outcome can be attributed to the fixed effects versus the entire model, including random effects.To determine which random effects were significant in your model, you need to look at the variance components for the random effects and their corresponding standard deviations. In mixed models, random effects themselves do not have p-values like fixed effects do. Instead, you evaluate their significance by looking at the variance of the random effects. If the variance is near zero, the random effect may not be contributing much to the model.
Here’s how you can extract and interpret the variance of the random effects to assess their significance:
## grp var1 var2 vcov sdcor
## 1 last (Intercept) <NA> 0.8144217 0.9024532## The random effects in the model are:## grp var1 var2 vcov sdcor Significant
## 1 last (Intercept) <NA> 0.8144217 0.9024532 Yes## The significant random effects are: last
Poisson model assumptions
Checking the binned residuals but because the data is non-parametric the residuals will not be normally distributed. Collinearity was tested.
Here we see that the Normal model is quite reasonable for this data,
as the residuals looks normally distributed.
Collinearity
Variance Inflation Factors (VIF) were calculated to assess multicollinearity among predictors. All VIF values were below the commonly used threshold of 5, suggesting that multicollinearity is not a concern for this model.
## OK: No outliers detected.
## - Based on the following method and threshold: cook (0.5).
## - For variable: (Whole model)Intraclass Correlation Coefficient
The Intraclass Correlation Coefficient (ICC) is a statistical measure used to evaluate the proportion of variance in a dependent variable that can be attributed to differences between groups or clusters. It is commonly used in the context of hierarchical or mixed models to quantify the degree of similarity within clusters.
## The intraclass correlation (ICC) of the model for the random effect group ' last ' is 0.449 .
## This indicates that 44.9 % of the variance in the outcome variable is attributable to differences between the last groups.##
## This is a low to moderate ICC for the last group, indicating that some variance is due to differences between these groups, but a substantial portion is within these groups.A low to moderate Intraclass Correlation Coefficient (ICC) for the group “physician last name” suggests that while there is some variation in the outcome variable (e.g., business days until appointment) that can be attributed to differences between individual physicians, a substantial portion of the variation occurs within these groups—meaning that much of the variability in appointment times is due to factors other than just the differences between physicians.
In practical terms, this indicates that:
Variation Between Physicians: The fact that the ICC is not zero means that there is some consistency in the appointment times associated with each physician. Some physicians might systematically have longer or shorter wait times, contributing to the variance in the data.
Variation Within Physicians: Since the ICC is low to moderate, it means that even within the same physician, there is considerable variability in appointment times. This could be due to a variety of factors, such as the type of insurance, the scenario, or other factors that are not captured by the physician’s identity alone.
Implications: The low to moderate ICC suggests that while the identity of the physician (as indicated by the last name) does have an effect, it is not the dominant factor driving differences in appointment times. Other factors—potentially those captured by fixed effects or residual variance—are also playing a significant role.
In summary, while who the physician is does matter to some extent, other variables are likely more influential in determining how long a patient waits for an appointment. This insight can guide you to look more closely at those other factors in your analysis or to consider whether there are ways to reduce variability within physicians, such as through standardized scheduling practices.
Dispersion
Overdispersion in your model implies that the variability in the observed data is greater than what the model predicts under the Poisson assumption. Specifically, in a Poisson model, the mean and variance of the count data are assumed to be equal.
## # Overdispersion test
##
## dispersion ratio = 1.896
## Pearson's Chi-Squared = 1090.412
## p-value = < 0.001## Interpretation of Overdispersion Test: Significant overdispersion detected. Consider using a Negative Binomial model or adding random effects to account for overdispersion.
## Warning: Autocorrelated residuals detected (p < .001).## [1] FALSETesting assumptions you can use the logLik function to get the log-likelihood of the model, and calculate the residual deviance as -2 * logLik(model). The residual degrees of freedom can be computed as the number of observations minus the number of parameters estimated (which includes both fixed effects and random effects).
The number of parameters estimated can be calculated as the number of fixed effects plus the number of random effects parameters. The number of fixed effects can be obtained from the length of fixef(model), and the number of random effects parameters can be obtained from the length of VarCorr(model).
If the dispersion parameter is considerably greater than 1, it indicates overdispersion. If it is less than 1, it indicates underdispersion. A value around 1 is considered ideal for Poisson regression.
## 'log Lik.' 4.890764 (df=12)This command will create a residuals plot that can help you check the
assumptions of your Poisson regression model. If the plot shows a random
scatter, then the assumptions are likely met. If the plot shows a clear
pattern or trend, then the assumptions might not be met, and you might
need to consider a different modeling approach.
Linearity of logit
The Poisson regression assumes that the log of the expected count is a linear function of the predictors. One way to check this is to plot the observed counts versus the predicted counts and see if the relationship looks linear.
Poisson Interactions
To include interaction terms in a regression model, you can use the : operator or the * operator in the formula. The : operator represents the interaction between two variables, while the * operator represents the interaction and also includes the main effects of the two variables. This will include interactions between insurance and each of the other significant variables (academic_affiliation, ACOG_District, central), in addition to the main effects of these variables.
Please note that interpreting interaction effects can be complex, especially in nonlinear models such as Poisson regression. The coefficients for the interaction terms represent the difference in the log rate of days for a one-unit change in x variable, for different levels of the other variables. However, the actual effects on the rate of days can vary depending on the values of the other variables.
The Anova test compares the simpler mini_poisson model with no interactions to the mini_poisson_interaction model. The very significant p-value (< 0.00000000000000022) suggests that the interaction model provides a substantially better fit to the data than the non-interaction model.
Based on your model mini_poisson_interaction, you have several interaction terms that might be of interest.
Insurance x academic
## Computing estimated marginal means...
## Estimated data:
## insurance academic rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield Academic 15.47704 1.4645970 Inf 12.85696 18.63105
## Medicaid Academic 18.52450 1.7552107 Inf 15.38489 22.30481
## Blue Cross/Blue Shield Not Academic 11.41071 0.6018770 Inf 10.28998 12.65350
## Medicaid Not Academic 13.65751 0.7485234 Inf 12.26648 15.20628
##
## Results are averaged over the levels of: census_division
## Confidence level used: 0.95
## Intervals are back-transformed from the log scale
## Range of estimated marginal means with CIs: 10.28998 22.30481
## Creating the plot...
## Saving plot to: ortho_sports_med/Figures/interaction_academic_comparison_plot_20240817_074434.png## Plot saved successfully.## $data
## academic = Academic:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 15.47704 1.4645970 Inf 12.85696 18.63105
## Medicaid 18.52450 1.7552107 Inf 15.38489 22.30481
##
## academic = Not Academic:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 11.41071 0.6018770 Inf 10.28998 12.65350
## Medicaid 13.65751 0.7485234 Inf 12.26648 15.20628
##
## Results are averaged over the levels of: census_division
## Confidence level used: 0.95
## Intervals are back-transformed from the log scale
##
## $plot
insurance x census divisions
## Computing estimated marginal means...
## Estimated data:
## insurance census_division rate SE df asymp.LCL
## Blue Cross/Blue Shield East North Central 15.360221 1.5400081 Inf 12.619914
## Medicaid East North Central 18.384683 1.8187654 Inf 15.144266
## Blue Cross/Blue Shield East South Central 8.306597 0.9550302 Inf 6.630685
## Medicaid East South Central 9.942185 1.1527294 Inf 7.921196
## Blue Cross/Blue Shield Middle Atlantic 15.584697 1.7387234 Inf 12.523714
## Medicaid Middle Atlantic 18.653359 2.0855939 Inf 14.982558
## asymp.UCL
## 18.69556
## 22.31845
## 10.40610
## 12.47880
## 19.39383
## 23.22353
##
## Results are averaged over the levels of: academic
## Confidence level used: 0.95
## Intervals are back-transformed from the log scale
## Range of estimated marginal means with CIs: 5.554889 33.0311
## Creating the plot...
## Saving plot to: ortho_sports_med/Figures/interaction_insurance_comparison_plot_20240817_074435.png## Plot saved successfully.## $data
## census_division = East North Central:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 15.360221 1.540008 Inf 12.619914 18.69556
## Medicaid 18.384683 1.818765 Inf 15.144266 22.31845
##
## census_division = East South Central:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 8.306597 0.955030 Inf 6.630685 10.40610
## Medicaid 9.942185 1.152729 Inf 7.921196 12.47880
##
## census_division = Middle Atlantic:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 15.584697 1.738723 Inf 12.523714 19.39383
## Medicaid 18.653359 2.085594 Inf 14.982558 23.22353
##
## census_division = Mountain:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 11.339279 1.987324 Inf 8.042734 15.98701
## Medicaid 13.572009 2.386715 Inf 9.615137 19.15723
##
## census_division = New England:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 20.702493 3.053952 Inf 15.504461 27.64322
## Medicaid 24.778861 3.634199 Inf 18.588297 33.03110
##
## census_division = Pacific:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 17.777620 1.817992 Inf 14.548803 21.72301
## Medicaid 21.278074 2.201240 Inf 17.372990 26.06094
##
## census_division = South Atlantic:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 6.582068 0.569798 Inf 5.554889 7.79919
## Medicaid 7.878092 0.696931 Inf 6.623994 9.36962
##
## census_division = West North Central:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 19.339858 3.077697 Inf 14.157781 26.41870
## Medicaid 23.147920 3.706487 Inf 16.912820 31.68166
##
## census_division = West South Central:
## insurance rate SE df asymp.LCL asymp.UCL
## Blue Cross/Blue Shield 12.237311 1.668419 Inf 9.367729 15.98592
## Medicaid 14.646865 2.009639 Inf 11.193209 19.16614
##
## Results are averaged over the levels of: academic
## Confidence level used: 0.95
## Intervals are back-transformed from the log scale
##
## $plot
- Intercept: The baseline (reference level) estimate is 2.88418. This represents the log of the expected number of business days until an appointment when the physician is in the reference categories (e.g., Academic = Yes, Insurance = Blue Cross/Blue Shield, and census_division = East North Central).
- Academic (Not Academic): The effect of not being academic is associated with a decrease in the expected log number of business days until an appointment by 0.30480.
- Insurance (Medicaid): The effect of having Medicaid insurance is associated with an increase in the expected log number of business days until an appointment by 0.17974.
- Census Division:
- East South Central: Decreases the expected log number of business days by 0.61473.
- Middle Atlantic: Increases the expected log number of business days by 0.01451 (not significant).
- Mountain: Decreases the expected log number of business days by 0.30351.
- New England: Increases the expected log number of business days by 0.29847.
- Pacific: Increases the expected log number of business days by 0.14616 (not significant).
- South Atlantic: Decreases the expected log number of business days by 0.84743.
- West North Central: Increases the expected log number of business days by 0.23039.
- West South Central: Decreases the expected log number of business days by 0.22729 (not significant).
Estimated Marginal Means (EMMeans) Interpretation
From the plot_and_save_emmeans function:
- Interaction between Insurance and Academic Status:
- Academic Status:
- For physicians who are academic:
- Blue Cross/Blue Shield: The expected number of business days until an appointment is 15.48 days (95% CI: 12.86, 18.63).
- Medicaid: The expected number of business days is higher at 18.52 days (95% CI: 15.38, 22.30).
- For physicians who are not academic:
- Blue Cross/Blue Shield: The expected number of business days until an appointment is 11.41 days (95% CI: 10.29, 12.65).
- Medicaid: The expected number of business days is 13.66 days (95% CI: 12.27, 15.21).
- For physicians who are academic:
- Academic Status:
Interpretation:
- Academic Status: Being an academic institution tends to increase the waiting time for an appointment, with longer waiting times observed for patients with Medicaid compared to those with Blue Cross/Blue Shield.
- Insurance: Medicaid patients generally experience longer waiting times compared to Blue Cross/Blue Shield patients, with the effect being more pronounced in academic settings.
- Geographical Variation: The census division also plays a significant role, with some regions like South Atlantic and East South Central showing significantly shorter waiting times, while others like New England have longer waiting times.
Overall, this analysis indicates that insurance type, academic status, and geographical region significantly influence the time patients wait for an appointment, with notable interactions between these factors.
fin
## - Allaire J, Xie Y, Dervieux C, McPherson J, Luraschi J, Ushey K, Atkins A, Wickham H, Cheng J, Chang W, Iannone R (2024). _rmarkdown: Dynamic Documents for R_. R package version 2.27, <https://github.com/rstudio/rmarkdown>. Xie Y, Allaire J, Grolemund G (2018). _R Markdown: The Definitive Guide_. Chapman and Hall/CRC, Boca Raton, Florida. ISBN 9781138359338, <https://bookdown.org/yihui/rmarkdown>. Xie Y, Dervieux C, Riederer E (2020). _R Markdown Cookbook_. Chapman and Hall/CRC, Boca Raton, Florida. ISBN 9780367563837, <https://bookdown.org/yihui/rmarkdown-cookbook>.
## - Arel-Bundock V (2022). "modelsummary: Data and Model Summaries in R." _Journal of Statistical Software_, *103*(1), 1-23. doi:10.18637/jss.v103.i01 <https://doi.org/10.18637/jss.v103.i01>.
## - Attali D, Baker C (2023). _ggExtra: Add Marginal Histograms to 'ggplot2', and More 'ggplot2' Enhancements_. R package version 0.10.1, <https://CRAN.R-project.org/package=ggExtra>.
## - Auguie B (2017). _gridExtra: Miscellaneous Functions for "Grid" Graphics_. R package version 2.3, <https://CRAN.R-project.org/package=gridExtra>.
## - Barrett T, Dowle M, Srinivasan A, Gorecki J, Chirico M, Hocking T (2024). _data.table: Extension of `data.frame`_. R package version 1.15.4, <https://CRAN.R-project.org/package=data.table>.
## - Bates D, Mächler M, Bolker B, Walker S (2015). "Fitting Linear Mixed-Effects Models Using lme4." _Journal of Statistical Software_, *67*(1), 1-48. doi:10.18637/jss.v067.i01 <https://doi.org/10.18637/jss.v067.i01>.
## - Bates D, Maechler M, Jagan M (2024). _Matrix: Sparse and Dense Matrix Classes and Methods_. R package version 1.7-0, <https://CRAN.R-project.org/package=Matrix>.
## - Ben-Shachar MS, Lüdecke D, Makowski D (2020). "effectsize: Estimation of Effect Size Indices and Standardized Parameters." _Journal of Open Source Software_, *5*(56), 2815. doi:10.21105/joss.02815 <https://doi.org/10.21105/joss.02815>, <https://doi.org/10.21105/joss.02815>.
## - Bengtsson H (2021). "A Unifying Framework for Parallel and Distributed Processing in R using Futures." _The R Journal_, *13*(2), 208-227. doi:10.32614/RJ-2021-048 <https://doi.org/10.32614/RJ-2021-048>, <https://doi.org/10.32614/RJ-2021-048>.
## - Bolker B, Robinson D (2024). _broom.mixed: Tidying Methods for Mixed Models_. R package version 0.2.9.5, <https://CRAN.R-project.org/package=broom.mixed>.
## - Calcagno V (2020). _glmulti: Model Selection and Multimodel Inference Made Easy_. R package version 1.0.8, <https://CRAN.R-project.org/package=glmulti>.
## - Couch SP, Bray AP, Ismay C, Chasnovski E, Baumer BS, Çetinkaya-Rundel M (2021). "infer: An R package for tidyverse-friendly statistical inference." _Journal of Open Source Software_, *6*(65), 3661. doi:10.21105/joss.03661 <https://doi.org/10.21105/joss.03661>.
## - Croissant Y, Millo G (2018). _Panel Data Econometrics with R_. Wiley. Croissant Y, Millo G (2008). "Panel Data Econometrics in R: The plm Package." _Journal of Statistical Software_, *27*(2), 1-43. doi:10.18637/jss.v027.i02 <https://doi.org/10.18637/jss.v027.i02>. Millo G (2017). "Robust Standard Error Estimators for Panel Models: A Unifying Approach." _Journal of Statistical Software_, *82*(3), 1-27. doi:10.18637/jss.v082.i03 <https://doi.org/10.18637/jss.v082.i03>.
## - Daróczi G, Tsegelskyi R (2022). _pander: An R 'Pandoc' Writer_. R package version 0.6.5, <https://CRAN.R-project.org/package=pander>.
## - Eddelbuettel D, Francois R, Allaire J, Ushey K, Kou Q, Russell N, Ucar I, Bates D, Chambers J (2024). _Rcpp: Seamless R and C++ Integration_. R package version 1.0.13, <https://CRAN.R-project.org/package=Rcpp>. Eddelbuettel D, François R (2011). "Rcpp: Seamless R and C++ Integration." _Journal of Statistical Software_, *40*(8), 1-18. doi:10.18637/jss.v040.i08 <https://doi.org/10.18637/jss.v040.i08>. Eddelbuettel D (2013). _Seamless R and C++ Integration with Rcpp_. Springer, New York. doi:10.1007/978-1-4614-6868-4 <https://doi.org/10.1007/978-1-4614-6868-4>, ISBN 978-1-4614-6867-7. Eddelbuettel D, Balamuta J (2018). "Extending R with C++: A Brief Introduction to Rcpp." _The American Statistician_, *72*(1), 28-36. doi:10.1080/00031305.2017.1375990 <https://doi.org/10.1080/00031305.2017.1375990>.
## - Falissard B (2022). _psy: Various Procedures Used in Psychometrics_. R package version 1.2, <https://CRAN.R-project.org/package=psy>.
## - Finkelstein N, Bartnik L, Hester J, Wickham H (2023). _mockery: Mocking Library for R_. R package version 0.4.4, <https://CRAN.R-project.org/package=mockery>.
## - Fox J, Venables B, Damico A, Salverda AP (2021). _english: Translate Integers into English_. R package version 1.2-6, <https://CRAN.R-project.org/package=english>.
## - Fox J, Weisberg S (2019). _An R Companion to Applied Regression_, 3rd edition. Sage, Thousand Oaks CA. <https://socialsciences.mcmaster.ca/jfox/Books/Companion/index.html>. Fox J, Weisberg S (2018). "Visualizing Fit and Lack of Fit in Complex Regression Models with Predictor Effect Plots and Partial Residuals." _Journal of Statistical Software_, *87*(9), 1-27. doi:10.18637/jss.v087.i09 <https://doi.org/10.18637/jss.v087.i09>. Fox J (2003). "Effect Displays in R for Generalised Linear Models." _Journal of Statistical Software_, *8*(15), 1-27. doi:10.18637/jss.v008.i15 <https://doi.org/10.18637/jss.v008.i15>. Fox J, Hong J (2009). "Effect Displays in R for Multinomial and Proportional-Odds Logit Models: Extensions to the effects Package." _Journal of Statistical Software_, *32*(1), 1-24. doi:10.18637/jss.v032.i01 <https://doi.org/10.18637/jss.v032.i01>.
## - Fox J, Weisberg S (2019). _An R Companion to Applied Regression_, Third edition. Sage, Thousand Oaks CA. <https://socialsciences.mcmaster.ca/jfox/Books/Companion/>.
## - Fox J, Weisberg S, Price B (2022). _carData: Companion to Applied Regression Data Sets_. R package version 3.0-5, <https://CRAN.R-project.org/package=carData>.
## - Frick H, Chow F, Kuhn M, Mahoney M, Silge J, Wickham H (2024). _rsample: General Resampling Infrastructure_. R package version 1.2.1, <https://CRAN.R-project.org/package=rsample>.
## - Gohel D, Moog S (2024). _officer: Manipulation of Microsoft Word and PowerPoint Documents_. R package version 0.6.6, <https://CRAN.R-project.org/package=officer>.
## - Gohel D, Skintzos P (2024). _flextable: Functions for Tabular Reporting_. R package version 0.9.6, <https://CRAN.R-project.org/package=flextable>.
## - Grolemund G, Wickham H (2011). "Dates and Times Made Easy with lubridate." _Journal of Statistical Software_, *40*(3), 1-25. <https://www.jstatsoft.org/v40/i03/>.
## - Grothendieck G, Kates L, Petzoldt T (2016). _proto: Prototype Object-Based Programming_. R package version 1.0.0, <https://CRAN.R-project.org/package=proto>.
## - Halekoh U, Højsgaard S, Yan J (2006). "The R Package geepack for Generalized Estimating Equations." _Journal of Statistical Software_, *15/2*, 1-11. Yan J, Fine JP (2004). "Estimating Equations for Association Structures." _Statistics in Medicine_, *23*, 859-880. Yan J (2002). "geepack: Yet Another Package for Generalized Estimating Equations." _R-News_, *2/3*, 12-14.
## - Hayashi H, Kojima H, Nishida K, Saito K, Yasuda Y (2024). _exploratory: R package for Exploratory_. R package version 10.2.1, commit 063e6164ddb6ea5abddf9c6aca376940d09a06d8, <https://github.com/exploratory-io/exploratory_func>.
## - Heinzen E, Sinnwell J, Atkinson E, Gunderson T, Dougherty G (2021). _arsenal: An Arsenal of 'R' Functions for Large-Scale Statistical Summaries_. R package version 3.6.3, <https://CRAN.R-project.org/package=arsenal>.
## - Kassambara A (2023). _ggpubr: 'ggplot2' Based Publication Ready Plots_. R package version 0.6.0, <https://CRAN.R-project.org/package=ggpubr>.
## - Kuhn M (2024). _modeldata: Data Sets Useful for Modeling Examples_. R package version 1.3.0, <https://CRAN.R-project.org/package=modeldata>.
## - Kuhn M (2024). _tune: Tidy Tuning Tools_. R package version 1.2.1, <https://CRAN.R-project.org/package=tune>.
## - Kuhn M, Couch S (2024). _workflowsets: Create a Collection of 'tidymodels' Workflows_. R package version 1.1.0, <https://CRAN.R-project.org/package=workflowsets>.
## - Kuhn M, Frick H (2024). _dials: Tools for Creating Tuning Parameter Values_. R package version 1.2.1, <https://CRAN.R-project.org/package=dials>.
## - Kuhn M, Vaughan D (2024). _parsnip: A Common API to Modeling and Analysis Functions_. R package version 1.2.1, <https://CRAN.R-project.org/package=parsnip>.
## - Kuhn M, Vaughan D, Hvitfeldt E (2024). _yardstick: Tidy Characterizations of Model Performance_. R package version 1.3.1, <https://CRAN.R-project.org/package=yardstick>.
## - Kuhn M, Wickham H (2020). _Tidymodels: a collection of packages for modeling and machine learning using tidyverse principles._. <https://www.tidymodels.org>.
## - Kuhn M, Wickham H, Hvitfeldt E (2024). _recipes: Preprocessing and Feature Engineering Steps for Modeling_. R package version 1.1.0, <https://CRAN.R-project.org/package=recipes>.
## - Kuhn, Max (2008). "Building Predictive Models in R Using the caret Package." _Journal of Statistical Software_, *28*(5), 1–26. doi:10.18637/jss.v028.i05 <https://doi.org/10.18637/jss.v028.i05>, <https://www.jstatsoft.org/index.php/jss/article/view/v028i05>.
## - Kuznetsova A, Brockhoff PB, Christensen RHB (2017). "lmerTest Package: Tests in Linear Mixed Effects Models." _Journal of Statistical Software_, *82*(13), 1-26. doi:10.18637/jss.v082.i13 <https://doi.org/10.18637/jss.v082.i13>.
## - Lenth R (2024). _emmeans: Estimated Marginal Means, aka Least-Squares Means_. R package version 1.10.3, <https://CRAN.R-project.org/package=emmeans>.
## - Liaw A, Wiener M (2002). "Classification and Regression by randomForest." _R News_, *2*(3), 18-22. <https://CRAN.R-project.org/doc/Rnews/>.
## - Lüdecke D (2018). "ggeffects: Tidy Data Frames of Marginal Effects from Regression Models." _Journal of Open Source Software_, *3*(26), 772. doi:10.21105/joss.00772 <https://doi.org/10.21105/joss.00772>.
## - Lüdecke D (2024). _sjPlot: Data Visualization for Statistics in Social Science_. R package version 2.8.16, <https://CRAN.R-project.org/package=sjPlot>.
## - Lüdecke D, Ben-Shachar M, Patil I, Makowski D (2020). "Extracting, Computing and Exploring the Parameters of Statistical Models using R." _Journal of Open Source Software_, *5*(53), 2445. doi:10.21105/joss.02445 <https://doi.org/10.21105/joss.02445>.
## - Lüdecke D, Ben-Shachar M, Patil I, Waggoner P, Makowski D (2021). "performance: An R Package for Assessment, Comparison and Testing of Statistical Models." _Journal of Open Source Software_, *6*(60), 3139. doi:10.21105/joss.03139 <https://doi.org/10.21105/joss.03139>.
## - Lüdecke D, Ben-Shachar M, Patil I, Wiernik B, Bacher E, Thériault R, Makowski D (2022). "easystats: Framework for Easy Statistical Modeling, Visualization, and Reporting." _CRAN_. R package, <https://easystats.github.io/easystats/>.
## - Lüdecke D, Patil I, Ben-Shachar M, Wiernik B, Waggoner P, Makowski D (2021). "see: An R Package for Visualizing Statistical Models." _Journal of Open Source Software_, *6*(64), 3393. doi:10.21105/joss.03393 <https://doi.org/10.21105/joss.03393>.
## - Lüdecke D, Waggoner P, Makowski D (2019). "insight: A Unified Interface to Access Information from Model Objects in R." _Journal of Open Source Software_, *4*(38), 1412. doi:10.21105/joss.01412 <https://doi.org/10.21105/joss.01412>.
## - Makowski D, Ben-Shachar M, Lüdecke D (2019). "bayestestR: Describing Effects and their Uncertainty, Existence and Significance within the Bayesian Framework." _Journal of Open Source Software_, *4*(40), 1541. doi:10.21105/joss.01541 <https://doi.org/10.21105/joss.01541>, <https://joss.theoj.org/papers/10.21105/joss.01541>.
## - Makowski D, Ben-Shachar M, Patil I, Lüdecke D (2020). "Estimation of Model-Based Predictions, Contrasts and Means." _CRAN_. <https://github.com/easystats/modelbased>.
## - Makowski D, Lüdecke D, Patil I, Thériault R, Ben-Shachar M, Wiernik B (2023). "Automated Results Reporting as a Practical Tool to Improve Reproducibility and Methodological Best Practices Adoption." _CRAN_. <https://easystats.github.io/report/>.
## - Makowski D, Wiernik B, Patil I, Lüdecke D, Ben-Shachar M (2022). "correlation: Methods for Correlation Analysis." Version 0.8.3, <https://CRAN.R-project.org/package=correlation>. Makowski D, Ben-Shachar M, Patil I, Lüdecke D (2020). "Methods and Algorithms for Correlation Analysis in R." _Journal of Open Source Software_, *5*(51), 2306. doi:10.21105/joss.02306 <https://doi.org/10.21105/joss.02306>, <https://joss.theoj.org/papers/10.21105/joss.02306>.
## - Merkle E, You D (2024). _nonnest2: Tests of Non-Nested Models_. R package version 0.5-7, <https://CRAN.R-project.org/package=nonnest2>.
## - Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F (2023). _e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien_. R package version 1.7-14, <https://CRAN.R-project.org/package=e1071>.
## - Microsoft, Weston S (2022). _foreach: Provides Foreach Looping Construct_. R package version 1.5.2, <https://CRAN.R-project.org/package=foreach>.
## - Miller TLboFcbA (2024). _leaps: Regression Subset Selection_. R package version 3.2, <https://CRAN.R-project.org/package=leaps>.
## - Muffly T (2023). _tyler: Common Functions for Mystery Caller or Audit Studies Evaluating Patient Access to Care_. R package version 1.2.0, https://mufflyt.github.io/tyler/, <https://your.package.url>.
## - Müller K (2020). _here: A Simpler Way to Find Your Files_. R package version 1.0.1, <https://CRAN.R-project.org/package=here>.
## - Müller K, Wickham H (2023). _tibble: Simple Data Frames_. R package version 3.2.1, <https://CRAN.R-project.org/package=tibble>.
## - Ogle DH, Doll JC, Wheeler AP, Dinno A (2023). _FSA: Simple Fisheries Stock Assessment Methods_. R package version 0.9.5, <https://CRAN.R-project.org/package=FSA>.
## - Ooms J (2024). _writexl: Export Data Frames to Excel 'xlsx' Format_. R package version 1.5.0, <https://CRAN.R-project.org/package=writexl>.
## - Paluszynska A, Biecek P, Jiang Y (2020). _randomForestExplainer: Explaining and Visualizing Random Forests in Terms of Variable Importance_. R package version 0.10.1, <https://CRAN.R-project.org/package=randomForestExplainer>.
## - Patil I, Makowski D, Ben-Shachar M, Wiernik B, Bacher E, Lüdecke D (2022). "datawizard: An R Package for Easy Data Preparation and Statistical Transformations." _Journal of Open Source Software_, *7*(78), 4684. doi:10.21105/joss.04684 <https://doi.org/10.21105/joss.04684>.
## - R Core Team (2024). _R: A Language and Environment for Statistical Computing_. R Foundation for Statistical Computing, Vienna, Austria. <https://www.R-project.org/>.
## - Rich B (2023). _table1: Tables of Descriptive Statistics in HTML_. R package version 1.4.3, <https://CRAN.R-project.org/package=table1>.
## - Robinson D, Hayes A, Couch S (2024). _broom: Convert Statistical Objects into Tidy Tibbles_. R package version 1.0.6, <https://CRAN.R-project.org/package=broom>.
## - Rosseel Y (2012). "lavaan: An R Package for Structural Equation Modeling." _Journal of Statistical Software_, *48*(2), 1-36. doi:10.18637/jss.v048.i02 <https://doi.org/10.18637/jss.v048.i02>.
## - Sarkar D (2008). _Lattice: Multivariate Data Visualization with R_. Springer, New York. ISBN 978-0-387-75968-5, <http://lmdvr.r-forge.r-project.org>.
## - Schloerke B, Cook D, Larmarange J, Briatte F, Marbach M, Thoen E, Elberg A, Crowley J (2024). _GGally: Extension to 'ggplot2'_. R package version 2.2.1, <https://CRAN.R-project.org/package=GGally>.
## - Slowikowski K (2024). _ggrepel: Automatically Position Non-Overlapping Text Labels with 'ggplot2'_. R package version 0.9.5, <https://CRAN.R-project.org/package=ggrepel>.
## - Tang Y, Horikoshi M, Li W (2016). "ggfortify: Unified Interface to Visualize Statistical Result of Popular R Packages." _The R Journal_, *8*(2), 474-485. doi:10.32614/RJ-2016-060 <https://doi.org/10.32614/RJ-2016-060>, <https://doi.org/10.32614/RJ-2016-060>. Horikoshi M, Tang Y (2018). _ggfortify: Data Visualization Tools for Statistical Analysis Results_. <https://CRAN.R-project.org/package=ggfortify>.
## - Urbanek S (2024). _rJava: Low-Level R to Java Interface_. R package version 1.0-11, <https://CRAN.R-project.org/package=rJava>.
## - Ushey K, Wickham H (2024). _renv: Project Environments_. R package version 1.0.7, <https://CRAN.R-project.org/package=renv>.
## - Vaughan D, Couch S (2024). _workflows: Modeling Workflows_. R package version 1.1.4, <https://CRAN.R-project.org/package=workflows>.
## - Vaughan D, Dancho M (2022). _furrr: Apply Mapping Functions in Parallel using Futures_. R package version 0.3.1, <https://CRAN.R-project.org/package=furrr>.
## - Wang T, Merkle EC (2018). "merDeriv: Derivative Computations for Linear Mixed Effects Models with Application to Robust Standard Errors." _Journal of Statistical Software, Code Snippets_, *87*(1), 1-16. doi:10.18637/jss.v087.c01 <https://doi.org/10.18637/jss.v087.c01>.
## - Wickham H (2016). _ggplot2: Elegant Graphics for Data Analysis_. Springer-Verlag New York. ISBN 978-3-319-24277-4, <https://ggplot2.tidyverse.org>.
## - Wickham H (2023). _forcats: Tools for Working with Categorical Variables (Factors)_. R package version 1.0.0, <https://CRAN.R-project.org/package=forcats>.
## - Wickham H (2023). _stringr: Simple, Consistent Wrappers for Common String Operations_. R package version 1.5.1, <https://CRAN.R-project.org/package=stringr>.
## - Wickham H, Averick M, Bryan J, Chang W, McGowan LD, François R, Grolemund G, Hayes A, Henry L, Hester J, Kuhn M, Pedersen TL, Miller E, Bache SM, Müller K, Ooms J, Robinson D, Seidel DP, Spinu V, Takahashi K, Vaughan D, Wilke C, Woo K, Yutani H (2019). "Welcome to the tidyverse." _Journal of Open Source Software_, *4*(43), 1686. doi:10.21105/joss.01686 <https://doi.org/10.21105/joss.01686>.
## - Wickham H, François R, Henry L, Müller K, Vaughan D (2023). _dplyr: A Grammar of Data Manipulation_. R package version 1.1.4, <https://CRAN.R-project.org/package=dplyr>.
## - Wickham H, Henry L (2023). _purrr: Functional Programming Tools_. R package version 1.0.2, <https://CRAN.R-project.org/package=purrr>.
## - Wickham H, Hester J, Bryan J (2024). _readr: Read Rectangular Text Data_. R package version 2.1.5, <https://CRAN.R-project.org/package=readr>.
## - Wickham H, Hester J, Csárdi G (2024). _pkgbuild: Find Tools Needed to Build R Packages_. R package version 1.4.4, <https://CRAN.R-project.org/package=pkgbuild>.
## - Wickham H, Pedersen T, Seidel D (2023). _scales: Scale Functions for Visualization_. R package version 1.3.0, <https://CRAN.R-project.org/package=scales>.
## - Wickham H, Vaughan D, Girlich M (2024). _tidyr: Tidy Messy Data_. R package version 1.3.1, <https://CRAN.R-project.org/package=tidyr>.
## - Wilke C (2024). _cowplot: Streamlined Plot Theme and Plot Annotations for 'ggplot2'_. R package version 1.1.3, <https://CRAN.R-project.org/package=cowplot>.
## - Xie Y (2024). _knitr: A General-Purpose Package for Dynamic Report Generation in R_. R package version 1.48, <https://yihui.org/knitr/>. Xie Y (2015). _Dynamic Documents with R and knitr_, 2nd edition. Chapman and Hall/CRC, Boca Raton, Florida. ISBN 978-1498716963, <https://yihui.org/knitr/>. Xie Y (2014). "knitr: A Comprehensive Tool for Reproducible Research in R." In Stodden V, Leisch F, Peng RD (eds.), _Implementing Reproducible Computational Research_. Chapman and Hall/CRC. ISBN 978-1466561595.
## - Zeileis A, Grothendieck G (2005). "zoo: S3 Infrastructure for Regular and Irregular Time Series." _Journal of Statistical Software_, *14*(6), 1-27. doi:10.18637/jss.v014.i06 <https://doi.org/10.18637/jss.v014.i06>.
## - Zeileis A, Hothorn T (2002). "Diagnostic Checking in Regression Relationships." _R News_, *2*(3), 7-10. <https://CRAN.R-project.org/doc/Rnews/>.
## - Zeileis A, Köll S, Graham N (2020). "Various Versatile Variances: An Object-Oriented Implementation of Clustered Covariances in R." _Journal of Statistical Software_, *95*(1), 1-36. doi:10.18637/jss.v095.i01 <https://doi.org/10.18637/jss.v095.i01>. Zeileis A (2004). "Econometric Computing with HC and HAC Covariance Matrix Estimators." _Journal of Statistical Software_, *11*(10), 1-17. doi:10.18637/jss.v011.i10 <https://doi.org/10.18637/jss.v011.i10>. Zeileis A (2006). "Object-Oriented Computation of Sandwich Estimators." _Journal of Statistical Software_, *16*(9), 1-16. doi:10.18637/jss.v016.i09 <https://doi.org/10.18637/jss.v016.i09>.
## - Zhu H (2024). _kableExtra: Construct Complex Table with 'kable' and Pipe Syntax_. R package version 1.4.0, <https://CRAN.R-project.org/package=kableExtra>.