Characteristic | Physician, N = 1471 | Physician Assistant, N = 51 | Nurse Practitioner, N = 41 | p-value2 |
---|---|---|---|---|
Years at UW | 3 (1, 13) | 7 (4, 9) | 6 (5, 7) | 0.8 |
Unknown | 1 | 0 | 0 | |
Sex | 0.004 | |||
Female | 42 (29%) | 3 (60%) | 4 (100%) | |
Male | 103 (71%) | 2 (40%) | 0 (0%) | |
Unknown | 2 | 0 | 0 | |
Months of usage | 21.2 (10.5, 21.6) | 21.5 (21.4, 21.5) | 21.6 (16.5, 21.6) | 0.7 |
Monthly hours active - Overall | 6 (3, 12) | 33 (18, 49) | 39 (34, 52) | <0.001 |
Monthly hours active - Clinical Review | 2.4 (0.9, 4.7) | 20.1 (9.5, 28.4) | 22.5 (20.4, 28.4) | <0.001 |
Monthly hours active - Communication | 0.12 (0.05, 0.22) | 0.62 (0.57, 1.22) | 2.62 (1.77, 3.40) | <0.001 |
Monthly hours active - Dashboard | 0.15 (0.10, 0.21) | 0.16 (0.07, 0.17) | 0.08 (0.02, 0.14) | 0.3 |
Monthly hours active - In Basket | 0.19 (0.09, 0.39) | 2.75 (0.62, 3.59) | 1.43 (0.68, 3.47) | 0.021 |
Monthly hours active - Orders | 0.04 (0.00, 0.21) | 2.27 (0.54, 5.34) | 4.82 (3.02, 6.51) | <0.001 |
Monthly hours active - Protocols | 0.32 (0.02, 1.00) | 0.00 (0.00, 0.00) | 0.00 (0.00, 0.02) | <0.001 |
Monthly hours active - Reading Palette | 1.24 (0.55, 2.94) | 0.00 (0.00, 0.60) | 0.00 (0.00, 0.00) | <0.001 |
Monthly hours active - Research | 0.003 (0.001, 0.014) | 0.008 (0.003, 0.020) | 0.004 (0.002, 0.010) | 0.8 |
Monthly hours active - Schedule/Patient Lists | 0.55 (0.15, 1.37) | 4.94 (2.85, 6.25) | 5.34 (4.05, 10.00) | <0.001 |
Monthly hours active - Visit Navigator | 0.01 (0.00, 0.05) | 1.63 (0.88, 1.70) | 1.37 (1.08, 2.28) | <0.001 |
Monthly hours active - Other | 0.12 (0.04, 0.23) | 0.44 (0.42, 1.22) | 0.61 (0.46, 0.69) | <0.001 |
1 Median (IQR); n (%) | ||||
2 Kruskal-Wallis rank sum test; Fisher’s exact test |
Radiologist Utilization of the EMR
Statistical analysis
Categorical variables are summarized by frequency and percentage. Continuous variables are summarized by median and interquartile range (IQR). The Kruskal-Wallis test is used to compare continuous variables across groups. The Wilcoxon rank-sum test is used to compare continuous variables between two groups. The chi-square test is used to compare categorical variables across groups. The Fisher exact test is used when the expected cell count is less than 5. Temporal trends of EMR usage by category are illustrated using area plot over time. P values < 0.05 are considered statistically significant. All analyeses are performed using R version 4.1.2.
User characteristics and user-level analysis
We first aggregate data at the user level. For each user, the number of months during which they have been using the system is defined as the difference between the latest and earliest dates of their records (ual_date
). The total number of hours active is the sum of the active times (number_of_seconds_active
), overall and by category (ual_category
). Finally, the overall and category-specific hours active per month is the total number of hours divided by the months of usage.
By training status
Comments:
- Physicians spend far less time using EMR than non-physicians, both overall and for each category.
- NP generally spends more time than PA, largely due to longer time in Clinical Review, Communication, and Orders (despite shorter time in Basket).
The next two sub-sections focus on physicians only.
By physician sex
There is little difference between the sexes.
Characteristic | Female, N = 421 | Male, N = 1031 | p-value2 |
---|---|---|---|
Training status | >0.9 | ||
Physician | 42 (100%) | 103 (100%) | |
Physician Assistant | 0 (0%) | 0 (0%) | |
Nurse Practitioner | 0 (0%) | 0 (0%) | |
Years at UW | 4 (1, 12) | 3 (1, 13) | 0.7 |
Unknown | 1 | 0 | |
Months of usage | 20.2 (11.3, 21.6) | 21.4 (9.4, 21.6) | >0.9 |
Monthly hours active - Overall | 7 (4, 13) | 6 (3, 11) | 0.3 |
Monthly hours active - Clinical Review | 2.51 (1.07, 3.87) | 2.17 (0.87, 4.83) | 0.8 |
Monthly hours active - Communication | 0.15 (0.04, 0.25) | 0.12 (0.05, 0.21) | 0.5 |
Monthly hours active - Dashboard | 0.15 (0.11, 0.22) | 0.14 (0.10, 0.21) | 0.5 |
Monthly hours active - In Basket | 0.17 (0.06, 0.63) | 0.20 (0.10, 0.36) | >0.9 |
Monthly hours active - Orders | 0.01 (0.00, 0.18) | 0.04 (0.00, 0.21) | 0.6 |
Monthly hours active - Protocols | 0.43 (0.04, 1.13) | 0.27 (0.03, 0.95) | 0.4 |
Monthly hours active - Reading Palette | 1.9 (0.9, 3.0) | 1.2 (0.5, 2.9) | 0.086 |
Monthly hours active - Research | 0.002 (0.001, 0.006) | 0.005 (0.001, 0.018) | 0.007 |
Monthly hours active - Schedule/Patient Lists | 0.51 (0.16, 1.23) | 0.58 (0.12, 1.42) | >0.9 |
Monthly hours active - Visit Navigator | 0.01 (0.00, 0.06) | 0.01 (0.00, 0.04) | 0.7 |
Monthly hours active - Other | 0.14 (0.05, 0.23) | 0.12 (0.04, 0.25) | 0.5 |
1 n (%); Median (IQR) | |||
2 Fisher’s exact test; Wilcoxon rank sum test |
By physician years at UW
Characteristic | 0 - 5, N = 861 | 5 - 15, N = 371 | 15+, N = 231 | p-value2 |
---|---|---|---|---|
Training status | >0.9 | |||
Physician | 86 (100%) | 37 (100%) | 23 (100%) | |
Physician Assistant | 0 (0%) | 0 (0%) | 0 (0%) | |
Nurse Practitioner | 0 (0%) | 0 (0%) | 0 (0%) | |
Years at UW | 1 (0, 2) | 11 (8, 13) | 22 (18, 27) | <0.001 |
Sex | 0.6 | |||
Female | 24 (28%) | 12 (33%) | 5 (22%) | |
Male | 61 (72%) | 24 (67%) | 18 (78%) | |
Unknown | 1 | 1 | 0 | |
Months of usage | 13.1 (6.5, 21.5) | 21.6 (21.5, 21.6) | 21.6 (21.2, 21.6) | <0.001 |
Monthly hours active - Overall | 9 (5, 13) | 5 (2, 7) | 4 (2, 5) | <0.001 |
Monthly hours active - Clinical Review | 3.73 (1.63, 5.42) | 1.62 (0.54, 2.67) | 1.02 (0.68, 1.86) | <0.001 |
Monthly hours active - Communication | 0.14 (0.05, 0.26) | 0.14 (0.04, 0.22) | 0.08 (0.04, 0.14) | 0.3 |
Monthly hours active - Dashboard | 0.16 (0.11, 0.22) | 0.11 (0.08, 0.17) | 0.12 (0.09, 0.18) | 0.013 |
Monthly hours active - In Basket | 0.14 (0.07, 0.30) | 0.33 (0.12, 0.69) | 0.28 (0.17, 0.42) | 0.005 |
Monthly hours active - Orders | 0.06 (0.01, 0.25) | 0.01 (0.00, 0.09) | 0.02 (0.00, 0.05) | 0.025 |
Monthly hours active - Protocols | 0.63 (0.16, 1.32) | 0.07 (0.01, 0.27) | 0.03 (0.00, 0.40) | <0.001 |
Monthly hours active - Reading Palette | 2.35 (0.91, 3.18) | 0.85 (0.51, 1.45) | 0.83 (0.47, 1.23) | <0.001 |
Monthly hours active - Research | 0.006 (0.002, 0.021) | 0.002 (0.001, 0.009) | 0.001 (0.000, 0.005) | <0.001 |
Monthly hours active - Schedule/Patient Lists | 0.80 (0.29, 1.52) | 0.29 (0.06, 0.85) | 0.11 (0.04, 0.54) | <0.001 |
Monthly hours active - Visit Navigator | 0.02 (0.00, 0.05) | 0.01 (0.00, 0.03) | 0.00 (0.00, 0.02) | 0.2 |
Monthly hours active - Other | 0.18 (0.09, 0.27) | 0.03 (0.02, 0.12) | 0.05 (0.03, 0.11) | <0.001 |
1 n (%); Median (IQR) | ||||
2 Fisher’s exact test; Kruskal-Wallis rank sum test; Pearson’s Chi-squared test |
Comments:
- New hires (< 5 years at UW) tend to spend more time, mainly driven by clinical review, schedule/patient lists, Protocols, and Reading Palette.
- Unclear whether it’s cohort or temporal effect (will be addressed in next section).
Temporal analysis
In this section we map out the monthly hours active over time.
By training status
The temporal trend of overall monthly hours active by training status is plotted in Figure 2, and decomposed by category in Figure 3.
Comments:
- PA and NP spend consistently and substantially more time on EMR than physicians do.
- There is no obvious time trend for physicians.
By physician sex
The temporal trend of overall monthly hours active by physician sex is plotted in Figure 4, and decomposed by category in Figure 5.
Comments:
- Females tend to spend slightly more time than males, particularly on Reading Palette.
By physician years at UW
The temporal trend of overall monthly hours active by physician years at UW is plotted in Figure 6, and decomposed by category in Figure 7.
Comments:
- Recent hires (\(<\) 5 yrs) tend to spend more time consistently than old hires (cohort effect, not temporal effect);
Physician seniority and division
Data in this section are restricted to 2021-12-01 – 2023-04-30.
Physician seniority
Characteristic | Before 2022-6-30, N = 1101 | After 2022-7-1, N = 1171 |
---|---|---|
Training | ||
Resident | 10 (9.8%) | 6 (5.4%) |
Fellow | 11 (11%) | 17 (15%) |
Attending | 81 (79%) | 89 (79%) |
Unknown | 8 | 5 |
1 n (%) |
The temporal trend of overall monthly hours active by physician seniority is plotted below.
Resident has too few data points. Combine it with fellow (Figure 8).
Summary statistics and formal tests on training status by two periods are tabulated in Table 4. Residents/Fellows spend substantially and significantly more time than attendings.
Period | Resident/Fellow, N = 341 | Attending, N = 891 | p-value2 |
---|---|---|---|
Before 2022-6-30 | 11.3 (10.0, 13.4) | 4.4 (2.5, 6.7) | <0.001 |
After 2022-7-1 | 12.6 (9.9, 15.0) | 4.1 (2.0, 6.7) | <0.001 |
Whole period | 12.6 (9.9, 13.7) | 4.1 (2.1, 6.6) | <0.001 |
1 Median (IQR) | |||
2 Wilcoxon rank sum test |
Physician division - attendings only
Characteristic | Before 2022-6-30, N = 811 | After 2022-7-1, N = 891 |
---|---|---|
Division | ||
MSK | 9 (11%) | 8 (9.0%) |
IR/NeuroIR | 13 (16%) | 13 (15%) |
Breast | 7 (8.6%) | 7 (7.9%) |
MRI/Abdominal | 17 (21%) | 19 (21%) |
Neuro | 10 (12%) | 12 (13%) |
Peds | 5 (6.2%) | 8 (9.0%) |
Chest/CV | 11 (14%) | 11 (12%) |
Community | 6 (7.4%) | 8 (9.0%) |
Nucs | 3 (3.7%) | 3 (3.4%) |
1 n (%) |
Because there is no switch in division, we focus on the whole period. Breast spends more time than other divisions (Figure 9 and Table 5).
Period | Breast, N = 71 | Chest/CV, N = 111 | Community, N = 81 | IR/NeuroIR, N = 131 | MRI/Abdominal, N = 181 | MSK, N = 91 | Neuro, N = 121 | Nucs, N = 31 | Peds, N = 81 | p-value2 |
---|---|---|---|---|---|---|---|---|---|---|
Whole period | 13.2 (11.9, 14.0) | 4.0 (2.6, 7.0) | 2.0 (1.8, 3.5) | 6.7 (5.3, 8.4) | 3.9 (1.8, 4.7) | 4.8 (4.1, 5.3) | 2.0 (1.1, 3.2) | 4.1 (2.6, 5.1) | 3.0 (1.8, 4.1) | <0.001 |
1 Median (IQR) | ||||||||||
2 Kruskal-Wallis rank sum test |
Period | Breast, N = 71 | Other, N = 821 | p-value2 |
---|---|---|---|
Whole period | 13.2 (11.9, 14.0) | 4.0 (2.0, 6.2) | <0.001 |
1 Median (IQR) | |||
2 Wilcoxon rank sum test |
Period | Breast, N = 71 | Other, N = 821 | p-value2 |
---|---|---|---|
Whole period | 13.2 (11.9, 14.0) | 4.0 (2.0, 6.2) | <0.001 |
1 Median (IQR) | |||
2 Wilcoxon rank sum test |
The temporal trend of overall monthly hours active for by Breast vs other is plotted in Figure 10, and decomposed by category in Figure 11.
Characteristic | Breast, N = 71 | Other, N = 831 | p-value2 |
---|---|---|---|
Years at UW | 10 (5, 12) | 10 (4, 16) | 0.6 |
Sex | 0.021 | ||
Female | 5 (71%) | 21 (26%) | |
Male | 2 (29%) | 61 (74%) | |
Unknown | 0 | 1 | |
Monthly hours active - Overall | 13.2 (11.9, 14.0) | 4.0 (2.0, 6.3) | <0.001 |
Monthly hours active - Overall (w/o Reading Palette) | 5.0 (4.7, 5.6) | 2.9 (1.3, 4.9) | 0.027 |
Monthly hours active - Clinical Review | 2.68 (2.03, 2.84) | 1.23 (0.55, 2.43) | 0.039 |
Monthly hours active - Communication | 0.17 (0.15, 0.21) | 0.15 (0.06, 0.26) | 0.4 |
Monthly hours active - Dashboard | 0.33 (0.31, 0.36) | 0.12 (0.08, 0.16) | <0.001 |
Monthly hours active - In Basket | 0.72 (0.34, 0.88) | 0.22 (0.09, 0.51) | 0.026 |
Monthly hours active - Orders | 0.01 (0.00, 0.02) | 0.04 (0.01, 0.10) | 0.11 |
Monthly hours active - Protocols | 0.21 (0.12, 0.25) | 0.07 (0.02, 0.34) | 0.3 |
Monthly hours active - Reading Palette | 7.98 (6.97, 8.69) | 0.85 (0.48, 1.21) | <0.001 |
Monthly hours active - Research | 0.008 (0.005, 0.040) | 0.006 (0.003, 0.012) | 0.3 |
Monthly hours active - Schedule/Patient Lists | 1.20 (0.80, 1.38) | 0.20 (0.06, 0.64) | 0.003 |
Monthly hours active - Visit Navigator | 0.00 (0.00, 0.01) | 0.01 (0.00, 0.03) | 0.10 |
Monthly hours active - Other | 0.11 (0.07, 0.19) | 0.06 (0.03, 0.18) | 0.2 |
1 Median (IQR); n (%) | |||
2 Wilcoxon rank sum test; Fisher’s exact test |
Comment:
- The difference between breast vs other is driven by Reading Palette.
- Without Reading Palette, breast still spends more time than other divisions (\(p=0.026\)).
Table 7 below summarizes the user characteristics and EMR usage by division, restricted to attendings only (same as Table 6).
Characteristic | Breast, N = 71 | Other, N = 831 | p-value2 |
---|---|---|---|
Years at UW | 10 (5, 12) | 10 (4, 16) | 0.6 |
Sex | 0.021 | ||
Female | 5 (71%) | 21 (26%) | |
Male | 2 (29%) | 61 (74%) | |
Unknown | 0 | 1 | |
Monthly hours active - Overall | 13.2 (11.9, 14.0) | 4.0 (2.0, 6.3) | <0.001 |
Monthly hours active - Overall (w/o Reading Palette) | 5.0 (4.7, 5.6) | 2.9 (1.3, 4.9) | 0.027 |
Monthly hours active - Clinical Review | 2.68 (2.03, 2.84) | 1.23 (0.55, 2.43) | 0.039 |
Monthly hours active - Communication | 0.17 (0.15, 0.21) | 0.15 (0.06, 0.26) | 0.4 |
Monthly hours active - Dashboard | 0.33 (0.31, 0.36) | 0.12 (0.08, 0.16) | <0.001 |
Monthly hours active - In Basket | 0.72 (0.34, 0.88) | 0.22 (0.09, 0.51) | 0.026 |
Monthly hours active - Orders | 0.01 (0.00, 0.02) | 0.04 (0.01, 0.10) | 0.11 |
Monthly hours active - Protocols | 0.21 (0.12, 0.25) | 0.07 (0.02, 0.34) | 0.3 |
Monthly hours active - Reading Palette | 7.98 (6.97, 8.69) | 0.85 (0.48, 1.21) | <0.001 |
Monthly hours active - Research | 0.008 (0.005, 0.040) | 0.006 (0.003, 0.012) | 0.3 |
Monthly hours active - Schedule/Patient Lists | 1.20 (0.80, 1.38) | 0.20 (0.06, 0.64) | 0.003 |
Monthly hours active - Visit Navigator | 0.00 (0.00, 0.01) | 0.01 (0.00, 0.03) | 0.10 |
Monthly hours active - Other | 0.11 (0.07, 0.19) | 0.06 (0.03, 0.18) | 0.2 |
1 Median (IQR); n (%) | |||
2 Wilcoxon rank sum test; Fisher’s exact test |
Breast
Characteristic | Fellow, N = 01 | Attending, N = 71 | p-value2 |
---|---|---|---|
Years at UW | NA (NA, NA) | 10.0 (4.5, 12.0) | |
Sex | >0.9 | ||
Female | 0 (NA%) | 5 (71%) | |
Male | 0 (NA%) | 2 (29%) | |
Monthly hours active - Overall | NA (NA, NA) | 13 (12, 14) | |
Monthly hours active - Overall (w/o Reading Palette) | NA (NA, NA) | 5.0 (4.7, 5.6) | |
Monthly hours active - Clinical Review | NA (NA, NA) | 2.68 (2.03, 2.84) | |
Monthly hours active - Communication | NA (NA, NA) | 0.17 (0.15, 0.21) | |
Monthly hours active - Dashboard | NA (NA, NA) | 0.33 (0.31, 0.36) | |
Monthly hours active - In Basket | NA (NA, NA) | 0.72 (0.34, 0.88) | |
Monthly hours active - Orders | NA (NA, NA) | 0.005 (0.004, 0.019) | |
Monthly hours active - Protocols | NA (NA, NA) | 0.21 (0.12, 0.25) | |
Monthly hours active - Reading Palette | NA (NA, NA) | 8.0 (7.0, 8.7) | |
Monthly hours active - Research | NA (NA, NA) | 0.008 (0.005, 0.040) | |
Monthly hours active - Schedule/Patient Lists | NA (NA, NA) | 1.20 (0.80, 1.38) | |
Monthly hours active - Other | NA (NA, NA) | 0.11 (0.07, 0.19) | |
Monthly hours active - Visit Navigator | NA (NA, NA) | 0.004 (0.000, 0.009) | |
1 Median (IQR); n (%) | |||
2 Fisher’s exact test |
The temporal trend of overall monthly hours active for Breast physicians by training status is plotted in Figure 13, and decomposed by category in Figure 14.
Comments:
- Generally attending << fellow. But there is an outlying attending (GEGIOS, ALISON R), which makes the test only borderline significant given the small sample size (Table 8).
Attending vs. Trainee (residents/fellows)
User characteristics and EMR usage by training status are summarized in Table 9. The temporal trend of overall monthly hours active for training status is plotted in Figure 15.
Characteristic | Resident/Fellow, N = 341 | Attending, N = 901 | p-value2 |
---|---|---|---|
Years at UW | 1 (1, 2) | 10 (4, 16) | <0.001 |
Unknown | 1 | 0 | |
Sex | >0.9 | ||
Female | 10 (30%) | 26 (29%) | |
Male | 23 (70%) | 63 (71%) | |
Unknown | 1 | 1 | |
Monthly hours active - Overall | 12.6 (9.9, 13.7) | 4.2 (2.1, 6.7) | <0.001 |
Monthly hours active - Overall (w/o Reading Palette) | 8.4 (7.4, 10.6) | 3.0 (1.4, 5.1) | <0.001 |
Monthly hours active - Clinical Review | 5.29 (4.12, 5.88) | 1.27 (0.63, 2.60) | <0.001 |
Monthly hours active - Communication | 0.19 (0.06, 0.28) | 0.16 (0.06, 0.26) | 0.6 |
Monthly hours active - Dashboard | 0.17 (0.12, 0.20) | 0.13 (0.08, 0.19) | 0.030 |
Monthly hours active - In Basket | 0.12 (0.09, 0.19) | 0.24 (0.09, 0.57) | 0.009 |
Monthly hours active - Orders | 0.26 (0.09, 0.39) | 0.04 (0.00, 0.09) | <0.001 |
Monthly hours active - Protocols | 1.08 (0.68, 1.53) | 0.10 (0.02, 0.34) | <0.001 |
Monthly hours active - Reading Palette | 2.97 (2.49, 3.36) | 0.91 (0.50, 1.48) | <0.001 |
Monthly hours active - Research | 0.010 (0.004, 0.033) | 0.006 (0.003, 0.013) | 0.065 |
Monthly hours active - Schedule/Patient Lists | 1.11 (0.70, 1.61) | 0.23 (0.06, 0.71) | <0.001 |
Monthly hours active - Visit Navigator | 0.05 (0.01, 0.08) | 0.01 (0.00, 0.03) | 0.003 |
Monthly hours active - Other | 0.20 (0.12, 0.42) | 0.06 (0.03, 0.18) | <0.001 |
1 Median (IQR); n (%) | |||
2 Wilcoxon rank sum test; Pearson’s Chi-squared test |