1 Overview

1.1 Case description

1.1.1

Short description of the task along with some example images from eCREST. Report period is from the start of eCREST to December 2022.

1.1.2 Task outline

The case that all students completed referred to a female (Mrs Taru Gandhi) presenting to a GP with chest pain. Mrs Gandhi is fifty-eight years old, working as a dinner lady and of Asian background. The students had access to her medical records and tests and were asked to go through the case with thirty-two questions available in total.

An expert panel of GPs identified that:

  • Of the thirty-two questions there were twenty essential questions (3.1) and six relevant questions (3.5).
  • Of the eleven available, the panel identified six essential physical examinations (3.2).
  • Of the nine available, the panel identified three essential bedside tests (3.3).
  • According to the advisory panel there were six relevant diagnoses (4.3 - 4.5), out of the thirteen available.
  • The advisory panel identified that ‘Review in GP’ was the correct follow up management plan (5.1) out of the six possible follow up management plans.
  • The panel also identified three essential follow up investigations out of the six available (5.2).

Lastly, students provided feedback on the usability of eCREST. (6.1)

1.1.3 Example images 1

The Waiting Room and Taru Ghandi case informationThe Waiting Room and Taru Ghandi case information

The Waiting Room and Taru Ghandi case information

Initial perceptions and questionsInitial perceptions and questions

Initial perceptions and questions

Review daignoses and further testsReview daignoses and further tests

Review daignoses and further tests

final diagnoses and management planfinal diagnoses and management plan

final diagnoses and management plan

FeedbackFeedback

Feedback

Additional information and usabilityAdditional information and usability

Additional information and usability

1.2 Summary findings

1.2.1

Short descriptions of key findings by section with full details referenced in the report.

1.2.2 Student Characteristics

Age - in yrs: Overall, students were between 19 and 54 years old, with most around 25. (2.3.2) Pharmacists were the oldest and UK medical students were the youngest. Physician associates and international medical students were similar in age. (2.3.4)

Gender - male or female: Overall, most students were female. (2.4.2) Physician associates had the highest proportion of female students, followed by pharmacists, and then both UK and international medical students. (2.4.4)

1.2.3 Patient investigations

Essential questions: Overall, students asked just under 70% of essential questions on average. (3.1.2) Groups were similar in the proportion of essential questions asked, but there was some evidence that international medical students did not seem to perform as well as other groups. (3.1.4)

Inspection of each of the 20 questions showed that some were asked more than others with those about coughing and pain asked most often and those about pets, smoking, and patient experiences least often. (3.1.5) In addition, patterns emerged regarding how likely some student groups were to ask specific questions. For instance pharmacists were more likely than UK medical students to ask patients the following questions:

  • Pain travelling elsewhere?
  • Any ideas about cause?
  • How is it affecting you?

Essential Physical exams: Overall, students selected around 55% of essential physical exams on average. (3.2.2) Groups were similar in the proportion of essential physical exams selected. (3.2.4)

Inspection of each of the 6 exams showed that some were selected more than others with cardiovascular system and respiratory exams the most common. (3.2.5) In addition, pharmacists were more likely than all other groups to select the “lower limb and foot” exam.

Essential Bedside tests: Overall, students selected around 70% of essential bedside tests on average. (3.3.2) Physician associates selected a greater proportion of essential bedside tests than international medical students and pharmacists. UK medical students also selected a greater proportion compared to pharmacists. (3.3.4)

Inspection of each of the 3 tests showed that oxygen saturation was selected more often than temperature and blood pressure. (3.1.5) In addition, patterns emerged regarding how likely some student groups were to ask specific questions. For instance, physician associates were more likely to select “temperature” than international medical students and pharmacists and all groups were more likely to select “oxygen saturation test” than pharmacists. (3.3.5)

Relevant questions: Overall, students asked around 50% of relevant questions on average. (3.5.2) Groups were similar in the proportion of relevant questions asked. (3.5.4)

Combined measure: Combining the proportion of essential questions asked, physical exams selected, and bedside tests selected in a single measure found physician associates performed better than international medical students but no other differences. (3.4.4) This finding remained consistent when relevant questions were added to the combined measure. (3.6.4)

1.2.4 Diagnoses

Lung cancer: Overall, just over 70% of students included lung cancer somewhere in their final diagnosis. (4.1.2)

Groups were similar in the proportion who included lung cancer in their final diagnosis, but there was some evidence that pharmacists did not seem to perform as well as other groups and may have been less likely to have included lung cancer. (4.1.4)

LRTI: Overall, around 75% of students included ower respiratory tract infection somewhere in their final diagnosis. (4.2.2)

Pharmacists were less likely to have included lower respiratory tract infection in their final diagnosis than all other groups. (4.2.4)

Any relevant diagnosis: Overall, in their initial diagnosis, students included around 44% of the relevant diagnoses on average. (4.3.2) Physician associates included a greater proportion of relevant diagnoses in their initial diagnosis than all other groups. Otherwise the groups were similar. (4.3.4)

All groups showed significant improvement and had more relevant diagnosis in their final diagnosis than they did initially. (4.5.4) UK medical students had the greatest improvement and pharmacists had the least improvement.

In the final diagnosis, the proportion of relevant diagnoses had increased to about 60%. (4.4.2) At this stage, both UK medical students and physician associates had a greater proportion of relevant diagnoses than both international medical students and pharmacists. (4.4.4)

1.2.5 Post-consultation follow up

Management plan: Overall, less than half of students selected the correct management plan “Review in GP”. (5.1.2)

Pharmacists were the least likely to select the correct management plan, physician associates were the most likely to do so. (5.1.4)

Essential investigations: Overall, the proportion of essential investigations selected was just under 40%. (5.2.2) Around 82% of students selected at least one essential investigation in their follow up plan. (5.2.5)

International medical students included the greatest proportion of essential investigations in their follow up plan, physician associates had the lowest. (5.2.4)

Inspection of each of the investigations revealed notable between group differences. (5.2.5 and 5.2.7) Both international medical students and pharmacists were more likely to select “electrocardiogram” than UK medical students and physician associates. Both international and UK medical students were more likely to select “Relevant cultures or serological tests” than physician assocaites. All groups were more likely to select “prescription of appropriate medication” than pharmacists.

1.2.6 Feedback

Overall feedback was positive across all student groups. (6.1.2) Pharmacists were least likely group to agree that the level of difficulty was appropriate. (6.1.4) Pharmacists and international medical students also had lower agreement that eCREST was easy to navigate (5.1.3) and that it should be used to supplement traditional teaching. (6.1.5)

2 Case 3: Student characteristics

2.1 Student institutions

2.1.1

Tables showing the number of students from each institution and then showing the number of students grouped by student type: Medical students (international or UK), Pharmacists, and Physician Associates)

2.1.2 Student institutions

2.1.3 Institutions grouped by student type

df$Type1 n percent
MedStudents_International 256 21.9%
MedStudents_UK 492 42.1%
Newcastle 89 7.6%
Pharmacists 167 14.3%
PhysicianAssociates 164 14.0%
Total 1168 100.0%

2.2 Case 3 Completers

2.2.1

Tables showing the number of students who completed each case and the number of students of each student type who completed case 3 (Taru Ghandi).

2.2.2 All Case 3 completers

PatientID n percent
1 637 29.9%
2 383 18.0%
3 657 30.9%
4 437 20.5%
5 13 0.6%
Total 2127 100.0%

2.2.3 All Case 3 completers grouped by student type and institutions

label levels MedStudents_International MedStudents_UK Newcastle Pharmacists PhysicianAssociates Total
Total N (%) 219 (33.3) 159 (24.2) 76 (11.6) 149 (22.7) 54 (8.2) 657
Class Barcelona 5 (2.3) 5 (0.8)
KUSOM1 17 (7.8) 17 (2.6)
KUSOM2 29 (13.2) 29 (4.4)
UNIC MBBS 2 (0.9) 2 (0.3)
UNIC MD 145 (66.2) 145 (22.1)
UNIC MSc 21 (9.6) 21 (3.2)
Barts 3 (1.9) 3 (0.5)
Barts 2020 10 (6.3) 10 (1.5)
Barts2 38 (23.9) 38 (5.8)
UCL 2020 8 (5.0) 8 (1.2)
UCL3 52 (32.7) 52 (7.9)
UEA 2020 19 (11.9) 19 (2.9)
UEA1 22 (13.8) 22 (3.3)
UEA2 7 (4.4) 7 (1.1)
Newcastle 76 (100.0) 76 (11.6)
UCL 149 (100.0) 149 (22.7)
QMUL 7 (13.0) 7 (1.1)
St Georges 40 (74.1) 40 (6.1)
Swansea 4 (7.4) 4 (0.6)
UEA PA 3 (5.6) 3 (0.5)

2.3 Student Age

2.3.1

Descriptive statistics for student age overall and then according to student type, the omnibus anova test for differences in age by student type, and all pairwise comparison follow-up tests.

2.3.2 Descriptive stats

  • The descriptive statistics for student age overall.
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 652 26.55 6.33 24 25.34 2.97 19 54 35 1.85 3.35 0.25
  • Then showing the descriptive statistics for student age according to student type.
    group1 n mean sd median trimmed mad min max range skew kurtosis se
    MedStudents_International 219 25.95 5.45 24 24.79 1.48 21 54 33 2.41 6.43 0.37
    MedStudents_UK 156 23.69 3.57 23 23.25 1.48 19 42 23 2.55 10.05 0.29
    Newcastle 76 22.20 2.03 22 21.92 0.00 21 38 17 6.32 45.47 0.23
    Pharmacists 148 32.45 6.74 30 31.57 4.45 20 53 33 1.14 0.56 0.55
    PhysicianAssociates 53 27.17 7.18 25 25.77 2.97 19 48 29 1.80 2.35 0.99

2.3.3 Omnibus test (ANOVA)

  • The top table shows the outcome measures of the ANOVA with the bottom table showing the corresponding effect sizes.
    Sum Sq Df F value Pr(>F)
    (Intercept) 147420.66 1 5263.06 0
    Type1 7970.87 4 71.14 0
    Residuals 18122.75 647 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 7971 1992.72 71.14 0 0.3 0.3 0.3 0.3 0.3 0.66 1
…2 647 18123 28.01 NA NA NA NA NA NA NA NA NA

2.3.4 Follow up tests (MDEs)

  • The table showing all the pairwise comparisons with mean difference estimates (95%CIs). Raw and holm corrected p values are provided.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 25.95 MedStudents_UK 23.69 2.25 1.34 3.17 0.000 0.000 ****
MedStudents_International 25.95 Newcastle 22.20 3.75 2.89 4.61 0.000 0.000 ****
MedStudents_International 25.95 Pharmacists 32.45 -6.51 -7.82 -5.20 0.000 0.000 ****
MedStudents_International 25.95 PhysicianAssociates 27.17 -1.22 -3.33 0.88 0.249 0.249 ns
MedStudents_UK 23.69 Newcastle 22.20 1.49 0.77 2.22 0.000 0.000 ****
MedStudents_UK 23.69 Pharmacists 32.45 -8.76 -9.99 -7.53 0.000 0.000 ****
MedStudents_UK 23.69 PhysicianAssociates 27.17 -3.48 -5.53 -1.42 0.001 0.002 **
Newcastle 22.20 Pharmacists 32.45 -10.26 -11.44 -9.07 0.000 0.000 ****
Newcastle 22.20 PhysicianAssociates 27.17 -4.97 -7.00 -2.94 0.000 0.000 ****
Pharmacists 32.45 PhysicianAssociates 27.17 5.28 3.03 7.53 0.000 0.000 ****
  • The figure below plots student age by student type. Points represent individual data points, the point estimate represents the mean estimate with error bars, bar chart overlaid.

2.4 Student Gender

2.4.1

Tables for students’ self-reported gender showing the number and (%) overall and then according to student type, the omnibus tests for association between gender and student type, and all pairwise follow-up tests of associations.

2.4.2 Descriptive tables

Overall and according to student type
label levels Female Male Total
Total N (%) 400 (60.9) 257 (39.1) 657
Type1 MedStudents_International 113 (51.6) 106 (48.4) 219 (100)
MedStudents_UK 84 (52.8) 75 (47.2) 159 (100)
Newcastle 56 (73.7) 20 (26.3) 76 (100)
Pharmacists 104 (69.8) 45 (30.2) 149 (100)
PhysicianAssociates 43 (79.6) 11 (20.4) 54 (100)

2.4.3 Omnibus tests (Chi2 & Fisher’s Exact Test)

  • We conducted chi-squared test with Yates continuity correction and Fishers exact test to assess whether student self-reported gender was associated with student type. Both tests indicate a significant association between self-reported gender and student type.
X-squared df p value Fisher p value
Test of Gender by Student type 30.43 4 0 0

2.4.4 Follow up tests (Fisher’s Exact Test)

  • The table showing all the pairwise comparisons with Odds Ratios (95%CIs). Raw p values are provided.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 0.95 0.62 1.46 0.835
Int Medical students vs. Pharmacists 0.38 0.20 0.70 0.001
Int medical students vs. Physician associates 0.46 0.29 0.73 0.001
UK medical students vs. Pharmacists 0.40 0.21 0.75 0.003
UK medical students vs. Physician associates 0.49 0.29 0.79 0.002
Pharmacists vs. Physician associates 1.21 0.63 2.39 0.641
  • The figure below plots student gender by student type. Columns represent %s with total ns provided.

  • Alternative figure with truncated axis to emphasize differences.

3 Case 3: Patient investigations

3.1 Essential Questions only [EQs]

3.1.1

Twenty of the thirty-two available questions were identified by experts during development as essential and are highlighted in bold font below.

Category Question Question
Essential Does anything make your pain worse? Have you ever coughed up any blood?
Essential Does anything make the pain better? Have you had a fever?
Essential Does the pain come and go? Are you more breathless than normal?
Essential How long has the pain been there? Any recent immobility?
Essential Where do you feel the pain? Do you have pets?
Essential Does the pain travel anywhere else? Do you smoke?
Essential I can see you also have a cough. Can you tell me more about it? How many cigarettes do you smoke per day?
Essential Have you noticed any other symptoms? How long have you been a smoker?
Essential When did the cough start? Do you have any ideas about what is causing this?
Essential Do you produce any phlegm? How is this affecting you?
Relevant Does your position alter the pain? Have you started any new medications recently?
Relevant Have you had any injury to your chest? Do you drink alcohol?
Relevant What do you do for a living? Have you lost or gained any weight?
Other Can you identify anything that makes the cough worse? What do you do with your spare time?
Other Does anything make the cough better? Do you take medication?
Other Have you noticed a change in your mood at all? Has your appetite changed?

Tables 3.1.2 to 3.1.4: Descriptive statistics for the number and % of EQs asked overall and then according to student type, the omnibus anova test for differences in the % of EQs asked by student type, and all pairwise comparison follow-up tests.

Tables 3.1.5 to 3.1.8: Tables for showing the number and (%) of students who asked each EQ overall and then according to student type, the omnibus tests for association between % of students who asked each EQ and student type, all pairwise follow-up tests of associations, and all figures.

3.1.2 Descriptive stats

  • Descriptive statistics for the total number of essential questions asked overall
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 13.52 4.75 14 13.77 5.93 0 20 20 -0.33 -0.82 0.19
  • Descriptive statistics for the proportion of essential questions asked overall
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 67.61 23.73 70 68.84 29.65 0 100 100 -0.33 -0.82 0.93
  • Descriptive statistics for the proportion of essential questions asked by student group
group1 n mean sd median trimmed mad min max range skew kurtosis se
MedStudents_International 219 65.87 23.59 65.0 66.69 29.65 0 100 100 -0.23 -0.83 1.59
MedStudents_UK 159 70.13 23.17 75.0 72.02 22.24 0 100 100 -0.67 -0.20 1.84
Newcastle 76 56.51 25.09 45.0 55.32 22.24 15 100 85 0.54 -1.00 2.88
Pharmacists 149 71.11 23.62 75.0 72.85 29.65 20 100 80 -0.42 -0.99 1.93
PhysicianAssociates 54 73.24 18.89 72.5 74.55 18.53 10 100 90 -0.78 1.14 2.57

3.1.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 950139.84 1 1746.36 0
Type1 14564.47 4 6.69 0
Residuals 354733.78 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 14564 3641.12 6.69 0 0.04 0.04 0.03 0.03 0.03 0.2 0.99
…2 652 354734 544.07 NA NA NA NA NA NA NA NA NA

3.1.4 Follow up tests (MDEs)

  • The table shows all pairwise comparisons of the proportion of essential questions asked by student type.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 65.87 MedStudents_UK 70.13 -4.26 -9.04 0.53 0.081 0.324 ns
MedStudents_International 65.87 Newcastle 56.51 9.35 2.84 15.87 0.005 0.035 **
MedStudents_International 65.87 Pharmacists 71.11 -5.24 -10.17 -0.31 0.037 0.185
MedStudents_International 65.87 PhysicianAssociates 73.24 -7.37 -13.37 -1.37 0.017 0.102
MedStudents_UK 70.13 Newcastle 56.51 13.61 6.86 20.36 0.000 0.001 ***
MedStudents_UK 70.13 Pharmacists 71.11 -0.98 -6.23 4.27 0.713 1.000 ns
MedStudents_UK 70.13 PhysicianAssociates 73.24 -3.11 -9.38 3.15 0.326 0.978 ns
Newcastle 56.51 Pharmacists 71.11 -14.59 -21.45 -7.74 0.000 0.000 ****
Newcastle 56.51 PhysicianAssociates 73.24 -16.73 -24.36 -9.09 0.000 0.000 ****
Pharmacists 71.11 PhysicianAssociates 73.24 -2.13 -8.50 4.24 0.509 1.000 ns

3.1.5 Descriptive tables

  • Table showing the proportion of students who asked each essential question, with comparisons described in the final column

3.1.6 Omnibus tests (Chi2 & Fisher’s Exact Test)

  • First we ran chi-square and fisher omnibus tests for all questions. Rows for which at least one test result was statistically significant are highlighted in bold font.
X-squared df p value Fisher p value
Does anything make your pain worse? 6.98 4 0.137 0.126
Does anything make the pain better? 8.49 4 0.075 0.083
Does the pain come and go? 5.20 4 0.267 0.267
How long has the pain been there? 13.47 4 0.009 0.016
Where do you feel the pain? 35.87 4 0.000 0.000
Does the pain travel anywhere else? 10.49 4 0.033 0.029
I can see you also have a cough. Can you tell me more about it? 23.02 4 0.000 0.000
Have you noticed any other symptoms? 8.64 4 0.071 0.068
When did the cough start? 12.54 4 0.014 0.020
Do you produce any phlegm? 11.01 4 0.026 0.022
Have you ever coughed up any blood? 35.54 4 0.000 0.000
Have you had a fever? 14.72 4 0.005 0.007
Are you more breathless than normal? 27.46 4 0.000 0.000
Any recent immobility? 13.83 4 0.008 0.008
Do you have pets? 3.89 4 0.421 0.427
Do you smoke? 6.13 4 0.189 0.184
How many cigarettes do you smoke per day? 8.23 4 0.083 0.084
How long have you been a smoker? 12.48 4 0.014 0.015
Do you have any ideas about what is causing this? 24.58 4 0.000 0.000
How is this affecting you? 17.02 4 0.002 0.002

3.1.7 Follow up tests (Fisher’s Exact Test)

We then conducted follow up tests using Fishers exact test for all pairwise comparisons by student type, which showed:

  • Pharmacists were more likely to ask “Does the pain travel anywhere else?” than International and UK medical students
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 0.89 0.55 1.44 0.641
Int Medical students vs. Pharmacists 0.88 0.48 1.67 0.765
Int medical students vs. Physician associates 1.87 1.08 3.34 0.022
UK medical students vs. Pharmacists 1.00 0.53 1.93 1.000
UK medical students vs. Physician associates 2.12 1.18 3.87 0.009
Pharmacists vs. Physician associates 2.11 1.03 4.32 0.035

  • Physician associates and UK medical students were more likely to ask “Have you ever coughed up any blood?” than International medical students.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 2.05 1.08 4.09 0.020
Int Medical students vs. Pharmacists 0.40 0.21 0.74 0.003
Int medical students vs. Physician associates 1.40 0.77 2.63 0.260
UK medical students vs. Pharmacists 0.19 0.09 0.41 0.000
UK medical students vs. Physician associates 0.68 0.32 1.44 0.297
Pharmacists vs. Physician associates 3.53 1.75 7.24 0.000

  • UK medical students and physician associates were more likely to ask “Are you more breathless than normal?” than International medical students.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 1.85 1.11 3.15 0.016
Int Medical students vs. Pharmacists 0.51 0.29 0.90 0.017
Int medical students vs. Physician associates 1.46 0.88 2.44 0.152
UK medical students vs. Pharmacists 0.27 0.14 0.52 0.000
UK medical students vs. Physician associates 0.79 0.44 1.42 0.403
Pharmacists vs. Physician associates 2.87 1.53 5.44 0.001

  • All groups were more likely to ask “Do you have any ideas about what is causing this?” than International medical students. Pharmacists were also more likely to than UK medical students.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 1.73 1.10 2.76 0.016
Int Medical students vs. Pharmacists 1.32 0.74 2.37 0.342
Int medical students vs. Physician associates 3.23 1.92 5.57 0.000
UK medical students vs. Pharmacists 0.76 0.41 1.43 0.366
UK medical students vs. Physician associates 1.86 1.05 3.37 0.030
Pharmacists vs. Physician associates 2.45 1.24 4.87 0.007

  • Pharmacists and physician associates were more likely to ask “How is this affecting you?” than International medical students. Pharmacists were also more likely to than UK medical students.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 1.26 0.82 1.93 0.296
Int Medical students vs. Pharmacists 1.45 0.83 2.53 0.182
Int medical students vs. Physician associates 2.23 1.42 3.50 0.000
UK medical students vs. Pharmacists 1.15 0.64 2.07 0.676
UK medical students vs. Physician associates 1.77 1.10 2.87 0.016
Pharmacists vs. Physician associates 1.54 0.84 2.79 0.152

3.1.8 All figures

  • Figures showing the proportion of students who asked each essential question

3.2 Physical exams only [PEs]

3.2.1

Six of the eleven available physical examinations were identified by experts during development as essential and are higlighted in bold font below.

Category Physical Examination Physical Examination
Essential Cardiovascular Lower limb and foot
Essential Respiratory Lymphatic System
Essential Musculoskeletal chest examination ENT
Other Abdomen Neurological (central and cranial nerves)
Other Eyes Neurological (peripheral)
Other Genital examination

Tables 3.2.2 to 3.1.4: Descriptive statistics for the number and % of PEs selected overall and then according to student type, the omnibus anova test for differences in the % of PEs asked by student type, and all pairwise comparison follow-up tests.

Tables 3.2.5 to 3.2.8: Tables for showing the number and (%) of students who selected each PEs overall and then according to student type, the omnibus tests for association between % of students who selected each PEs and student type, all pairwise follow-up tests of associations, and all figures.

3.2.2 Descriptive stats

  • Descriptive statistics for the total number of physical exams selected overall
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 657 3.35 1.2 3 3.38 1.48 0 6 6 -0.38 0.47 0.05
  • Descriptive statistics for the proportion of physical exams selected overall
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 657 55.81 19.98 50 56.36 24.71 0 100 100 -0.38 0.47 0.78
  • Descriptive statistics for the proportion of physical exams selected by student group
    group1 n mean sd median trimmed mad min max range skew kurtosis se
    MedStudents_International 219 55.02 19.88 50.00 55.65 24.71 0 100 100 -0.42 0.30 1.34
    MedStudents_UK 159 54.72 18.58 50.00 55.43 24.71 0 100 100 -0.60 1.35 1.47
    Newcastle 76 56.80 18.49 58.33 57.26 12.35 0 100 100 -0.38 0.15 2.12
    Pharmacists 149 57.38 21.95 50.00 57.85 24.71 0 100 100 -0.30 0.11 1.80
    PhysicianAssociates 54 56.48 21.09 50.00 56.82 24.71 0 100 100 -0.14 -0.21 2.87

3.2.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 663025.11 1 1655.01 0.00
Type1 792.69 4 0.49 0.74
Residuals 261202.07 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 793 198.17 0.5 0.74 0 0 0 0 0 0.06 0.17
…2 652 261202 400.62 NA NA NA NA NA NA NA NA NA

3.2.4 Follow up tests (MDEs)

  • The table shows all pairwise comparisons of the proportion of physical exams selected by student type.
group1 estimate1 group2 estimate2 estimate conf.low conf.high p p.adj p.adj.signif
MedStudents_International 55.02 MedStudents_UK 54.72 0.31 -3.61 4.23 0.878 1 ns
MedStudents_International 55.02 Newcastle 56.80 -1.78 -6.74 3.19 0.481 1 ns
MedStudents_International 55.02 Pharmacists 57.38 -2.36 -6.78 2.06 0.294 1 ns
MedStudents_International 55.02 PhysicianAssociates 56.48 -1.46 -7.77 4.85 0.647 1 ns
MedStudents_UK 54.72 Newcastle 56.80 -2.08 -7.19 3.02 0.422 1 ns
MedStudents_UK 54.72 Pharmacists 57.38 -2.67 -7.24 1.91 0.252 1 ns
MedStudents_UK 54.72 PhysicianAssociates 56.48 -1.76 -8.18 4.65 0.586 1 ns
Newcastle 56.80 Pharmacists 57.38 -0.58 -6.07 4.90 0.834 1 ns
Newcastle 56.80 PhysicianAssociates 56.48 0.32 -6.76 7.39 0.929 1 ns
Pharmacists 57.38 PhysicianAssociates 56.48 0.90 -5.82 7.62 0.791 1 ns

3.2.5 Descriptive tables

  • Table showing the proportion of students who selected each essential physical exam

3.2.6 Omnibus tests (Chi2 & Fisher’s Exact Test)

  • First we ran chi-square and fisher omnibus tests for all physical exams.
X-squared df p value Fisher p value
Cardiovascular system 18.60 4 0.001 0.003
ENT 5.65 4 0.227 0.213
Lower Limb and foot 25.02 4 0.000 0.000
Lymphatic System 11.59 4 0.021 0.022
Musculoskeletal chest examination 6.57 4 0.161 0.163
Respiratory 3.49 4 0.479 0.504

3.2.7 Follow up tests (Fisher’s Exact Test)

We conducted follow up tests using Fishers exact test for all pairwise comparisons by student type, which showed:

  • Pharmacists were more likely to select “Lower Limb and foot” than all other students. International medical students were more likely to select this than UK medical students.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 0.47 0.26 0.86 0.011
Int Medical students vs. Pharmacists 0.81 0.39 1.60 0.632
Int medical students vs. Physician associates 1.82 1.12 2.95 0.013
UK medical students vs. Pharmacists 1.70 0.76 3.77 0.172
UK medical students vs. Physician associates 3.82 2.09 7.20 0.000
Pharmacists vs. Physician associates 2.24 1.12 4.66 0.015

3.2.8 All figures

  • Figures showing the proportion of students who asked each essential question

3.3 Bedside tests only [BTs]

3.3.1

Three of the nine available bedside tests were identified by experts during development as essential and are listed below.

Category Test Test Test
Essential Blood pressure Temperature Oxygen saturation
Other Blood Glucose Peak Flow Weight
Other Height Urinalysis Patient Health Questionnaire (PHQ-9)

Tables 3.3.2 to 3.3.4: Descriptive statistics for the number and % of BTs selected overall and then according to student type, the omnibus anova test for differences in the % of BTs asked by student type, and all pairwise comparison follow-up tests.

Tables 3.3.5 to 3.3.8: Tables for showing the number and (%) of students who selected each BTs overall and then according to student type, the omnibus tests for association between % of students who selected each BTs and student type, all pairwise follow-up tests of associations, and all figures.

3.3.2 Descriptive stats

  • Descriptive statistics for total number of bedside tests selected overall
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 657 2.09 1 2 2.24 1.48 0 3 3 -0.84 -0.42 0.04
  • Descriptive statistics for proportion of bedside tests selected overall
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 657 69.81 33.29 66.67 74.7 49.42 0 100 100 -0.84 -0.42 1.3
  • Descriptive statistics for proportion of bedside tests selected by student group
    group1 n mean sd median trimmed mad min max range skew kurtosis se
    MedStudents_International 219 69.86 34.16 66.67 74.58 49.42 0 100 100 -0.88 -0.43 2.31
    MedStudents_UK 159 74.42 29.10 66.67 78.81 49.42 0 100 100 -1.03 0.37 2.31
    Newcastle 76 59.65 32.80 66.67 61.83 49.42 0 100 100 -0.41 -0.87 3.76
    Pharmacists 149 65.10 35.80 66.67 68.60 49.42 0 100 100 -0.56 -1.03 2.93
    PhysicianAssociates 54 83.33 28.78 100.00 90.15 0.00 0 100 100 -1.81 2.48 3.92

3.3.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 1068904.11 1 992.18 0
Type1 24411.34 4 5.66 0
Residuals 702421.06 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 24411 6102.84 5.66 0 0.03 0.03 0.03 0.03 0.03 0.19 0.98
…2 652 702421 1077.33 NA NA NA NA NA NA NA NA NA

3.3.4 Follow up tests (MDE)

  • The table shows all pairwise comparisons of the proportion of essential bedside tests selected by student type.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 69.86 MedStudents_UK 74.42 -4.56 -10.98 1.86 0.163 0.489 ns
MedStudents_International 69.86 Newcastle 59.65 10.21 1.48 18.94 0.022 0.110
MedStudents_International 69.86 Pharmacists 65.10 4.76 -2.58 12.11 0.203 0.489 ns
MedStudents_International 69.86 PhysicianAssociates 83.33 -13.47 -22.50 -4.44 0.004 0.028 **
MedStudents_UK 74.42 Newcastle 59.65 14.77 6.04 23.50 0.001 0.008 **
MedStudents_UK 74.42 Pharmacists 65.10 9.32 1.98 16.67 0.013 0.078
MedStudents_UK 74.42 PhysicianAssociates 83.33 -8.91 -17.94 0.12 0.053 0.212 ns
Newcastle 59.65 Pharmacists 65.10 -5.45 -14.87 3.97 0.255 0.489 ns
Newcastle 59.65 PhysicianAssociates 83.33 -23.68 -34.43 -12.93 0.000 0.000 ****
Pharmacists 65.10 PhysicianAssociates 83.33 -18.23 -27.92 -8.54 0.000 0.003 ***

3.3.5 Descriptive tables

  • Table showing the proportion of students who selected each bedside test

3.3.6 Omnibus tests (Chi2 & Fisher’s Exact Test)

  • First we ran chi-square and fisher omnibus tests for all bedside tests.
X-squared df p value Fisher p value
Blood pressure 6.16 4 0.188 0.184
Temperature 21.45 4 0.000 0.000
Oxygen saturation 28.71 4 0.000 0.000

3.3.7 Follow up tests (Fisher’s Exact Test)

We conducted follow up tests using Fishers exact test for all pairwise comparisons by student type, which showed:

  • Physician associates were more likely to select “Temperature test” than International medical students and pharmacists. UK medical students were more likely to select this test than pharmacists.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 1.45 0.91 2.34 0.115
Int Medical students vs. Pharmacists 0.64 0.37 1.14 0.129
Int medical students vs. Physician associates 0.71 0.45 1.12 0.126
UK medical students vs. Pharmacists 0.45 0.24 0.82 0.007
UK medical students vs. Physician associates 0.49 0.29 0.81 0.004
Pharmacists vs. Physician associates 1.10 0.61 2.00 0.777

  • All student types were more likely to select “Oxygen saturation test” than pharmacists
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 1.98 1.05 3.87 0.031
Int Medical students vs. Pharmacists 0.46 0.25 0.86 0.011
Int medical students vs. Physician associates 0.57 0.34 0.95 0.024
UK medical students vs. Pharmacists 0.23 0.11 0.49 0.000
UK medical students vs. Physician associates 0.29 0.15 0.55 0.000
Pharmacists vs. Physician associates 1.24 0.65 2.33 0.543

  • No differences were found for blood pressure so no follow up tests were conducted. Though it appears physician associates may have been more likely to select this test.

3.4 EQs, PEs, and BTs combined

3.4.1

Descriptive statistics for the number and % of combined EQs, PEs, and BTs asked/selected overall and then according to student type, the omnibus anova test for differences in the % of combined EQs, PEs, and BTs asked/selected student type, and all pairwise comparison follow-up tests.

3.4.2 Descriptive stats

  • Descriptive statistics for the proportion of Essential questions, physical exams, and bedside tests selected overall
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 657 65.4 18.55 68.97 66.46 20.45 0 97 97 -0.54 -0.13 0.72
  • Descriptive statistics for the proportion of Essential questions, physical exams, and bedside tests selected by student group
    group1 n mean sd median trimmed mad min max range skew kurtosis se
    MedStudents_International 219 64.04 19.21 65.52 64.91 20.45 0.00 96.55 96.55 -0.48 -0.15 1.30
    MedStudents_UK 159 67.38 18.37 68.97 69.15 20.45 0.00 96.55 96.55 -1.00 1.15 1.46
    Newcastle 76 56.90 18.29 53.45 55.95 23.01 24.14 93.10 68.97 0.45 -0.90 2.10
    Pharmacists 149 67.65 17.68 68.97 68.97 20.45 20.69 96.55 75.86 -0.59 -0.45 1.45
    PhysicianAssociates 54 70.82 14.91 72.41 71.55 15.34 27.59 96.55 68.97 -0.50 0.27 2.03

3.4.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 898066.0 1 2699.96 0
Type1 8863.7 4 6.66 0
Residuals 216869.9 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 8864 2215.93 6.66 0 0.04 0.04 0.03 0.03 0.03 0.2 0.99
…2 652 216870 332.62 NA NA NA NA NA NA NA NA NA

3.4.4 Follow up tests (MDEs)

  • The table shows all pairwise comparisons of the proportion of essential questions, physical exams, and bedside tests by student type.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 64.04 MedStudents_UK 67.38 -3.35 -7.18 0.49 0.087 0.348 ns
MedStudents_International 64.04 Newcastle 56.90 7.14 2.26 12.02 0.004 0.028 **
MedStudents_International 64.04 Pharmacists 67.65 -3.61 -7.43 0.22 0.064 0.320 ns
MedStudents_International 64.04 PhysicianAssociates 70.82 -6.78 -11.56 -2.00 0.006 0.036 **
MedStudents_UK 67.38 Newcastle 56.90 10.49 5.44 15.53 0.000 0.001 ****
MedStudents_UK 67.38 Pharmacists 67.65 -0.26 -4.31 3.78 0.898 0.898 ns
MedStudents_UK 67.38 PhysicianAssociates 70.82 -3.44 -8.39 1.52 0.172 0.516 ns
Newcastle 56.90 Pharmacists 67.65 -10.75 -15.79 -5.71 0.000 0.000 ****
Newcastle 56.90 PhysicianAssociates 70.82 -13.92 -19.70 -8.14 0.000 0.000 ****
Pharmacists 67.65 PhysicianAssociates 70.82 -3.17 -8.11 1.77 0.206 0.516 ns

3.5 Relevant Questions only [RQs]

3.5.1

Six of the thirty two available questions were identified by experts during development as relevant and are highlighted in bold font below.

Category Question Question
Essential Does anything make your pain worse? Have you ever coughed up any blood?
Essential Does anything make the pain better? Have you had a fever?
Essential Does the pain come and go? Are you more breathless than normal?
Essential How long has the pain been there? Any recent immobility?
Essential Where do you feel the pain? Do you have pets?
Essential Does the pain travel anywhere else? Do you smoke?
Essential I can see you also have a cough. Can you tell me more about it? How many cigarettes do you smoke per day?
Essential Have you noticed any other symptoms? How long have you been a smoker?
Essential When did the cough start? Do you have any ideas about what is causing this?
Essential Do you produce any phlegm? How is this affecting you?
Relevant Does your position alter the pain? Have you started any new medications recently?
Relevant Have you had any injury to your chest? Do you drink alcohol?
Relevant What do you do for a living? Have you lost or gained any weight?
Other Can you identify anything that makes the cough worse? What do you do with your spare time?
Other Does anything make the cough better? Do you take medication?
Other Have you noticed a change in your mood at all? Has your appetite changed?

Descriptive statistics for the number and % of RQs asked overall and then according to student type, the omnibus anova test for differences in the % of RQs asked by student type, and all pairwise comparison follow-up tests.

3.5.2 Descriptive stats

  • Descriptive statistics for the total number of relevant questions asked overall
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 3.02 1.83 3 3.01 1.48 0 6 6 0.13 -1.01 0.07
  • Descriptive statistics for the proportion of relevant questions asked overall
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 50.41 30.57 50 50.22 24.71 0 100 100 0.13 -1.01 1.19
  • Descriptive statistics for the proportion of relevant questions asked by student group
group1 n mean sd median trimmed mad min max range skew kurtosis se
MedStudents_International 219 48.78 29.88 50.00 48.31 24.71 0 100 100 0.19 -0.95 2.02
MedStudents_UK 159 49.48 30.26 50.00 49.35 24.71 0 100 100 0.07 -1.01 2.40
Newcastle 76 47.15 30.72 41.67 45.70 37.06 0 100 100 0.44 -1.02 3.52
Pharmacists 149 54.25 32.43 50.00 55.23 24.71 0 100 100 0.00 -1.15 2.66
PhysicianAssociates 54 53.70 28.53 50.00 54.17 24.71 0 100 100 -0.07 -0.86 3.88

3.5.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 521158.0 1 558.03 0.00
Type1 4310.6 4 1.15 0.33
Residuals 608914.5 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 4311 1077.65 1.15 0.33 0.01 0.01 0 0 0 0.08 0.37
…2 652 608914 933.92 NA NA NA NA NA NA NA NA NA

3.5.4 Follow up tests (MDEs)

  • The table shows all pairwise comparisons of the proportion of relevant questions asked by student type.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 48.78 MedStudents_UK 49.48 -0.69 -6.86 5.48 0.825 1 ns
MedStudents_International 48.78 Newcastle 47.15 1.63 -6.40 9.67 0.688 1 ns
MedStudents_International 48.78 Pharmacists 54.25 -5.47 -12.04 1.10 0.102 1 ns
MedStudents_International 48.78 PhysicianAssociates 53.70 -4.92 -13.62 3.78 0.264 1 ns
MedStudents_UK 49.48 Newcastle 47.15 2.33 -6.10 10.75 0.586 1 ns
MedStudents_UK 49.48 Pharmacists 54.25 -4.77 -11.82 2.27 0.183 1 ns
MedStudents_UK 49.48 PhysicianAssociates 53.70 -4.23 -13.29 4.83 0.357 1 ns
Newcastle 47.15 Pharmacists 54.25 -7.10 -15.82 1.61 0.110 1 ns
Newcastle 47.15 PhysicianAssociates 53.70 -6.55 -16.94 3.83 0.214 1 ns
Pharmacists 54.25 PhysicianAssociates 53.70 0.55 -8.78 9.87 0.908 1 ns

#Essential questions, physical exams, bedside tests, and relevant questions combined

3.6 EQs, PEs, BTs, and RQs combined

3.6.1

Descriptive statistics for the number and % of combined EQs, PEs, BTs, and RQs asked/selected overall and then according to student type, the omnibus anova test for differences in the % of combined EQs, PEs, BTs, and RQs asked/selected student type, and all pairwise comparison follow-up tests.

3.6.2 Descriptive stats

  • Descriptive statistics for the proportion of Essential questions, physical exams, bedside tests, and relevant questions selected overall
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 657 62.83 19.56 62.86 63.59 21.18 0 97 97 -0.33 -0.45 0.76
  • Descriptive statistics for the proportion of Essential questions, physical exams, bedside tests, and relevant questions selected by student group
    group1 n mean sd median trimmed mad min max range skew kurtosis se
    MedStudents_International 219 61.42 20.16 62.86 62.10 21.18 0.00 97.14 97.14 -0.30 -0.46 1.36
    MedStudents_UK 159 64.31 19.32 65.71 65.58 21.18 0.00 97.14 97.14 -0.65 0.30 1.53
    Newcastle 76 55.23 19.49 51.43 53.96 21.18 22.86 94.29 71.43 0.56 -0.82 2.24
    Pharmacists 149 65.35 19.01 65.71 66.45 21.18 17.14 97.14 80.00 -0.41 -0.64 1.56
    PhysicianAssociates 54 67.88 16.19 68.57 68.51 12.71 25.71 97.14 71.43 -0.34 0.07 2.20

3.6.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 826214.30 1 2213.25 0.000
Type1 7503.81 4 5.03 0.001
Residuals 243393.54 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 7504 1875.95 5.03 0.001 0.03 0.03 0.02 0.02 0.02 0.18 0.96
…2 652 243394 373.30 NA NA NA NA NA NA NA NA NA

3.6.4 Follow up tests (MDEs)

  • The table shows all pairwise comparisons of the proportion of Essential questions, physical exams, bedside tests, and relevant questions selected by student type.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 61.42 MedStudents_UK 64.31 -2.89 -6.92 1.14 0.159 0.636 ns
MedStudents_International 61.42 Newcastle 55.23 6.20 1.02 11.37 0.019 0.114
MedStudents_International 61.42 Pharmacists 65.35 -3.93 -8.00 0.14 0.059 0.295 ns
MedStudents_International 61.42 PhysicianAssociates 67.88 -6.46 -11.60 -1.32 0.014 0.098
MedStudents_UK 64.31 Newcastle 55.23 9.09 3.73 14.44 0.001 0.008 **
MedStudents_UK 64.31 Pharmacists 65.35 -1.04 -5.34 3.26 0.635 0.700 ns
MedStudents_UK 64.31 PhysicianAssociates 67.88 -3.57 -8.89 1.75 0.186 0.636 ns
Newcastle 55.23 Pharmacists 65.35 -10.12 -15.51 -4.74 0.000 0.003 ***
Newcastle 55.23 PhysicianAssociates 67.88 -12.66 -18.87 -6.45 0.000 0.001 ****
Pharmacists 65.35 PhysicianAssociates 67.88 -2.53 -7.88 2.81 0.350 0.700 ns

4 Case 3: Diagnoses

Six of the thirteen available diagnoses were identified by experts during development as relevant and are highlighted in bold font below. **Lower respiratory tract infection was considered the top diagnosis for students to consider.

Category Diagnosis Diagnosis
Relevant Lower respiratory tract infection** COPD
Relevant Pulmonary Embolism Lung cancer
Relevant Pulmonary Tuberculosis Costochondritis
Other Acute Coronary Syndrome Pneumothorax
Other Interstitial lung disease Psychogenic cough
Other Musculoskeletal injury Stable Angina
Other Pericarditis

4.1 Lung cancer

4.1.1

Tables for inclusion of lung cancer as a final diagnosis showing the number and (%) overall and then according to student type, the omnibus tests for association between lung cancer diagnosis and student type, and all pairwise follow-up tests of associations.

4.1.2 Descriptive tables

  • Proportion of students including a lung cancer diagnosis overall and according to student type.
label levels Lung cancer included Lung cancer not included Total
Total N (%) 476 (72.5) 181 (27.5) 657
Type1 MedStudents_International 154 (70.3) 65 (29.7) 219 (100)
MedStudents_UK 119 (74.8) 40 (25.2) 159 (100)
Newcastle 58 (76.3) 18 (23.7) 76 (100)
Pharmacists 101 (67.8) 48 (32.2) 149 (100)
PhysicianAssociates 44 (81.5) 10 (18.5) 54 (100)

4.1.3 Omnibus tests (Chi2 & Fisher’s Exact Test)

  • We conducted chi-squared test with yates continuity correction and Fishers exact test to assess whether proportion of students who considered lung cancer as a possible diagnosis was associated with student type. Both tests indicate no significant association between proportion of students who considered lung cancer as a possible diagnosis and student type.
X-squared df p value Fisher p value
Test of Lung cancer diagnosis by Student type 5.35 4 0.2528 0.2628

4.1.4 Follow up tests (Fisher’s Exact Test)

  • The table showing all the pairwise comparisons with Odds Ratios (95%CIs). Raw p values are provided.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 0.80 0.49 1.29 0.354
Int Medical students vs. Pharmacists 0.74 0.38 1.39 0.375
Int medical students vs. Physician associates 1.13 0.70 1.81 0.646
UK medical students vs. Pharmacists 0.92 0.46 1.82 0.872
UK medical students vs. Physician associates 1.41 0.84 2.40 0.207
Pharmacists vs. Physician associates 1.53 0.79 3.07 0.217

4.2 Lower respiratory tract infection

4.2.1

Tables for inclusion of LRTI as a final diagnosis showing the number and (%) overall and then according to student type, the omnibus tests for association between LRTI diagnosis and student type, and all pairwise follow-up tests of associations.

4.2.2 Descriptive tables

  • Proportion of students including a LRTI diagnosis overall and according to student type.
label levels LRTI included LRTI not included Total
Total N (%) 503 (76.6) 154 (23.4) 657
Type1 MedStudents_International 174 (79.5) 45 (20.5) 219 (100)
MedStudents_UK 133 (83.6) 26 (16.4) 159 (100)
Newcastle 61 (80.3) 15 (19.7) 76 (100)
Pharmacists 92 (61.7) 57 (38.3) 149 (100)
PhysicianAssociates 43 (79.6) 11 (20.4) 54 (100)

4.2.3 Omnibus tests (Chi2 & Fisher’s Exact Test)

  • We conducted chi-squared test with yates continuity correction and Fishers exact test to assess whether proportion of students who considered lung cancer as a possible diagnosis was associated with student type. Both tests indicate a significant association between proportion of students who considered LRTI as a possible diagnosis and student type.
X-squared df p value Fisher p value
Test of LRTI diagnosis by Student type 24.56 4 0.0001 0.0001

4.2.4 Follow up tests (Fisher’s Exact Test)

  • The table showing all the pairwise comparisons with Odds Ratios (95%CIs). Raw p values are provided.
Odds ratio CI lower CI upper raw p value
Int vs. UK medical students 0.76 0.42 1.33 0.351
Int Medical students vs. Pharmacists 0.95 0.46 1.89 1.000
Int medical students vs. Physician associates 2.39 1.46 3.92 0.000
UK medical students vs. Pharmacists 1.26 0.58 2.67 0.582
UK medical students vs. Physician associates 3.16 1.80 5.64 0.000
Pharmacists vs. Physician associates 2.51 1.26 5.22 0.006

4.3 Relevant initial diagnosis

4.3.1

Tables 4.3.2 to 4.3.4: Descriptive statistics for the number and % of relevant initial diagnoses selected overall and then according to student type, the omnibus anova test for differences in the % of relevant initial diagnoses selected by student type, and all pairwise comparison follow-up tests.

4.3.2 Descriptive stats

  • Descriptive statistics for the total number of relevant initial diagnoses
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 2.77 0.97 3 2.79 1.48 0 5 5 0 -0.32 0.04
  • Descriptive statistics for the proportion of relevant initial diagnoses overall
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 46.12 16.14 50 46.49 24.71 0 83 83 0 -0.32 0.63
  • Descriptive statistics for the proportion of relevant initial diagnoses by student group
group1 n mean sd median trimmed mad min max range skew kurtosis se
MedStudents_International 219 43.76 15.45 50.00 43.88 24.71 16.67 83.33 66.67 0.11 -0.39 1.04
MedStudents_UK 159 43.40 14.41 50.00 43.80 24.71 16.67 66.67 50.00 -0.03 -0.71 1.14
Newcastle 76 58.77 13.74 66.67 59.95 0.00 16.67 83.33 66.67 -0.79 0.88 1.58
Pharmacists 149 44.07 17.34 50.00 44.08 24.71 0.00 83.33 83.33 0.10 0.06 1.42
PhysicianAssociates 54 51.54 14.21 50.00 51.14 12.35 16.67 83.33 66.67 0.19 -0.10 1.93

4.3.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 419362.00 1 1773.67 0
Type1 16778.69 4 17.74 0
Residuals 154157.39 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 16779 4194.67 17.74 0 0.1 0.1 0.09 0.09 0.09 0.33 1
…2 652 154157 236.44 NA NA NA NA NA NA NA NA NA

4.3.4 Follow up tests (MDEs)

  • The table shows all pairwise comparisons of the proportion of relevant initial diagnoses by student type.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 43.76 MedStudents_UK 43.40 0.36 -2.68 3.41 0.815 1.000 ns
MedStudents_International 43.76 Newcastle 58.77 -15.01 -18.75 -11.28 0.000 0.000 ****
MedStudents_International 43.76 Pharmacists 44.07 -0.31 -3.78 3.16 0.860 1.000 ns
MedStudents_International 43.76 PhysicianAssociates 51.54 -7.78 -12.15 -3.42 0.001 0.004 ***
MedStudents_UK 43.40 Newcastle 58.77 -15.38 -19.22 -11.53 0.000 0.000 ****
MedStudents_UK 43.40 Pharmacists 44.07 -0.68 -4.26 2.91 0.711 1.000 ns
MedStudents_UK 43.40 PhysicianAssociates 51.54 -8.15 -12.61 -3.69 0.000 0.003 ***
Newcastle 58.77 Pharmacists 44.07 14.70 10.52 18.89 0.000 0.000 ****
Newcastle 58.77 PhysicianAssociates 51.54 7.23 2.29 12.17 0.005 0.020 **
Pharmacists 44.07 PhysicianAssociates 51.54 -7.47 -12.23 -2.72 0.002 0.010 **

4.4 Relevant final diagnosis

4.4.1

Tables 4.4.2 to 4.4.4: Descriptive statistics for the number and % of relevant final diagnoses selected overall and then according to student type, the omnibus anova test for differences in the % of relevant final diagnoses selected by student type, and all pairwise comparison follow-up tests.

4.4.2 Descriptive stats

  • Descriptive statistics for the total number of relevant final diagnoses
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 3.59 0.95 4 3.63 1.48 1 5 4 -0.32 -0.44 0.04
  • Descriptive statistics for the proportion of relevant final diagnoses overall
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 59.89 15.87 66.67 60.56 24.71 17 83 67 -0.32 -0.44 0.62
  • Descriptive statistics for the proportion of relevant final diagnoses by student group
group1 n mean sd median trimmed mad min max range skew kurtosis se
MedStudents_International 219 58.60 15.59 66.67 58.85 24.71 16.67 83.33 66.67 -0.25 -0.59 1.05
MedStudents_UK 159 62.89 15.45 66.67 64.21 24.71 16.67 83.33 66.67 -0.63 0.21 1.23
Newcastle 76 59.87 14.45 66.67 60.48 24.71 16.67 83.33 66.67 -0.34 -0.04 1.66
Pharmacists 149 56.49 16.86 50.00 56.47 24.71 16.67 83.33 66.67 -0.05 -0.71 1.38
PhysicianAssociates 54 65.74 14.63 66.67 66.67 24.71 33.33 83.33 50.00 -0.39 -0.72 1.99

4.4.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 752029.43 1 3069.02 0
Type1 5371.84 4 5.48 0
Residuals 159765.15 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 5372 1342.96 5.48 0 0.03 0.03 0.03 0.03 0.03 0.18 0.98
…2 652 159765 245.04 NA NA NA NA NA NA NA NA NA

4.4.4 Follow up tests (MDEs)

  • The table shows all pairwise comparisons of the proportion of relevant final diagnoses by student type.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 58.60 MedStudents_UK 62.89 -4.29 -7.47 -1.11 0.008 0.056 **
MedStudents_International 58.60 Newcastle 59.87 -1.27 -5.15 2.61 0.519 0.675 ns
MedStudents_International 58.60 Pharmacists 56.49 2.11 -1.31 5.53 0.225 0.675 ns
MedStudents_International 58.60 PhysicianAssociates 65.74 -7.14 -11.62 -2.66 0.002 0.016 **
MedStudents_UK 62.89 Newcastle 59.87 3.02 -1.05 7.10 0.144 0.595 ns
MedStudents_UK 62.89 Pharmacists 56.49 6.41 2.77 10.04 0.001 0.005 ***
MedStudents_UK 62.89 PhysicianAssociates 65.74 -2.85 -7.49 1.79 0.226 0.675 ns
Newcastle 59.87 Pharmacists 56.49 3.38 -0.88 7.64 0.119 0.595 ns
Newcastle 59.87 PhysicianAssociates 65.74 -5.87 -11.00 -0.74 0.025 0.150
Pharmacists 56.49 PhysicianAssociates 65.74 -9.25 -14.06 -4.45 0.000 0.002 ***

4.5 Change in relevant diagnosis

4.5.1

Tables 4.5.2 to 4.5.4: Descriptive statistics for the number and % for the change in relevant diagnoses selected by the students (from initial to final) overall and then according to student type, the omnibus anova test for differences in the % change in relevant diagnosis by student type, and all pairwise comparison follow-up tests.

Tables 3.3.5 to 3.3.8: Tables for showing the number and (%) of students with changes in relevant diagnosis overall and then according to student type, the omnibus tests for association between % of students who had more relevant diagnoses and student type, all pairwise follow-up tests of associations, and all figures.

4.5.2 Descriptive stats

  • Descriptive statistics for the total change in relevant diagnoses (positive numbers indicate more relevant diagnoses were included in the final diagnosis than in the initial diagnosis)
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 0.83 1.02 1 0.77 1.48 -2 4 6 0.48 -0.01 0.04
  • Descriptive statistics for the percentage change in relevant diagnoses
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 657 13.77 16.92 16.67 12.78 24.71 -33 67 100 0.48 -0.01 0.66
  • Descriptive statistics for the percentage change in relevant diagnoses by student group
group1 n mean sd median trimmed mad min max range skew kurtosis se
MedStudents_International 219 14.84 16.49 16.67 13.75 24.71 -16.67 66.67 83.33 0.53 -0.04 1.11
MedStudents_UK 159 19.50 16.58 16.67 18.73 24.71 -16.67 66.67 83.33 0.23 -0.22 1.32
Newcastle 76 1.10 12.57 0.00 0.81 0.00 -33.33 50.00 83.33 0.63 2.39 1.44
Pharmacists 149 12.42 16.57 16.67 11.02 24.71 -33.33 66.67 100.00 0.64 0.40 1.36
PhysicianAssociates 54 14.20 16.64 16.67 13.26 24.71 -16.67 50.00 66.67 0.40 -0.77 2.26

4.5.3 Omnibus test (ANOVA)

Sum Sq Df F value Pr(>F)
(Intercept) 48230.59 1 185.11 0
Type1 17955.45 4 17.23 0
Residuals 169883.21 652 NA NA
df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
Type1 4 17955 4488.86 17.23 0 0.1 0.1 0.09 0.09 0.09 0.32 1
…2 652 169883 260.56 NA NA NA NA NA NA NA NA NA

4.5.4 Follow up tests (MDEs)

  • The table below shows the percentage change in relevant diagnoses from initial to final within each student group.
Mean change SD CI lower CI higher df t raw p value
Int Medical Student 14.84 16.49 12.64 17.04 218 13.32 0
UK Medical students 19.50 16.58 16.90 22.09 158 14.83 0
Pharmacists 12.42 16.57 9.73 15.10 148 9.15 0
Physician Associates 14.20 16.64 9.66 18.74 53 6.27 0

  • The table shows all pairwise comparisons of the percentage change in relevant diagnoses by student type.
Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
MedStudents_International 14.84 MedStudents_UK 19.50 -4.66 -8.05 -1.27 0.007 0.035 **
MedStudents_International 14.84 Newcastle 1.10 13.74 10.15 17.34 0.000 0.000 ****
MedStudents_International 14.84 Pharmacists 12.42 2.42 -1.03 5.88 0.168 0.504 ns
MedStudents_International 14.84 PhysicianAssociates 14.20 0.64 -4.38 5.66 0.800 1.000 ns
MedStudents_UK 19.50 Newcastle 1.10 18.40 14.55 22.25 0.000 0.000 ****
MedStudents_UK 19.50 Pharmacists 12.42 7.08 3.36 10.80 0.000 0.001 ***
MedStudents_UK 19.50 PhysicianAssociates 14.20 5.30 0.10 10.50 0.046 0.184
Newcastle 1.10 Pharmacists 12.42 -11.32 -15.23 -7.41 0.000 0.000 ****
Newcastle 1.10 PhysicianAssociates 14.20 -13.10 -18.43 -7.77 0.000 0.000 ****
Pharmacists 12.42 PhysicianAssociates 14.20 -1.78 -7.02 3.46 0.501 1.000 ns
  • Alternative figure with the percentage of relevant initial and final diagnoses

    • *For interpretation, note that, on the x axis, “100%” corresponds to including 6 out of 6 relevant diagnoses. Consequently, “50%” corresponds to including 3 out of 6.

    4.5.5 Descriptive tables

    Table showing the change scores in relevant diagnoses from initial to final and the number of students with each.

    • For instance, one student had two fewer relevant diagnoses in their final diagnoses that they had included in their initial diagnoses and five students included four more relevant diagnoses in their final diagnoses.
    label levels 4 3 2 1 0 -1 -2 Total
    Total N (%) 5 (0.8) 34 (5.2) 122 (18.6) 212 (32.3) 251 (38.2) 31 (4.7) 2 (0.3) 657
    Type1 MedStudents_International 2 (0.9) 11 (5.0) 43 (19.6) 76 (34.7) 79 (36.1) 8 (3.7) 219 (100)
    MedStudents_UK 2 (1.3) 11 (6.9) 45 (28.3) 59 (37.1) 38 (23.9) 4 (2.5) 159 (100)
    Pharmacists 1 (0.7) 8 (5.4) 21 (14.1) 48 (32.2) 65 (43.6) 5 (3.4) 1 (0.7) 149 (100)
    Newcastle 1 (1.3) 1 (1.3) 14 (18.4) 47 (61.8) 12 (15.8) 1 (1.3) 76 (100)
    PhysicianAssociates 3 (5.6) 12 (22.2) 15 (27.8) 22 (40.7) 2 (3.7) 54 (100)
    • Table showing the number of students whose change scores in relevant diagnoses from initial to final were better, worse, or the same.
    label levels More Same Fewer Total
    Total N (%) 373 (56.8) 251 (38.2) 33 (5.0) 657
    Type1 MedStudents_International 132 (60.3) 79 (36.1) 8 (3.7) 219 (100)
    MedStudents_UK 117 (73.6) 38 (23.9) 4 (2.5) 159 (100)
    Newcastle 16 (21.1) 47 (61.8) 13 (17.1) 76 (100)
    Pharmacists 78 (52.3) 65 (43.6) 6 (4.0) 149 (100)
    PhysicianAssociates 30 (55.6) 22 (40.7) 2 (3.7) 54 (100)
    • Showing the number of students whose change scores in relevant diagnoses from initial to final were better, worse, or the same.
    label levels Had more relevant diagnoses Had the same or fewer relevant diagnoses Total
    Total N (%) 373 (56.8) 284 (43.2) 657
    Type1 MedStudents_International 132 (60.3) 87 (39.7) 219 (100)
    MedStudents_UK 117 (73.6) 42 (26.4) 159 (100)
    Newcastle 16 (21.1) 60 (78.9) 76 (100)
    Pharmacists 78 (52.3) 71 (47.7) 149 (100)
    PhysicianAssociates 30 (55.6) 24 (44.4) 54 (100)

    4.5.6 Omnibus tests (Chi2 & Fisher’s Exact Test)

    • We conducted chi-squared test with yates continuity correction and Fishers exact test to assess whether the number of students who had more relevant diagnoses was associated with student type. Both tests indicate a significant association between the number of students who had more relevant diagnoses and student type.
    X-squared df p value Fisher p value
    Test of number of students having more relevant diagnoses at final than initial by Student type 60.14 4 0 0

    4.5.7 Follow up tests (Fisher’s Exact Test)

    • The table showing all the pairwise comparisons with Odds Ratios (95%CIs). Raw p values are provided.
    Odds ratio CI lower CI upper raw p value
    Int vs. UK medical students 1.83 1.15 2.94 0.008
    Int Medical students vs. Pharmacists 0.18 0.09 0.33 0.000
    Int medical students vs. Physician associates 0.72 0.47 1.13 0.135
    UK medical students vs. Pharmacists 0.10 0.05 0.19 0.000
    UK medical students vs. Physician associates 0.40 0.24 0.65 0.000
    Pharmacists vs. Physician associates 4.09 2.10 8.34 0.000

    5 Case 3: Post-consultation follow up

    5.1 Management plan

    5.1.1

    Review in GP was identified by experts during development as the correct management plan out of the 6 available and is highlighted in bold font below.

    Category Management plan
    Other Immediate referral to acute secondary care
    Other Non-urgent referral to secondary care
    Other Refer as a 2 week wait to secondary care
    Correct Review in General Practice
    Other She does not require follow-up
    Other Urgent referral to secondary care
    Other Refer for psychological therapy or counselling

    Tables for students’ management plans showing the number and (%) overall and then according to student type, the omnibus tests for association between management plan and student type, and all pairwise follow-up tests of associations.

    5.1.2 Descriptive tables

    • Showing the number of students who chose each option overall and by student group. (Review in GP was identified as the correct management plan by the expert GP panel)
    label levels Immediate referral to acute secondary care Non-urgent referral to secondary care Refer as a 2 week wait to secondary care Review in General Practice She does not require follow-up Urgent referral to secondary care Total
    Total N (%) 79 (12.0) 43 (6.5) 105 (16.0) 268 (40.8) 7 (1.1) 155 (23.6) 657
    Type1 MedStudents_International 31 (14.2) 14 (6.4) 22 (10.0) 85 (38.8) 5 (2.3) 62 (28.3) 219 (100)
    MedStudents_UK 13 (8.2) 9 (5.7) 27 (17.0) 86 (54.1) 1 (0.6) 23 (14.5) 159 (100)
    Newcastle 7 (9.2) 6 (7.9) 19 (25.0) 26 (34.2) 18 (23.7) 76 (100)
    Pharmacists 27 (18.1) 12 (8.1) 23 (15.4) 38 (25.5) 49 (32.9) 149 (100)
    PhysicianAssociates 1 (1.9) 2 (3.7) 14 (25.9) 33 (61.1) 1 (1.9) 3 (5.6) 54 (100)
    • Showing the number and % of students who either did or did not selecte review in GP overall and by student group.
    label levels Did not select review in GP Selected review in GP Total
    Total N (%) 389 (59.2) 268 (40.8) 657
    Type1 MedStudents_International 134 (61.2) 85 (38.8) 219 (100)
    MedStudents_UK 73 (45.9) 86 (54.1) 159 (100)
    Newcastle 50 (65.8) 26 (34.2) 76 (100)
    Pharmacists 111 (74.5) 38 (25.5) 149 (100)
    PhysicianAssociates 21 (38.9) 33 (61.1) 54 (100)

    5.1.3 Omnibus tests (Chi2 & Fisher’s Exact Test)

    • We conducted chi-squared test with Yates continuity correction and Fisher’s exact test to assess whether student self-reported gender was associated with student type. Both tests indicate a significant association between correct management plan selection and student type.
    X-squared df p value Fisher p value
    Test of Correct management plan by Student type 37.01 4 0 0

    5.1.4 Follow up tests (Fisher’s Exact Test)

    • The table showing all the pairwise comparisons with Odds Ratios (95%CIs). Raw p values are provided.
    Odds ratio CI lower CI upper raw p value
    Int vs. UK medical students 1.85 1.20 2.87 0.003
    Int Medical students vs. Pharmacists 0.82 0.45 1.46 0.496
    Int medical students vs. Physician associates 0.54 0.33 0.87 0.010
    UK medical students vs. Pharmacists 0.44 0.24 0.81 0.005
    UK medical students vs. Physician associates 0.29 0.17 0.48 0.000
    Pharmacists vs. Physician associates 0.66 0.35 1.26 0.211

    5.2 Essential investigations [EIs]

    5.2.1

    Three of the six available investigations were identified by experts during development as essential and are listed below.

    Category Follow up investigation Follow up investigation Follow up investigation
    Essential Electrocardiogram Relevant cultures or serological tests Prescription of appropriate medication
    Other CXR Bloods PEFR, Spirometry and Reversibility testing

    Tables 5.2.2 to 5.2.4: Descriptive statistics for the number and % of EIs selected overall and then according to student type, the omnibus anova test for differences in the % of EIs selected by student type, and all pairwise comparison follow-up tests.

    Tables 5.2.5 to 5.2.8: Tables for showing the number and (%) of students who selected each EI overall and then according to student type, the omnibus tests for association between % of students who selected each EIs and student type, all pairwise follow-up tests of associations, and all figures.

    5.2.2 Descriptive stats

    • Descriptive statistics for the total number of essential investigations selected
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 657 1.13 0.75 1 1.12 0 0 3 3 0.27 -0.27 0.03
    • Descriptive statistics for the proportion of essential investigations selected overall
    vars n mean sd median trimmed mad min max range skew kurtosis se
    X1 1 657 37.75 25.15 33.33 37.38 0 0 100 100 0.27 -0.27 0.98
    • Descriptive statistics for the proportion of essential investigations selected by student group
    group1 n mean sd median trimmed mad min max range skew kurtosis se
    MedStudents_International 219 42.31 26.43 33.33 42.00 49.42 0 100 100 0.20 -0.41 1.79
    MedStudents_UK 159 38.36 24.07 33.33 39.02 49.42 0 100 100 -0.03 -0.69 1.91
    Newcastle 76 33.77 25.82 33.33 31.72 0.00 0 100 100 0.66 0.35 2.96
    Pharmacists 149 34.68 24.47 33.33 34.16 0.00 0 100 100 0.24 -0.39 2.00
    PhysicianAssociates 54 31.48 20.90 33.33 30.30 0.00 0 100 100 0.49 1.08 2.84

    5.2.3 Omnibus test (ANOVA)

    Sum Sq Df F value Pr(>F)
    (Intercept) 392105.5 1 630.27 0.000
    Type1 9353.9 4 3.76 0.005
    Residuals 405623.3 652 NA NA
    df sumsq meansq statistic p.value etasq partial.etasq omegasq partial.omegasq epsilonsq cohens.f power
    Type1 4 9354 2338.47 3.76 0.005 0.02 0.02 0.02 0.02 0.02 0.15 0.89
    …2 652 405623 622.12 NA NA NA NA NA NA NA NA NA

    5.2.4 Follow up tests (MDEs)

    • The table shows all pairwise comparisons of the proportion of essential investigations selected by student type.
    Group1: Mean Group2: Mean Mean difference estimate CI lower CI higher raw p value Holm p value
    MedStudents_International 42.31 MedStudents_UK 38.36 3.95 -1.19 9.09 0.132 0.792 ns
    MedStudents_International 42.31 Newcastle 33.77 8.54 1.70 15.38 0.015 0.120
    MedStudents_International 42.31 Pharmacists 34.68 7.64 2.36 12.92 0.005 0.045 **
    MedStudents_International 42.31 PhysicianAssociates 31.48 10.83 4.17 17.50 0.002 0.020 **
    MedStudents_UK 38.36 Newcastle 33.77 4.59 -2.37 11.56 0.195 0.920 ns
    MedStudents_UK 38.36 Pharmacists 34.68 3.69 -1.76 9.14 0.184 0.920 ns
    MedStudents_UK 38.36 PhysicianAssociates 31.48 6.88 0.09 13.68 0.047 0.329
    Newcastle 33.77 Pharmacists 34.68 -0.90 -7.97 6.16 0.801 1.000 ns
    Newcastle 33.77 PhysicianAssociates 31.48 2.29 -5.83 10.42 0.578 1.000 ns
    Pharmacists 34.68 PhysicianAssociates 31.48 3.19 -3.70 10.09 0.361 1.000 ns

    5.2.5 Descriptive tables

    • Showing the number of students who selected 0, 1, 2, or 3 of the essential investigations.
    label levels 0 1 2 3 Total
    Total N (%) 126 (19.2) 341 (51.9) 167 (25.4) 23 (3.5) 657
    Type1 MedStudents_International 34 (15.5) 105 (47.9) 67 (30.6) 13 (5.9) 219 (100)
    MedStudents_UK 29 (18.2) 79 (49.7) 49 (30.8) 2 (1.3) 159 (100)
    Newcastle 18 (23.7) 43 (56.6) 11 (14.5) 4 (5.3) 76 (100)
    Pharmacists 34 (22.8) 78 (52.3) 34 (22.8) 3 (2.0) 149 (100)
    PhysicianAssociates 11 (20.4) 36 (66.7) 6 (11.1) 1 (1.9) 54 (100)
    • Showing the number of students who selected at least one of the essential investigations.
    label levels Did not select any essential investigations Selected at least one essential investigation Total
    Total N (%) 126 (19.2) 531 (80.8) 657
    Type1 MedStudents_International 34 (15.5) 185 (84.5) 219 (100)
    MedStudents_UK 29 (18.2) 130 (81.8) 159 (100)
    Newcastle 18 (23.7) 58 (76.3) 76 (100)
    Pharmacists 34 (22.8) 115 (77.2) 149 (100)
    PhysicianAssociates 11 (20.4) 43 (79.6) 54 (100)
    • Table showing the proportion of students who selected each of the essential investigations.

    5.2.6 Omnibus tests (Chi2 & Fisher’s Exact Test)

    • We conducted chi-squared test with yates continuity correction and Fishers exact test to assess whether selecting at least one essential investigation was associated with student type. Both tests indicate no significant association between selecting at least one essential investigation and student type.
    X-squared df p value Fisher p value
    Test of selecting at least one essential investigation by Student type 4.3 4 0.3675 0.3472
    • We then ran chi-square and fisher omnibus tests for all essential investigations.
    X-squared df p value Fisher p value
    Electrocardiogram 34.59 4 0.000 0.000
    Relevant cultures or serological tests 11.11 4 0.025 0.024
    Prescription of appropriate medication 19.56 4 0.001 0.000

    5.2.7 Follow up tests (Fisher’s Exact Test)

    • The table showing all the pairwise comparisons for having selected at least one essential investigation with Odds Ratios (95%CIs). Raw p values are provided.
    Odds ratio CI lower CI upper raw p value
    Int vs. UK medical students 0.82 0.46 1.48 0.488
    Int Medical students vs. Pharmacists 0.59 0.30 1.20 0.118
    Int medical students vs. Physician associates 0.62 0.35 1.09 0.100
    UK medical students vs. Pharmacists 0.72 0.35 1.49 0.384
    UK medical students vs. Physician associates 0.76 0.42 1.36 0.327
    Pharmacists vs. Physician associates 1.05 0.51 2.10 0.869

    We conducted follow up tests using Fishers exact test for all pairwise comparisons by student type, which showed:

    • Physician associates and UK medical students were the least likely to select “Electrocardiogram”.
    Odds ratio CI lower CI upper raw p value
    Int vs. UK medical students 0.52 0.33 0.80 0.002
    Int Medical students vs. Pharmacists 0.22 0.11 0.41 0.000
    Int medical students vs. Physician associates 0.91 0.59 1.41 0.672
    UK medical students vs. Pharmacists 0.42 0.20 0.83 0.007
    UK medical students vs. Physician associates 1.76 1.09 2.85 0.016
    Pharmacists vs. Physician associates 4.21 2.13 8.71 0.000

    • Physician associates were the least likely to select “Relevant cultures or serological tests” particularly compared to UK and international medical students.
    Odds ratio CI lower CI upper raw p value
    Int vs. UK medical students 0.86 0.56 1.32 0.467
    Int Medical students vs. Pharmacists 1.06 0.61 1.86 0.894
    Int medical students vs. Physician associates 0.68 0.44 1.06 0.088
    UK medical students vs. Pharmacists 1.24 0.69 2.23 0.486
    UK medical students vs. Physician associates 0.79 0.49 1.28 0.358
    Pharmacists vs. Physician associates 0.64 0.35 1.16 0.121

    • Pharmacists were the least likely to select “Prescription of appropriate medication” - compared to all other groups.
    Odds ratio CI lower CI upper raw p value
    Int vs. UK medical students 1.51 0.94 2.45 0.080
    Int Medical students vs. Pharmacists 1.39 0.74 2.57 0.286
    Int medical students vs. Physician associates 0.47 0.25 0.86 0.010
    UK medical students vs. Pharmacists 0.92 0.48 1.72 0.881
    UK medical students vs. Physician associates 0.31 0.16 0.57 0.000
    Pharmacists vs. Physician associates 0.34 0.16 0.71 0.002

    6 Case 3: Student feedback

    Students were asked to indicate their level of agreement with the following statements:

    • “It was easy to navigate through eCREST”,
    • “The level of difficulty of the material was appropriate”,
    • “eCREST should be used to supplement traditional teaching”,
    • “eCREST helped me to learn clinical reasoning skills that I could apply to my clinical work”,
    • “Overall, using eCREST enhanced my learning”,
    • “I would use eCREST for my future professional development”,

    1=Strongly disagree, 2=Disagree, 3=Neither agree or disagree, 4=Agree, 5=Strongly agree

    For analyses these were re-coded as (1=Agree or strongly agree vs. 0=Stongly disagree, disagree, or neither)

    6.1 eCREST Usability

    6.1.1

    Table 6.2 shows the number and proportion of students who agreed or strongly agreed with each statement.

    Tables 6.3 to 6.6 show figures for each statement.

    6.1.2

    • Table showing the number and percentage of students who agreed or strongly agreed with each statement (shown in bolded font).
    label levels MedStudents_International MedStudents_UK Newcastle Pharmacists PhysicianAssociates Total p
    Total N (%) 203 (36.7) 119 (21.5) 60 (10.8) 131 (23.7) 40 (7.2) 553
    It was easy to navigate through eCREST Agree or strongly agree 174 (85.7) 116 (97.5) 51 (85.0) 112 (85.5) 38 (95.0) 491 (88.8) 0.033
    Neither 22 (10.8) 3 (2.5) 8 (13.3) 16 (12.2) 1 (2.5) 50 (9.0)
    Strongly disagree or disagree 7 (3.4) 1 (1.7) 3 (2.3) 1 (2.5) 12 (2.2)
    The level of difficulty of the material was appropriate Agree or strongly agree 165 (81.3) 114 (95.8) 38 (63.3) 84 (64.1) 37 (92.5) 438 (79.2) <0.001
    Neither 29 (14.3) 5 (4.2) 14 (23.3) 31 (23.7) 2 (5.0) 81 (14.6)
    Strongly disagree or disagree 9 (4.4) 8 (13.3) 16 (12.2) 1 (2.5) 34 (6.1)
    eCREST should be used to supplement traditional teaching Agree or strongly agree 156 (76.8) 110 (92.4) 39 (65.0) 104 (79.4) 35 (87.5) 444 (80.3) 0.001
    Neither 31 (15.3) 7 (5.9) 11 (18.3) 20 (15.3) 4 (10.0) 73 (13.2)
    Strongly disagree or disagree 16 (7.9) 2 (1.7) 10 (16.7) 7 (5.3) 1 (2.5) 36 (6.5)
    eCREST helped me to learn clinical reasoning skills that I could apply to my clinical work Agree or strongly agree 158 (77.8) 99 (83.2) 37 (61.7) 106 (80.9) 35 (87.5) 435 (78.7) 0.071
    Neither 35 (17.2) 16 (13.4) 19 (31.7) 20 (15.3) 4 (10.0) 94 (17.0)
    Strongly disagree or disagree 10 (4.9) 4 (3.4) 4 (6.7) 5 (3.8) 1 (2.5) 24 (4.3)
    Overall, using eCREST enhanced my learning Agree or strongly agree 162 (79.8) 103 (86.6) 36 (60.0) 110 (84.0) 34 (85.0) 445 (80.5) 0.004
    Neither 31 (15.3) 14 (11.8) 18 (30.0) 18 (13.7) 5 (12.5) 86 (15.6)
    Strongly disagree or disagree 10 (4.9) 2 (1.7) 6 (10.0) 3 (2.3) 1 (2.5) 22 (4.0)
    I would use eCREST for my future professional development Agree or strongly agree 131 (79.9) 4 (100.0) 38 (63.3) 105 (80.2) 36 (90.0) 314 (78.7) 0.044
    Neither 23 (14.0) 16 (26.7) 23 (17.6) 3 (7.5) 65 (16.3)
    Strongly disagree or disagree 10 (6.1) 6 (10.0) 3 (2.3) 1 (2.5) 20 (5.0)

    6.1.3

    6.1.4 Easy

    6.1.5 Difficulty

    6.1.6 Teaching

    6.1.7 Reasoning

    6.1.8 Overall

    6.1.9 Use