## Total encounters: 47497
## Deterioration events: 1324
## Event rate: 2.79 %
## Analytic n: 47497
## Training set: n = 33247 | events = 921
## Test set: n = 14250 | events = 403
## # A tibble: 2 × 2
## separation Encounters
## <fct> <int>
## 1 Admitted 14447
## 2 Not admitted 33050
## [1] 47497
Variable | Overall | No deterioration | Deterioration | p-value2 |
|---|---|---|---|---|
Age group | <0.001 | |||
16-19 | 2,809 (5.9%) | 2,785 (6.0%) | 24 (1.8%) | |
20-39 | 18,079 (38.1%) | 17,933 (38.8%) | 146 (11.0%) | |
40-59 | 13,175 (27.7%) | 12,933 (28.0%) | 242 (18.3%) | |
60-79 | 9,640 (20.3%) | 9,127 (19.8%) | 513 (38.7%) | |
80+ | 3,794 (8.0%) | 3,395 (7.4%) | 399 (30.1%) | |
Sex | 0.006 | |||
Male | 22,883 (48.2%) | 22,195 (48.1%) | 688 (52.0%) | |
Female | 24,611 (51.8%) | 23,975 (51.9%) | 636 (48.0%) | |
Not recorded | 3 | 3 | 0 | |
Mode of arrival | <0.001 | |||
Ambulance | 9,720 (20.5%) | 9,021 (19.5%) | 699 (52.8%) | |
Public transport | 220 (0.5%) | 219 (0.5%) | 1 (0.1%) | |
Private vehicle | 36,410 (76.7%) | 35,830 (77.6%) | 580 (43.8%) | |
Helicopter | 9 (0.0%) | 9 (0.0%) | 0 (0.0%) | |
Air ambulance | 2 (0.0%) | 2 (0.0%) | 0 (0.0%) | |
Internal ambulance | 154 (0.3%) | 134 (0.3%) | 20 (1.5%) | |
Police | 486 (1.0%) | 479 (1.0%) | 7 (0.5%) | |
Walked in | 250 (0.5%) | 248 (0.5%) | 2 (0.2%) | |
Retrieval team | 3 (0.0%) | 3 (0.0%) | 0 (0.0%) | |
Internal bed | 243 (0.5%) | 228 (0.5%) | 15 (1.1%) | |
Triage category (ATS) | <0.001 | |||
3 - Urgent | 27,318 (57.5%) | 26,242 (56.8%) | 1,076 (81.3%) | |
4 - Semi-Urgent | 16,490 (34.7%) | 16,259 (35.2%) | 231 (17.4%) | |
5 - Non-Urgent | 3,689 (7.8%) | 3,672 (8.0%) | 17 (1.3%) | |
Time of arrival | <0.001 | |||
0800-1759 | 28,281 (59.5%) | 27,426 (59.4%) | 855 (64.6%) | |
1800-2259 | 10,507 (22.1%) | 10,253 (22.2%) | 254 (19.2%) | |
2300-0759 | 8,709 (18.3%) | 8,494 (18.4%) | 215 (16.2%) | |
Prior hospital admission (30 days) | 2,608 (5.5%) | 2,449 (5.3%) | 159 (12.0%) | <0.001 |
Presenting problem category | <0.001 | |||
Abdominal/GI | 9,323 (19.6%) | 9,015 (19.5%) | 308 (23.3%) | |
Cardiovascular | 1,110 (2.3%) | 1,080 (2.3%) | 30 (2.3%) | |
General symptoms | 5,331 (11.2%) | 5,013 (10.9%) | 318 (24.0%) | |
Febrile illness | 1,239 (2.6%) | 1,173 (2.5%) | 66 (5.0%) | |
Injury | 6,359 (13.4%) | 6,297 (13.6%) | 62 (4.7%) | |
Respiratory | 2,112 (4.4%) | 1,954 (4.2%) | 158 (11.9%) | |
Musculoskeletal | 3,981 (8.4%) | 3,898 (8.4%) | 83 (6.3%) | |
Neurological | 3,227 (6.8%) | 3,148 (6.8%) | 79 (6.0%) | |
Mental health | 1,940 (4.1%) | 1,909 (4.1%) | 31 (2.3%) | |
Toxicological | 253 (0.5%) | 250 (0.5%) | 3 (0.2%) | |
ENT/eye/head & neck | 4,700 (9.9%) | 4,657 (10.1%) | 43 (3.2%) | |
Administrative | 1,568 (3.3%) | 1,525 (3.3%) | 43 (3.2%) | |
Genitourinary | 1,977 (4.2%) | 1,940 (4.2%) | 37 (2.8%) | |
Endocrine | 110 (0.2%) | 101 (0.2%) | 9 (0.7%) | |
Obstetrics/Gynaecology | 1,734 (3.7%) | 1,723 (3.7%) | 11 (0.8%) | |
Skin/allergy | 2,521 (5.3%) | 2,479 (5.4%) | 42 (3.2%) | |
Other medical | 12 (0.0%) | 11 (0.0%) | 1 (0.1%) | |
START score | 11.3 (6.7) | 11.1 (6.6) | 18.3 (5.4) | <0.001 |
1n (%); Mean (SD) | ||||
2Pearson's Chi-squared test; Welch Two Sample t-test | ||||
| Outcome | n | Rate (%) |
|---|---|---|
| RRC activation | 1056 | 2.22 |
| Code Blue | 385 | 0.81 |
| ICU admission | 115 | 0.24 |
| In-hospital death | 217 | 0.46 |
| Composite (any) | 1324 | 2.79 |
## START score AUC: 0.7997
## 95% CI (DeLong): 0.7885 - 0.811
Criterion | Rule | Training sensitivity (%) | Training specificity (%) |
|---|---|---|---|
Youden | START >= 16 | 71.1 | 74.6 |
Sensitivity >=85% | START >= 13 | 85.1 | 56.2 |
Sensitivity >=90% | START >= 11 | 91.4 | 45.6 |
Training | Fixed | Se (%) | Sp (%) | PPV (%) | NPV (%) | Flagged | Missed |
|---|---|---|---|---|---|---|---|
Youden | START >= 16 | 69.5 | 75.1 | 7.5 | 98.8 | 3,726 | 123 / 403 |
Sensitivity >=85% | START >= 13 | 85.4 | 57.0 | 5.5 | 99.3 | 6,302 | 59 / 403 |
Sensitivity >=90% | START >= 11 | 92.1 | 46.7 | 4.8 | 99.5 | 7,757 | 32 / 403 |
All thresholds were selected using training data only and applied unchanged to the test cohort. | |||||||
Se: sensitivity; Sp: specificity; PPV/NPV: positive/negative predictive value. CI: exact binomial 95% confidence interval; NE: not estimable. | |||||||
Flagged percentages use all test encounters; missed percentages use test deterioration events. Sensitivity targets apply to training estimates, not guaranteed test performance. | |||||||
Figure X. Observed deterioration rate across the START score range. Shaded band shows 95% confidence interval. Dashed line indicates optimal cutpoint (score = 14) by Youden index. Histogram shows score distribution.
Each START component variable entered separately.
Characteristic | OR | 95% CI | p-value |
|---|---|---|---|
Age group | |||
16-19 | — | — | |
20-39 | 0.94 | 0.62, 1.49 | 0.8 |
40-59 | 2.17 | 1.46, 3.40 | <0.001 |
60-79 | 6.52 | 4.42, 10.1 | <0.001 |
80+ | 13.6 | 9.22, 21.2 | <0.001 |
Ambulance arrival | |||
No | — | — | |
Yes | 4.80 | 4.29, 5.36 | <0.001 |
Triage category (ATS) | |||
3 - Urgent | — | — | |
4 - Semi-Urgent | 0.35 | 0.30, 0.40 | <0.001 |
5 - Non-Urgent | 0.11 | 0.07, 0.18 | <0.001 |
Time of arrival | |||
0800-1759 | — | — | |
1800-2259 | 0.79 | 0.69, 0.91 | 0.001 |
2300-0759 | 0.81 | 0.70, 0.94 | 0.007 |
Prior hospital admission (30 days) | |||
No | — | — | |
Yes | 2.44 | 2.05, 2.88 | <0.001 |
Presenting problem category | |||
Abdominal/GI | — | — | |
Cardiovascular | 0.81 | 0.54, 1.17 | 0.3 |
General symptoms | 1.86 | 1.58, 2.18 | <0.001 |
Febrile illness | 1.65 | 1.24, 2.15 | <0.001 |
Injury | 0.29 | 0.22, 0.38 | <0.001 |
Respiratory | 2.37 | 1.94, 2.88 | <0.001 |
Musculoskeletal | 0.62 | 0.48, 0.79 | <0.001 |
Neurological | 0.73 | 0.57, 0.94 | 0.016 |
Mental health | 0.48 | 0.32, 0.68 | <0.001 |
Toxicological | 0.35 | 0.09, 0.93 | 0.073 |
ENT/eye/head & neck | 0.27 | 0.19, 0.37 | <0.001 |
Administrative | 0.83 | 0.59, 1.13 | 0.2 |
Genitourinary | 0.56 | 0.39, 0.78 | <0.001 |
Endocrine | 2.61 | 1.22, 4.92 | 0.007 |
Obstetrics/Gynaecology | 0.19 | 0.10, 0.32 | <0.001 |
Skin/allergy | 0.50 | 0.35, 0.68 | <0.001 |
Other medical | 2.66 | 0.15, 13.7 | 0.3 |
Abbreviations: CI = Confidence Interval, OR = Odds Ratio | |||
## START-predictor model: Firth mean bias-reduced logistic regression
## deterioration ~ age + ambulance + triage + time_code + prior_adm +
## destiny
## Training observations: 33247
##
## Rare-category coefficient and standard error:
## Estimate Std. Error
## destinyOther medical 0.7621785 1.674997
Characteristic | Adjusted OR | 95% CI | p-value |
|---|---|---|---|
Age group | |||
16-19 | — | — | |
20-39 | 0.67 | 0.41, 1.08 | 0.10 |
40-59 | 1.33 | 0.84, 2.11 | 0.2 |
60-79 | 3.14 | 2.00, 4.94 | <0.001 |
80+ | 4.49 | 2.83, 7.13 | <0.001 |
Ambulance arrival | |||
No | — | — | |
Yes | 2.63 | 2.27, 3.06 | <0.001 |
Triage category (ATS) | |||
3 - Urgent | — | — | |
4 - Semi-Urgent | 0.53 | 0.44, 0.63 | <0.001 |
5 - Non-Urgent | 0.29 | 0.17, 0.48 | <0.001 |
Time of arrival | |||
0800-1759 | — | — | |
1800-2259 | 0.84 | 0.70, 1.00 | 0.044 |
2300-0759 | 0.77 | 0.63, 0.93 | 0.006 |
Prior hospital admission (30 days) | |||
No | — | — | |
Yes | 1.39 | 1.13, 1.73 | 0.002 |
Presenting problem category | |||
Abdominal/GI | — | — | |
Cardiovascular | 0.45 | 0.27, 0.74 | 0.002 |
General symptoms | 1.51 | 1.24, 1.85 | <0.001 |
Febrile illness | 1.53 | 1.09, 2.15 | 0.013 |
Injury | 0.38 | 0.27, 0.53 | <0.001 |
Respiratory | 1.38 | 1.08, 1.76 | 0.010 |
Musculoskeletal | 0.70 | 0.52, 0.95 | 0.024 |
Neurological | 0.55 | 0.41, 0.75 | <0.001 |
Mental health | 0.56 | 0.36, 0.87 | 0.010 |
Toxicological | 0.52 | 0.18, 1.54 | 0.2 |
ENT/eye/head & neck | 0.44 | 0.30, 0.65 | <0.001 |
Administrative | 0.93 | 0.62, 1.39 | 0.7 |
Genitourinary | 0.51 | 0.34, 0.76 | <0.001 |
Endocrine | 0.79 | 0.26, 2.38 | 0.7 |
Obstetrics/Gynaecology | 0.54 | 0.28, 1.05 | 0.069 |
Skin/allergy | 0.92 | 0.62, 1.37 | 0.7 |
Other medical | 2.14 | 0.08, 57.1 | 0.6 |
Abbreviations: CI = Confidence Interval, OR = Odds Ratio | |||
## Firth START-predictor MV model AUC (test set): 0.8178
## 95% CI (DeLong): 0.7991 - 0.8365
Measure | Ordinary logistic regression | Firth logistic regression |
|---|---|---|
Mean predicted training risk (%) | 2.770 | 2.806 |
Mean predicted test risk (%) | 2.723 | 2.758 |
Test AUROC (95% CI) | 0.8162 | 0.8178 |
Test calibration-in-the-large (95% CI; ideal 0) | 0.0416 | 0.0276 |
Test calibration slope (95% CI; ideal 1) | 0.9343 | 0.9600 |
Numerical convergence | Yes | Yes |
Other medical: coefficient (standard error) | -9.690 (168.329) | 0.762 (1.675) |
Training: n = 33,247, observed event rate 2.770%. Test: n = 14,250, observed event rate 2.828%. | ||
Both models use identical START predictor coding and training/test partitions. AUROC CIs use DeLong; calibration CIs use Wald intervals. | ||
Numerical convergence does not establish a finite MLE under separation. The Other medical training category has nine encounters and no events; its MLE estimate is unstable. | ||
Calibration regressions assess the original predictions only. They do not recalibrate either model. | ||
## Firth START-predictor MV model calibration
## Hosmer-Lemeshow: X² = 15.202 | df = 8 | p = 0.0553
## START MV vs. START analogue
| Model | AUC | 95% CI |
|---|---|---|
| Original START score | 0.7982 | 0.779 - 0.818 |
| Firth START-predictor MV model | 0.8178 | 0.799 - 0.837 |
## Model C scale: 0.07330840 log-odds per point.
## Intercept retained for approximate probabilities: -4.03929 ; excluded from the point total.
## Reference categories contribute zero points; negative weights are retained.
| Term | Coefficient | Points | Rounded coefficient |
|---|---|---|---|
| age20-39 | -0.40529 | -6 | -0.43985 |
| age40-59 | 0.28548 | 4 | 0.29323 |
| age60-79 | 1.14558 | 16 | 1.17293 |
| age80+ | 1.50230 | 20 | 1.46617 |
| ambulanceYes | 0.96835 | 13 | 0.95301 |
| triage4 - Semi-Urgent | -0.64084 | -9 | -0.65978 |
| triage5 - Non-Urgent | -1.24892 | -17 | -1.24624 |
| time_code1800-2259 | -0.17847 | -2 | -0.14662 |
| time_code2300-0759 | -0.26562 | -4 | -0.29323 |
| prior_admYes | 0.33239 | 5 | 0.36654 |
| destinyCardiovascular | -0.80103 | -11 | -0.80639 |
| destinyGeneral symptoms | 0.41468 | 6 | 0.43985 |
| destinyFebrile illness | 0.42686 | 6 | 0.43985 |
| destinyInjury | -0.97479 | -13 | -0.95301 |
| destinyRespiratory | 0.32164 | 4 | 0.29323 |
| destinyMusculoskeletal | -0.35065 | -5 | -0.36654 |
| destinyNeurological | -0.59050 | -8 | -0.58647 |
| destinyMental health | -0.57472 | -8 | -0.58647 |
| destinyToxicological | -0.64744 | -9 | -0.65978 |
| destinyENT/eye/head & neck | -0.82436 | -11 | -0.80639 |
| destinyAdministrative | -0.07331 | -1 | -0.07331 |
| destinyGenitourinary | -0.66991 | -9 | -0.65978 |
| destinyEndocrine | -0.24196 | -3 | -0.21993 |
| destinyObstetrics/Gynaecology | -0.61519 | -8 | -0.58647 |
| destinySkin/allergy | -0.08269 | -1 | -0.07331 |
| destinyOther medical | 0.76218 | 10 | 0.73308 |
## Common test sample: n = 14250 | events = 403 | excluded = 0
| Model | AUROC | 95% CI |
|---|---|---|
| A. Original START score | 0.7982 | 0.779 - 0.818 |
| B. Firth START-predictor MV model | 0.8178 | 0.799 - 0.837 |
| C. Integer simplification of B | 0.8174 | 0.799 - 0.836 |
## Paired DeLong: AUROC difference (B - C) = 0.0004 | 95% CI: -0.0003 to 0.0011 | p = 0.222
Model | AUROC | 95% CI (DeLong) |
|---|---|---|
A: Original START | 0.7982 | 0.7788 to 0.8177 |
C: Integer simplification | 0.8174 | 0.7986 to 0.8362 |
Common test sample: n = 14,250, events = 403. Paired DeLong difference (C - A) = 0.0192; 95% CI 0.0091 to 0.0292; p = 0.000189. | ||
Criterion | Rule | Training sensitivity (%) | Training specificity (%) |
|---|---|---|---|
Youden | C score >= 8 | 75.8 | 76.3 |
Sensitivity >=85% | C score >= 0 | 86.5 | 61.2 |
Sensitivity >=90% | C score >= -3 | 90.1 | 56.3 |
Training cohort only: n = 33,247, events = 921. All rules use the existing signed Model C integer total; negative cutpoints are valid. | |||
Youden ties favour higher specificity, then a higher cutpoint. Sensitivity targets select the highest integer cutpoint meeting the target in training. | |||
Training | Fixed | Se (%) | Sp (%) | PPV (%) | NPV (%) | Flagged | Missed |
|---|---|---|---|---|---|---|---|
Youden | C score >= 8 | 72.0 | 76.9 | 8.3 | 99.0 | 3,485 | 113 / 403 |
Sensitivity >=85% | C score >= 0 | 84.6 | 61.9 | 6.1 | 99.3 | 5,612 | 62 / 403 |
Sensitivity >=90% | C score >= -3 | 88.8 | 57.0 | 5.7 | 99.4 | 6,314 | 45 / 403 |
Test cohort: n = 14,250, events = 403. Thresholds were derived in the overall training cohort and applied unchanged; no test-set optimisation or model refitting. | |||||||
Se: sensitivity; Sp: specificity; PPV/NPV: positive/negative predictive value. CIs are exact binomial. Flagged percentages use all test encounters; missed percentages use test deterioration events. | |||||||
The 85% and 90% targets concern training point estimates and are not guaranteed in the test cohort. These are Model C thresholds, distinct from published START thresholds. NE: not estimable. | |||||||
Subgroup | N | Events | Event rate (%) | A: START | B: Firth | C: Integer |
|---|---|---|---|---|---|---|
Admitted | 4,273 | 391 | 9.15 | 0.6652 | 0.6931 | 0.6928 |
Not admitted | 9,977 | 12 | 0.12 | 0.8283 | 0.8554 | 0.8559 |
Existing held-out test cohort only. A, B and C use identical encounters within each subgroup. AUROC CIs use DeLong; event-rate CIs are exact binomial. | ||||||
Admission status describes final disposition. These analyses are exploratory; the non-admitted subgroup has few events. Differences between subgroup AUROCs do not establish an interaction. NE: not estimable. | ||||||
Subgroup | Training criterion | Se (%) | Sp (%) | PPV (%) | NPV (%) | Flagged | Missed |
|---|---|---|---|---|---|---|---|
Admitted | Youden | 69.8 | 55.2 | 13.57 | 94.78 | 2,012 | 118 / 391 |
Admitted | Sensitivity >=85% | 85.4 | 34.1 | 11.54 | 95.87 | 2,894 | 57 / 391 |
Admitted | Sensitivity >=90% | 92.3 | 25.1 | 11.04 | 97.01 | 3,269 | 30 / 391 |
Not admitted | Youden | 58.3 | 82.9 | 0.41 | 99.94 | 1,714 | 5 / 12 |
Not admitted | Sensitivity >=85% | 83.3 | 65.9 | 0.29 | 99.97 | 3,408 | 2 / 12 |
Not admitted | Sensitivity >=90% | 83.3 | 55.1 | 0.22 | 99.96 | 4,488 | 2 / 12 |
Model A (published START) thresholds were selected in the overall training cohort and applied unchanged within each test subgroup. | |||||||
Se: sensitivity; Sp: specificity; PPV/NPV: positive/negative predictive value. Metric CIs are exact binomial. Flagged percentages use the subgroup denominator; missed counts use subgroup deterioration events. | |||||||
Sensitivity targets concern the overall training cohort and are not guaranteed within subgroups. These thresholds apply to original START, not Model C. | |||||||
Model | AUROC | 95% CI (DeLong) |
|---|---|---|
ATS category alone | 0.6396 | 0.6219 to 0.6573 |
ATS category + age band | 0.7709 | 0.7492 to 0.7926 |
Original START | 0.7982 | 0.7788 to 0.8177 |
Firth START-predictor model | 0.8178 | 0.7991 to 0.8365 |
Training: n = 33,247, events = 921. Evaluation: identical 14,250 held-out encounters, including 403 events, for every model. | ||
Reduced models use Firth regression with categorical ATS and the existing age bands (16-19, 20-39, 40-59, 60-79, 80+), without interaction or variable selection. ATS-only AUROC matches raw ATS ranking. | ||
Contrast | AUROC | 95% CI | Raw p | Holm p |
|---|---|---|---|---|
Original START minus ATS category alone | 0.1587 | 0.1413 to 0.1760 | < 1e-16 | < 1e-16 |
Firth START-predictor model minus ATS category alone | 0.1783 | 0.1563 to 0.2003 | < 1e-16 | < 1e-16 |
Original START minus ATS category + age band | 0.0273 | 0.0127 to 0.0420 | 0.000257 | 0.000257 |
Firth START-predictor model minus ATS category + age band | 0.0469 | 0.0337 to 0.0602 | 3.7e-12 | 7.4e-12 |
Paired DeLong contrasts use identical test encounters. Positive differences favour the START benchmark. Holm adjustment covers these four exploratory comparisons; confidence intervals are unadjusted. | ||||
This assesses discrimination. A nonsignificant difference would not establish equivalence or comparable clinical utility. | ||||
Component | Encounters | % of 12 cases |
|---|---|---|
RRC | 0 | 0.0 |
Code Blue | 0 | 0.0 |
ICU admission | 0 | 0.0 |
Mortality | 12 | 100.0 |
Existing held-out test cohort only. Component counts are inclusive: an encounter can have more than one component. | ||
These are recorded binary flags, not adjudicated causes or timings. In particular, the Mortality flag does not establish the time, place or circumstances of death. | ||
Recorded component combination | Encounters |
|---|---|
Mortality only | 12 |
Combinations are mutually exclusive and sum to 12 encounters. | |
An encounter-level audit, including the source identifier, is retained in na12_audit; identifiers are not printed in this report. | |