Table 1. Charasteristics of patients at IRU admission
var value
N 273
Age (years) 53.0 ± 17.8
Sex (Females) 93 (34.1)
Age Score
0 107 (39.2)
1 74 (27.1)
2 92 (33.7)
Saliva Score = 1 82 (30.0)
Vegetative State 105 (38.5)
Coughing Score
0 66 (24.2)
1 50 (18.3)
2 46 (16.8)
3 111 (40.7)
Pathogenesis of brain lesion Score
0 22 (8.1)
1 140 (51.3)
2 16 (5.9)
3 95 (34.8)
CRS-R 11.0 (5.0 - 21.0)
Feeding
0 12 (4.4)
1 158 (57.9)
2 103 (37.7)
Brainstem 43 (15.8)
Length of stay ICU (days) 36.0 (28.0 - 49.0)
Decann 168 (61.5)

Methods

To evaluate the DecaPreT model we used a two steps approach: first we followed the procedures presented in Vergouwe et al. (2016) to eventually update the original prediction model. Then, we tested whether the inclusion of new or modified predictors might improve the performance of the updated model.

Model update

To update the original DecaPreT model we developed four logistic model: i) The Original model with the original regression coefficients; ii) The Calibration in the large model with the intercept as the only free parameter; iii) The Logistic calibration model, where both intercept and slope are recalibrated; iv) The Revision model, where all the coefficients of the original predictors are re-estimated.

The last step of the selection procedure consists of a series of likelihood ratio tests of the updated models against the original model. At the end one of the four logistic model is selected.

Model reclassification

To further test for possible improvement of the DecaPret model, we modified the model selected in the update phase fitting new predictors. We compared these Reclassification models evaluating three different class of metrics: discrimination by C-statistics, calibration by calibration plot and difference in predicted and loess-calibrated probabilities (E_{max}, Frank E. Harrell Jr. (2001)), and reclassification by continuous Net Reclassification Index (NRI>0) and Integral Discrimination Index (IDI).

Results

Table 2. Likelihood ratio test for Original, Calibration at large, Logistic calibration vs the Revised model
comparison df Chisq p
revis_orig 6 36.62088 2.1e-06
revis_recal 5 34.39320 2.0e-06
revis_logcal 4 34.38091 6.0e-07
Table 3. Discrimination and Calibration metrics of the developed models
model C (ROC) Brier Slope Emax Eavg
Model Revision 0.879 (0.838-0.926) 0.480 (0.390-0.610) 0.944 (0.901-0.947) 0.032 (0.020-0.130) 0.013 (0.000-0.050)
CRS Score 0.879 (0.850-0.925) 0.480 (0.380-0.610) 0.916 (0.899-0.943) 0.023 (0.010-0.120) 0.010 (0.000-0.040)
CRS continuous 0.887 (0.847-0.928) 0.480 (0.390-0.620) 0.951 (0.911-0.965) 0.031 (0.020-0.120) 0.012 (0.000-0.050)
Age continuous 0.899 (0.869-0.942) 0.520 (0.430-0.650) 0.925 (0.907-0.952) 0.053 (0.020-0.130) 0.011 (0.000-0.040)
ICU score 0.890 (0.864-0.933) 0.500 (0.410-0.630) 0.922 (0.906-0.956) 0.030 (0.020-0.130) 0.005 (0.000-0.040)
ICU continuous 0.889 (0.862-0.931) 0.500 (0.410-0.630) 0.899 (0.863-0.910) 0.026 (0.010-0.120) 0.003 (0.000-0.040)
Feeding 0.899 (0.879-0.948) 0.520 (0.440-0.650) 0.894 (0.857-0.915) 0.052 (0.020-0.140) 0.014 (0.000-0.040)
Brainstem 0.905 (0.862-0.936) 0.520 (0.430-0.640) 0.933 (0.880-0.932) 0.059 (0.020-0.140) 0.014 (0.000-0.040)
Revision continuous 0.876 (0.855-0.917) 0.470 (0.360-0.580) 0.949 (0.923-0.973) 0.063 (0.020-0.150) 0.022 (0.000-0.060)
age, CRS and ICU continuous 0.884 (0.852-0.917) 0.480 (0.380-0.600) 0.969 (0.924-0.968) 0.032 (0.020-0.140) 0.014 (0.000-0.050)
age, CRS and days in ICU score 0.886 (0.851-0.934) 0.480 (0.390-0.610) 0.947 (0.921-0.964) 0.023 (0.020-0.140) 0.010 (0.000-0.050)
Complete continuous 0.876 (0.827-0.928) 0.470 (0.370-0.600) 0.954 (0.920-0.969) 0.080 (0.020-0.170) 0.027 (0.000-0.060)
Complete score 0.892 (0.869-0.935) 0.500 (0.420-0.620) 0.933 (0.898-0.945) 0.030 (0.020-0.120) 0.005 (0.000-0.040)
Stepwise continuous 0.898 (0.865-0.942) 0.520 (0.440-0.640) 0.911 (0.886-0.939) 0.060 (0.020-0.130) 0.009 (0.000-0.040) 

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null

## [1] 1null
Table 4. Reclassification metrics of the developed models
model ci.nri p.nri ci.idi p.idi
fit5 0.148 (-0.056-0.352) 0.156 0.013 (-0.014-0.039) 0.341
fit6 -0.105 (-0.336-0.126) 0.374 0.016 (-0.013-0.044) 0.290
fit7 0.495 (0.262-0.729) 0.000 0.022 (-0.001-0.045) 0.061
fit8 0.724 (0.497-0.950) 0.000 0.057 (0.026-0.089) 0.000
fit9 0.469 (0.232-0.706) 0.000 0.038 (0.008-0.069) 0.015
fit10 0.462 (0.224-0.699) 0.000 0.041 (0.008-0.074) 0.015
fit11 0.538 (0.303-0.773) 0.000 0.062 (0.028-0.095) 0.000
fit12 0.483 (0.246-0.720) 0.000 0.062 (0.028-0.095) 0.000
fit13 0.148 (-0.056-0.352) 0.156 0.013 (-0.013-0.039) 0.335
fit14 0.500 (0.266-0.734) 0.000 0.020 (-0.003-0.043) 0.083
fit15 -0.017 (-0.260-0.227) 0.893 0.002 (-0.002-0.005) 0.376
fit16 0.469 (0.232-0.706) 0.000 0.038 (0.008-0.069) 0.015
fit17 0.593 (0.363-0.823) 0.000 0.056 (0.023-0.090) 0.001