Descriptive analysis

For a total of 771 subjects, the distributions of Rapid AI and neuroradiologist ASPECTS scores are plotted below by discharge status. The discriminating power the scores is indicated by the degree to which the distribution with good outcomes are separated from that with bad ones. It is clear that the clinician score is perfectly discriminating, while the Rapid AI score is less so.

Receiver operating characteristic (ROC) analysis

The ROC curves of the two ASPECTS scores with respect to discharge status are plotted in Figure 1.

Figure 1. Receiver operating characteristic (ROC) curves for identifying good vs bad outcomes, areas under the curve (AUC), and 95% confidence intervals (CI).

Figure 1. Receiver operating characteristic (ROC) curves for identifying good vs bad outcomes, areas under the curve (AUC), and 95% confidence intervals (CI).

Comparing the AUC

We use a paired permutation test for the null hypothesis \(H_0:\mbox{AUC}_\mbox{clin.}=\mbox{AUC}_\mbox{AI}\). The \(p\)-value based on the difference in AUC among \(N=2000\) permutation samples is calculated to be \(<0.0001\). Therefore, we conclude that the Rapid AI score is significantly less accurate in predicting discharge status than the neuroradiologist score is.

Prediction of cerebral perfusion (CBF) by ASPECTS scores

The CBF shows only moderate associations with the ASPECTS scores by AI and clinician, with Spearman correlations -0.317 and -0.464, respectively.). A paired permutation test for their difference leads to \(p\)-value 0.0052, which means that the clinician score is significantly better than AI.

Correlation of NIHSS with ASPECTS scores

The NIHSS shows only moderate associations with the ASPECTS scores by AI and clinician, with Spearman correlations -0.287 and -0.255, respectively.). A paired permutation test for their difference leads to \(p\)-value 0.5712, which means that there is no significant difference between the two types of scores in their correlation with NIHSS.