Professor Green,

Thank you greatly for your time reviewing the prognostic modeling paper. Your suggestions, again, greatly improved the quality of the manuscript. I addressed/accepted the majority of your comments in the google doc. What follows are lengthier responses to some of your inquires.

Our model compared to GCS

You asked a great question, how does our model compare to GCS alone? The figure below compares our model to GCS as well as CRASH and IMPACT. CRASH and IMPACT are the gold standard prognostic models in this space, TBI prognostic models at hospital presentation. IMPACT only uses data from high income countries. CRASH uses data from HIC and LMICs. CRASH and IMPACT each used ~ 10,000 patient for model construction.

Our approach

First, this figure is awesome and it is an accurate summary of our approach. As you said, we believe the current model of intuition and basic treatment algorithms are inadequate. Additionally, we think current methods of presenting risk scores can be improved. A risk score with or without surgery may be more easily digested than a risk score from 0-100. A risk score of 0-100 must be incorporated by the provider into the decision to operate or not.

I have had many conversations with Joao about this approach, it’s a tough one for me to wrap my head completely around. He has seen this approach used before, but rarely, and we were unable to identify any similar examples in the literature. No question, we are the minority with this technique. There are limitations to this approach. I have added these limitations to the end of the strengths and weaknesses paragraph. The benefit of this approach is the ability to assess prognosis with or without treatments, as you pointed out in your comments.

I realize this is an unsatisfactory answer. I will continue to inquire, learn and search the literature.

Imputation

We decided on 10 iterations for imputation based on Buuren et al.’s mice: Multivariate Imputation by Chained Equations in R.1 In the document, they mention 10-20 iterations are sufficient. I, too, do not know this area well.

Matrix figure for ROC curves

You are right in that changing this figure to a matrix layout would greatly improve readability. The goal of the superimposed figure was to show we used multiple techniques to machine learning and one had the highest ROC. Given the clinical oriented readership, we felt this simple objective was appropriate. Then the following table is for anyone curious about the actual numbers. I apologize, I was unable to convince the team to change this figure.

Surgery ward to ICU variable

This variable is confusing and is an outcome of sorts. We agreed with your comment and removed the surg_to_icu variable from the model.

Web and mobile application

You had a couple comments here which were spot on. The A/B components of the figure were unclear. I have adjusted the figure to more clearly indiciate one is a web application and one is a mobile application.

You also commented that confidence intervals (CI) should be included and you are 100% correct. Before we deploy this app we must include CI. Unfortunately, I do not think we will be able to make this adjustment before resubmission of this paper. I dont have ownership of this version of the shiny app code and the change will take some time. I am sorry I could not address this comment.

Additional comments addressed