This SOP provides a concise and reusable framework for clinical statistical modeling.
It is intended for common clinical and epidemiologic modeling tasks, including:
The objective is to ensure that model development, assessment, validation, interpretation, and reporting are scientifically justified, reproducible, and clinically meaningful.
Before analysis, clearly define:
The modeling objective should be classified as one of the following:
The model-building strategy and interpretation should follow the scientific objective.
Prespecify:
Variables that occur after the outcome or directly encode the outcome should not be used as predictors because they may introduce outcome leakage.
Before modeling, assess:
Potential errors should be investigated rather than automatically deleted.
A statistical outlier should not automatically be treated as a data error.
# Duplicate IDs
sum(duplicated(data$id))
# Missing values
colSums(is.na(data))
# Basic distributions
summary(data)Summarize missingness:
The missing-data strategy should be prespecified.
Common approaches include:
For multiple imputation, the imputation model should generally include:
When prediction-model validation uses resampling, imputation should ideally be incorporated within the resampling procedure.
Before fitting the final model:
Continuous variables should generally be retained as continuous when possible rather than arbitrarily categorized.
Candidate variables should preferably be selected using:
Variables should not be removed solely because their univariable P-value exceeds 0.05.
Traditional stepwise selection may be used for exploratory purposes but is generally not preferred as the primary model-building strategy because it can produce unstable variable selection and optimistic performance.
When the number of candidate predictors is large relative to the available information, consider:
The statistical model should match the outcome structure.
| Outcome | Common Model |
|---|---|
| Continuous | Linear regression |
| Binary | Logistic regression |
| Count | Poisson / Negative Binomial |
| Time-to-event | Cox proportional hazards model |
| Repeated continuous | Linear mixed model / MMRM |
| Repeated binary | GEE / GLMM |
The primary model should be prespecified whenever possible.
Do not automatically assume that continuous predictors have linear effects.
Assess nonlinear relationships when clinically or statistically plausible.
Common approaches include:
Example:
library(rms)
fit_spline <- lrm(
outcome ~ rcs(age, 4) + rcs(bmi, 4) + sex,
data = model_data
)Avoid arbitrary categorization unless there is a strong clinical reason.
Interactions should be evaluated when there is a prespecified clinical or scientific rationale.
Example:
fit_interaction <- glm(
outcome ~ treatment * sex + age + bmi,
family = binomial(),
data = model_data
)When an interaction is important, interpretation should focus on the treatment or predictor effect within relevant levels of the interacting variable rather than interpreting the main effect alone.
Exploratory interaction testing should be interpreted cautiously.
Assess strong correlation among predictors when relevant.
Possible methods include:
Highly correlated variables may lead to unstable coefficients even when overall prediction remains acceptable.
Fit the prespecified or final candidate model using the selected modeling approach.
Example for a binary outcome:
fit <- glm(
outcome ~ age + bmi + blood_pressure +
cholesterol + sex + smoking,
family = binomial(),
data = model_data
)
summary(fit)Model coefficients, standard errors, confidence intervals, and clinically relevant effect measures should be reported when inference is an objective.
Model diagnostics should be appropriate to the model type.
Common checks include:
Influential observations should be investigated rather than automatically removed.
For prediction models, performance should generally be evaluated across several dimensions.
For binary models, commonly use:
For survival models, commonly use:
Discrimination evaluates how well the model separates individuals with different outcomes.
Assess agreement between predicted and observed outcomes.
Common measures include:
Calibration is essential and should not be replaced by AUC alone.
For binary prediction models, the Brier score may be reported.
A lower Brier score indicates better overall prediction accuracy.
Apparent performance calculated in the same data used to develop the model is usually optimistic.
Internal validation should be performed when the model is intended for prediction.
Preferred methods include:
Bootstrap validation can be used to estimate optimism-corrected:
If variable selection, imputation, or tuning is part of the modeling algorithm, these steps should ideally be repeated within each resampling iteration.
When independent data are available, evaluate the frozen model in:
Assess at least:
If discrimination remains acceptable but calibration has shifted, recalibration may be considered.
When the model is intended to support clinical decisions, evaluate whether predictions provide meaningful clinical benefit.
Decision-curve analysis (DCA) may be used to evaluate net benefit across clinically relevant threshold probabilities.
DCA should not be interpreted simply as an automatic method for choosing a cutoff.
Threshold-based measures such as:
should be reported only when the threshold is clinically justified.
Perform sensitivity analyses that address important modeling assumptions.
Common examples include:
Sensitivity analyses should evaluate whether conclusions are materially changed.
When clinically relevant, evaluate model performance or effects in prespecified subgroups.
Examples include:
For prediction models, subgroup evaluation may include:
Small subgroup results should be interpreted cautiously.
The final report should clearly describe:
For prediction models, recommended performance reporting includes:
| Domain | Common Measure |
|---|---|
| Discrimination | AUC / C-index |
| Calibration | Calibration plot, intercept, slope |
| Overall error | Brier score |
| Clinical utility | Decision-curve analysis |
| Validation | Bootstrap / external validation |
The analysis should be reproducible.
Recommended practices include:
sessionInfo()Research question
↓
Define target population and outcome
↓
Define candidate predictors / covariates
↓
Data quality checks
↓
Missing-data assessment
↓
Descriptive analysis / EDA
↓
Check outliers and multicollinearity
↓
Assess continuous-variable functional forms
↓
Handle missing data
↓
Fit prespecified model
↓
Assess nonlinear terms / interactions
↓
Model diagnostics
↓
Evaluate performance
↓
Internal validation
↓
Sensitivity analyses
↓
External / temporal validation if available
↓
Clinical interpretation and reportingBefore final modeling, confirm: