Background

This is a report focusing on end of life care preferences and their implementation by the clinical care team. In particular, we will focus on the National Quality Forum Measure #1626:

Percentage of vulnerable adults admitted to ICU who survive at least 48 hours who have their care preferences documented within 48 hours OR documentation as to why this was not done.

For our purposeses:

Care preference documentation within 48hrs of admission, age >74.

Care provider/Patient interactions from the MIMIC-III Intensive Care Unit Database are analyzed within the framework of a hospital admission utilizing a series of Mixed Effects Models. The patient is the random effect, so we can explain the difference in variance explained by patients compared to that explained by other effects (when the patient is introduced to other demographic and clinical variables) in the model.

The use of fixed and random effects is deliberate. A fixed effect is used when a categorical variable contains all levels of interest. A random effect is used when a categorical variable contains only a sample of all levels of interest, the samples which do not exist are unobserved variables and, since their distributions do not exist, they will be estimated. The levels of the random effect are assumed to be independent of each other; this assumption requires a priori knowledge of the phenomenon being modeled.

Model 0 (No Covariates)

Care Provider (Random)

Model 1a (Patient Demographics)

Gender
Age
Ethnicity
Marrital Status
Care Provider (Random)

Model 1b (Clinical Variables)

Sequential Organ Failure Assessment (SOFA)
Care Provider (Random)

Model 2a (Demographics + ICU)

Demographics (Model 1a)
First Intensive Care Unit
Care Provider (Random)

Model 2b (Clinical Variables + ICU)

SOFA (Model 1b)
First Intensive Care Unit
Care Provider (Random)

Model 3 (Demographics + Clinical Variables + ICU)

We will use the Sequential Organ Failure Assessment (SOFA) from Illness Severity Scores (github) to determine the severity of illness at admission.

Demographics (as in Model 1)
First Intensive Care Unit
SOFA Score (As Model 2)
Care Provider (Random)

Attending check

attending_check <- function(dat){
    ## Temporary data frame
    tmp_frame <- data.frame()
    ## Results frame
    res <- data.frame()
    ## For each hospital admission
    for (name in unique(dat$HADM_ID)){
        ## Subset admission
        tmp_frame <- dat[dat$HADM_ID == name, ]
        ## If any care providers are "Attending"
        if (any("Attending" %in% tmp_frame$CG_DESCRIPTION)){
            ## add hospital admission to results
            res <- rbind(res, tmp_frame)
        }
    }
    ## Return control to outer level
    return(res)
}

Provider Caremeasure Check

provider_caremeasure_check <- function(dat){
    ## Create Caremeasure Variable
    dat$NQF <- rep(0, each = nrow(dat))
    ## Temporary hospital admission data frame
    tmp_frame <- data.frame()
    ## Temporary careprovider data frame
    cg_frame <- data.frame()
    ## Results frame
    res <- data.frame()
    
    ## For each hospital admission
    for (name in unique(dat$HADM_ID)){
        ## Subset admission
        tmp_frame <- dat[dat$HADM_ID == name, ]
        
        ## For each care provider
        for (id in unique(tmp_frame$CGID)){
            ## Subset provider
            cg_frame <- tmp_frame[tmp_frame$CGID == id, ]
            
            ## if care measure were implemented during the admission by the provider
            if (any(cg_frame$CIM.machine == 1)){
                ## Apply credit for entire admission
                cg_frame$NQF <- rep(1, each = nrow(cg_frame))
            }
            ## Bind
            res <- rbind(res, cg_frame)
        }
    }
    ## Return control to outer level
    return(res)
}

plotDat <- function(dat, column, x_col, bs, mn, xl, yl){
  tmp <- as.matrix(table(dat[[column]], dat[[x_col]]))
  prop <- prop.table(tmp, margin = 2)#2 for column-wise proportions
  par(mar = c(5.0, 4.0, 4.0, 15), xpd = TRUE)
  barplot(prop, col = cm.colors(length(rownames(prop))), beside = bs, width = 2, main = mn, xlab = xl, ylab = yl)
  legend("topright", inset = c(-0.5,0), fill = cm.colors(length(rownames(prop))), legend=rownames(prop))
}

expire_check <- function(dat){
    ## Create Caremeasure Variable
    dat$HOSP_DEATH <- rep(0, each = nrow(dat))
    ## Temporary data frame
    tmp_frame <- data.frame()
    ## Results frame
    res <- data.frame()
    ## For each hospital admission
    for (name in unique(dat$HADM_ID)){
        ## Subset admission
        tmp_frame <- dat[dat$HADM_ID == name, ]
        ## If any admission has a death

            ## Check if pt expired in hospital
            if (any(tmp_frame$HOSPITAL_EXPIRE_FLAG == 1)){
                tmp_frame$HOSP_DEATH <- rep(1, each = nrow(tmp_frame))
            }
            ## add hospital admission to results
            res <- rbind(res, tmp_frame)
        
    }
    ## Return control to outer level
    return(res)
}

rocplot

rocplot <- function(pred, truth, ...) {
  predob = prediction(pred, truth)
  perf = performance(predob, "tpr", "fpr")
  plot(perf, ...)
  area <- auc(truth, pred)
  area <- format(round(area, 4), nsmall = 4)
  text(x=0.7, y=0.1, labels = paste("AUC =", area))

  # the reference x=y line
  segments(x0=0, y0=0, x1=1, y1=1, col="gray", lty=2)
}

model_info <- function(fit){
  #Summary info
  model_sum <- summary(fit)
  #Odds ratio, confidence interval
  odds_ratio <- cbind(OR = exp(fit$coef), exp(confint(fit)))
  #Create list for return
  my_list <- list(model_sum, odds_ratio)
  #names
  names(my_list) <- c("Model Summary","OR Summary")
  return(my_list)
}

Data Loading & Cleansing

## Latest Dataset of NeuroNER Predictions
dat <- read.csv("~/nqf_caregivers/data/20180607_EOL_data_ICU.csv", header = T, stringsAsFactors = F)
dim(dat)

## [1] 10250    57

## Note: Notes had been logged by multiple Care providers, we will reintroduce those annotations

## Load Labeled Note Data for NQF Caremeasure Cohort (From NOTEEVENTS table)
tmp <- read.csv("~/nqf_caregivers/data/note_labels_over75.csv", header = T, stringsAsFactors = F)
dim(tmp)

## [1] 11575    25

## Keep only TEXT and ROW_ID from tmp
tmp <- tmp[ ,c("ROW_ID", "TEXT")]

## Inner join
dat <- merge(tmp, dat, by = "TEXT")

## Clean tmp
rm(tmp)

## Check column names
colnames(dat)

##  [1] "TEXT"                 "ROW_ID.x"             "SUBJECT_ID"          
##  [4] "HADM_ID"              "ROW_ID.y"             "CHARTDATE"           
##  [7] "CHARTTIME"            "STORETIME"            "CATEGORY"            
## [10] "DESCRIPTION"          "CGID"                 "ISERROR"             
## [13] "ADMITTIME"            "DISCHTIME"            "DEATHTIME"           
## [16] "ADMISSION_TYPE"       "ADMISSION_LOCATION"   "DISCHARGE_LOCATION"  
## [19] "INSURANCE"            "LANGUAGE"             "RELIGION"            
## [22] "MARITAL_STATUS"       "ETHNICITY"            "EDREGTIME"           
## [25] "EDOUTTIME"            "DIAGNOSIS"            "HOSPITAL_EXPIRE_FLAG"
## [28] "HAS_CHARTEVENTS_DATA" "GENDER"               "DOB"                 
## [31] "DOD"                  "DOD_HOSP"             "DOD_SSN"             
## [34] "EXPIRE_FLAG"          "ICUSTAY_ID"           "DBSOURCE"            
## [37] "FIRST_CAREUNIT"       "LAST_CAREUNIT"        "FIRST_WARDID"        
## [40] "LAST_WARDID"          "INTIME"               "OUTTIME"             
## [43] "LOS"                  "AGE"                  "ADMISSION_NUMBER"    
## [46] "DAYS_UNTIL_DEATH"     "TIME_SINCE_ADMIT"     "CGID.1"              
## [49] "HADM_ID.1"            "FAM.machine"          "CIM.machine"         
## [52] "LIM.machine"          "CAR.machine"          "COD.machine"         
## [55] "check.CGID"           "check.dadm_id"        "CIM.or.FAM"          
## [58] "Died.in.Hospital"

## What is HADM_ID.1?
head(table(dat$HADM_ID.1))

## 
##   #N/A 100102 100153 100347 100391 100525 
##     32      8      3     12     15      2

## What is HADM_ID?
head(table(dat$HADM_ID))

## 
## 100102 100153 100347 100391 100525 100575 
##      8      3     12     15      2     11

## #N/A? Clean HADM_ID.1
dat$HADM_ID.1 <- NULL

## What is CGID.1?
head(table(dat$CGID.1))

## 
##  #N/A 14010 14022 14037 14045 14056 
##    32    22     1    93    37    14

## What is CGID
head(table(dat$CGID))

## 
## 14010 14022 14037 14045 14056 14080 
##    22     1    93    37    14     6

## #N/A? Clean CGID.1
dat$CGID.1 <- NULL

## What is check.CGID
head(table(dat$check.CGID))

##     0 
## 11575

## Clean it
dat$check.CGID <- NULL

## What is check.dadm_id?
head(table(dat$check.dadm_id))

##     0 
## 11575

## Clean it
dat$check.dadm_id <- NULL

## Clean column names
dat$ROW_ID.y <- NULL

colnames(dat)[which(colnames(dat) == "ROW_ID.x")] <- "ROW_ID"


## Load CAREGIVERS Table for join on CGID
cg <- read.csv("~/nqf_caregivers/data/mimic/CAREGIVERS.csv", 
               header = T, stringsAsFactors = F)

## Change column name of "NOTEEVENTS.DESCRIPTION" to explicitly mention that it describes the note
colnames(dat)[which(colnames(dat) == "DESCRIPTION")] <- "NOTE_DESCRIPTION"

## Change column name of "CAREGIVERS. DESCRIPTION" to explicitly mention that it describes the careprovider
colnames(cg)[which(colnames(cg) == "DESCRIPTION")] <- "CG_DESCRIPTION"

## Remove ROW_ID from CG
cg$ROW_ID <- NULL

## Remove TEXT
dat$TEXT <- NULL

## Merge to caregivers
dat <- merge(dat, cg, by = "CGID")
dim(dat)

## [1] 11575    54

## Clean CG
rm(cg)

sofa <- read.csv("~/nqf_caregivers/data/sofa.csv", header = T, stringsAsFactors = F)
#oasis <- read.csv("~/nqf_caregivers/data/oasis.csv", header = T, stringsAsFactors = F)
#saps <- read.csv("~/nqf_caregivers/data/saps.csv", header = T, stringsAsFactors = F)

colnames(sofa) <- toupper(colnames(sofa))

dat <- merge(dat, sofa, by = c("SUBJECT_ID", "HADM_ID", "ICUSTAY_ID"))
dim(dat)

## [1] 11575    61

## Clean environment
rm(sofa)

## Clean ethnicity to Black/White/Other
dat[(grepl("WHITE|PORTUGUESE", dat$ETHNICITY)),]$ETHNICITY <- "WHITE" 
dat[(grepl("ASIAN", dat$ETHNICITY)),]$ETHNICITY <- "OTHER" 
dat[(grepl("BLACK", dat$ETHNICITY)),]$ETHNICITY <- "BLACK" 
dat[(grepl("HISPANIC", dat$ETHNICITY)),]$ETHNICITY <- "OTHER"
dat[(grepl("MIDDLE|NATIVE|MULTI|DECLINED|UNABLE|OTHER|NOT", dat$ETHNICITY)),]$ETHNICITY <- "OTHER"

## Clean Marital Status to Married, Single, Widowed, Unknown
dat$MARITAL_STATUS[dat$MARITAL_STATUS == ""] <- "UNKNOWN (DEFAULT)"
dat$MARITAL_STATUS[dat$MARITAL_STATUS == "UNKNOWN (DEFAULT)"] <- "UNKNOWN"
dat$MARITAL_STATUS[dat$MARITAL_STATUS == "SEPARATED"] <- "SINGLE"
dat$MARITAL_STATUS[dat$MARITAL_STATUS == "DIVORCED"] <- "SINGLE"

## Clean admission data
dat$ADMISSION_TYPE[dat$ADMISSION_TYPE == "URGENT"] <- "EMERGENCY"

Provider-Level Analysis

At this level, we will see if providers documented patient care preferences at least once in the first 48hrs of the hospital admission. Here, the clinician/patient interaction is the unit of analysis.

temp <- provider_caremeasure_check(attending_check(dat))

temp <- expire_check(temp)

Data Exploration

plotDat(temp, "NQF", "GENDER", F, "Gender (Provider-Level)", "Gender", "Frequency")

test <- table(temp$GENDER, temp$NQF)
test

##    
##        0    1
##   F 2187 3344
##   M 2768 2860

chisq.test(test)

## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  test
## X-squared = 104.66, df = 1, p-value < 2.2e-16

pairwiseNominalIndependence(
  as.matrix(test), 
  fisher = F, gtest = F, chisq = T, method = "fdr")

##   Comparison  p.Chisq p.adj.Chisq
## 1      F : M 1.45e-24    1.45e-24

plotDat(temp, "NQF", "ADMISSION_TYPE", F, "Admission Type (Provider-Level)", "Admission Type", "Frequency")

test <- table(temp$ADMISSION_TYPE, temp$NQF)
test

##            
##                0    1
##   ELECTIVE   294   28
##   EMERGENCY 4661 6176

chisq.test(test)

## 
##  Pearson's Chi-squared test with Yates' continuity correction
## 
## data:  test
## X-squared = 293.48, df = 1, p-value < 2.2e-16

pairwiseNominalIndependence(
  as.matrix(test), 
  fisher = F, gtest = F, chisq = T, method = "fdr")

##             Comparison  p.Chisq p.adj.Chisq
## 1 ELECTIVE : EMERGENCY 8.65e-66    8.65e-66

plotDat(temp, "NQF","ETHNICITY", F, "Ethnicity (Provider-Level)", "Ethnicity", "Frequency")

test <- table(temp$ETHNICITY, temp$NQF)
test

##        
##            0    1
##   BLACK  405  619
##   OTHER  475  636
##   WHITE 4075 4949

chisq.test(test)

## 
##  Pearson's Chi-squared test
## 
## data:  test
## X-squared = 13.069, df = 2, p-value = 0.001452

pairwiseNominalIndependence(
  as.matrix(test), 
  fisher = F, gtest = F, chisq = T, method = "fdr")

##      Comparison  p.Chisq p.adj.Chisq
## 1 BLACK : OTHER 0.145000     0.14500
## 2 BLACK : WHITE 0.000706     0.00212
## 3 OTHER : WHITE 0.137000     0.14500

plotDat(temp, "NQF", "MARITAL_STATUS", F, "Marital Status (Provider-Level)", "Marital Status", "Frequency")

test <- table(temp$MARITAL_STATUS, temp$NQF)
test

##          
##              0    1
##   MARRIED 2315 2343
##   SINGLE   884 1469
##   UNKNOWN  227  187
##   WIDOWED 1529 2205

chisq.test(test)

## 
##  Pearson's Chi-squared test
## 
## data:  test
## X-squared = 133.74, df = 3, p-value < 2.2e-16

pairwiseNominalIndependence(
  as.matrix(test), 
  fisher = F, gtest = F, chisq = T, method = "fdr")

##          Comparison  p.Chisq p.adj.Chisq
## 1  MARRIED : SINGLE 7.70e-22    4.62e-21
## 2 MARRIED : UNKNOWN 5.12e-02    5.12e-02
## 3 MARRIED : WIDOWED 1.53e-15    4.59e-15
## 4  SINGLE : UNKNOWN 5.65e-11    1.13e-10
## 5  SINGLE : WIDOWED 9.39e-03    1.13e-02
## 6 UNKNOWN : WIDOWED 7.79e-08    1.17e-07

x <- barplot(table(mtcars$cyl), xaxt="n")
labs <- paste(names(table(mtcars$cyl)), "cylinders")
text(cex=1, x=x-.25, y=-1.25, labs, xpd=TRUE, srt=45)

plotDat(temp, "NQF", "FIRST_CAREUNIT", F, "First Careunit (Provider-Level)", "First Careunit", "Frequency")

test <- table(temp$FIRST_CAREUNIT, temp$NQF)
test

##        
##            0    1
##   CCU    696  905
##   CSRU   273  111
##   MICU  3031 4312
##   SICU   593  546
##   TSICU  362  330

chisq.test(test)

## 
##  Pearson's Chi-squared test
## 
## data:  test
## X-squared = 185.04, df = 4, p-value < 2.2e-16

pairwiseNominalIndependence(
  as.matrix(test), 
  fisher = F, gtest = F, chisq = T, method = "fdr")

##      Comparison  p.Chisq p.adj.Chisq
## 1    CCU : CSRU 4.13e-22    2.06e-21
## 2    CCU : MICU 1.13e-01    1.26e-01
## 3    CCU : SICU 1.08e-05    1.54e-05
## 4   CCU : TSICU 1.17e-04    1.46e-04
## 5   CSRU : MICU 2.10e-30    2.10e-29
## 6   CSRU : SICU 1.10e-10    2.75e-10
## 7  CSRU : TSICU 2.91e-09    5.82e-09
## 8   MICU : SICU 9.46e-12    3.15e-11
## 9  MICU : TSICU 2.43e-08    4.05e-08
## 10 SICU : TSICU 9.56e-01    9.56e-01

boxplot(temp$AGE ~ temp$NQF, 
        main = "Caremeasure Implementation\n by Age (Provider-Level)",
        xlab = "Implementation (1 == Yes)",
        ylab = "Age (Years)")

t.test(temp$AGE ~ temp$NQF)

## 
##  Welch Two Sample t-test
## 
## data:  temp$AGE by temp$NQF
## t = -12.58, df = 10825, p-value < 2.2e-16
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -1.425367 -1.041058
## sample estimates:
## mean in group 0 mean in group 1 
##        83.40638        84.63959

boxplot(temp$SOFA ~ temp$NQF, 
        main = "Caremeasure Implementation\n by SOFA Score (Provider-Level)",
        xlab = "Implementation (1 == Yes)",
        ylab = "SOFA Score")

t.test(temp$SOFA ~ temp$NQF)

## 
##  Welch Two Sample t-test
## 
## data:  temp$SOFA by temp$NQF
## t = 0.11386, df = 10337, p-value = 0.9094
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.1062057  0.1193046
## sample estimates:
## mean in group 0 mean in group 1 
##        5.037336        5.030787

Data Standardization and Factoring (for modeling)

Continuous data, Age and SOFA, will be standardized in the form:

\[z = \frac{x - \mu}{\sigma}\]

temp$AGE <- (temp$AGE - mean(temp$AGE))/sd(temp$AGE)
temp$SOFA <- (temp$SOFA - mean(temp$SOFA)/sd(temp$SOFA))

Baseline Model (Provider Random Effects)

gm_initial <- glmer(NQF ~ (1 | CGID),
                   data = temp, 
                   family = binomial, 
                   control = glmerControl(optimizer = "Nelder_Mead"),
                   nAGQ = 10)

## View
summary(gm_initial)

## Generalized linear mixed model fit by maximum likelihood (Adaptive
##   Gauss-Hermite Quadrature, nAGQ = 10) [glmerMod]
##  Family: binomial  ( logit )
## Formula: NQF ~ (1 | CGID)
##    Data: temp
## Control: glmerControl(optimizer = "Nelder_Mead")
## 
##      AIC      BIC   logLik deviance df.resid 
##  13715.3  13730.0  -6855.7  13711.3    11157 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -5.2922 -0.8253  0.3867  0.7561  3.6611 
## 
## Random effects:
##  Groups Name        Variance Std.Dev.
##  CGID   (Intercept) 2.138    1.462   
## Number of obs: 11159, groups:  CGID, 493
## 
## Fixed effects:
##             Estimate Std. Error z value Pr(>|z|)  
## (Intercept)  0.16423    0.07617   2.156   0.0311 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

sjstats::icc(gm_initial)

## 
## Generalized linear mixed model
##  Family: binomial (logit)
## Formula: NQF ~ (1 | CGID)
## 
##   ICC (CGID): 0.393845

Model 1a (Patient Demographics)

Gender
Age
Ethnicity
Marrital Status
Care Provider (Random)

gm_a <- glmer(NQF ~ GENDER +
                     AGE +
                     ETHNICITY +
                     MARITAL_STATUS +
                     (1 | CGID), 
                   data = temp, 
                   family = binomial, 
                   control = glmerControl(optimizer = "Nelder_Mead"),
                   nAGQ = 10)


## View
summary(gm_a)

## Generalized linear mixed model fit by maximum likelihood (Adaptive
##   Gauss-Hermite Quadrature, nAGQ = 10) [glmerMod]
##  Family: binomial  ( logit )
## Formula: NQF ~ GENDER + AGE + ETHNICITY + MARITAL_STATUS + (1 | CGID)
##    Data: temp
## Control: glmerControl(optimizer = "Nelder_Mead")
## 
##      AIC      BIC   logLik deviance df.resid 
##  13472.3  13538.1  -6727.1  13454.3    11150 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -5.4389 -0.7877  0.3476  0.7307  4.1859 
## 
## Random effects:
##  Groups Name        Variance Std.Dev.
##  CGID   (Intercept) 2.112    1.453   
## Number of obs: 11159, groups:  CGID, 493
## 
## Fixed effects:
##                       Estimate Std. Error z value Pr(>|z|)    
## (Intercept)            0.22100    0.11608   1.904  0.05692 .  
## GENDERM               -0.33510    0.04934  -6.792 1.11e-11 ***
## AGE                    0.25396    0.02348  10.817  < 2e-16 ***
## ETHNICITYOTHER         0.03975    0.10762   0.369  0.71187    
## ETHNICITYWHITE        -0.02417    0.08113  -0.298  0.76581    
## MARITAL_STATUSSINGLE   0.38954    0.06319   6.164 7.08e-10 ***
## MARITAL_STATUSUNKNOWN -0.14375    0.12465  -1.153  0.24882    
## MARITAL_STATUSWIDOWED  0.17955    0.05818   3.086  0.00203 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##                 (Intr) GENDER AGE    ETHNICITYO ETHNICITYW MARITAL_STATUSS
## GENDERM         -0.266                                                    
## AGE              0.089 -0.054                                             
## ETHNICITYOT     -0.495 -0.037 -0.051                                      
## ETHNICITYWH     -0.647 -0.064 -0.064  0.675                               
## MARITAL_STATUSS -0.302  0.248  0.020  0.093      0.095                    
## MARITAL_STATUSU -0.130  0.066 -0.077 -0.068      0.048      0.161         
## MARITAL_STATUSW -0.369  0.368 -0.177  0.101      0.096      0.432         
##                 MARITAL_STATUSU
## GENDERM                        
## AGE                            
## ETHNICITYOT                    
## ETHNICITYWH                    
## MARITAL_STATUSS                
## MARITAL_STATUSU                
## MARITAL_STATUSW  0.205

sjt.glmer(gm_a)

	NQF
	Odds Ratio	CI	p
Fixed Parts
(Intercept)	1.25	0.99 – 1.57	.057
GENDER (M)	0.72	0.65 – 0.79	<.001
AGE	1.29	1.23 – 1.35	<.001
ETHNICITY (OTHER)	1.04	0.84 – 1.28	.712
ETHNICITY (WHITE)	0.98	0.83 – 1.14	.766
MARITAL_STATUS (SINGLE)	1.48	1.30 – 1.67	<.001
MARITAL_STATUS (UNKNOWN)	0.87	0.68 – 1.11	.249
MARITAL_STATUS (WIDOWED)	1.20	1.07 – 1.34	.002
Random Parts
τ_{00, CGID}	2.112
N_CGID	493
ICC_CGID	0.391
Observations	11159
Deviance	12244.948

Model 1b (Clinical Variables)

SOFA
Care Provider (Random)

gm_b <- glmer(NQF ~ SOFA +
                     (1 | CGID), 
                   data = temp, 
                   family = binomial, 
                   control = glmerControl(optimizer = "Nelder_Mead"),
                   nAGQ = 10)

## View
summary(gm_b)

## Generalized linear mixed model fit by maximum likelihood (Adaptive
##   Gauss-Hermite Quadrature, nAGQ = 10) [glmerMod]
##  Family: binomial  ( logit )
## Formula: NQF ~ SOFA + (1 | CGID)
##    Data: temp
## Control: glmerControl(optimizer = "Nelder_Mead")
## 
##      AIC      BIC   logLik deviance df.resid 
##  13717.3  13739.3  -6855.7  13711.3    11156 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -5.2906 -0.8251  0.3866  0.7564  3.6627 
## 
## Random effects:
##  Groups Name        Variance Std.Dev.
##  CGID   (Intercept) 2.138    1.462   
## Number of obs: 11159, groups:  CGID, 493
## 
## Fixed effects:
##              Estimate Std. Error z value Pr(>|z|)  
## (Intercept) 0.1633135  0.0801374   2.038   0.0416 *
## SOFA        0.0002792  0.0075712   0.037   0.9706  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##      (Intr)
## SOFA -0.311

sjt.glmer(gm_b)

	NQF
	Odds Ratio	CI	p
Fixed Parts
(Intercept)	1.18	1.01 – 1.38	.042
SOFA	1.00	0.99 – 1.02	.971
Random Parts
τ_{00, CGID}	2.138
N_CGID	493
ICC_CGID	0.394
Observations	11159
Deviance	12489.153

Model 2a (Demographics + ICU)

Demographics (Model 1a)
First Intensive Care Unit
Care Provider (Random)

gm_2a <- glmer(NQF ~ GENDER +
                      AGE +
                      ETHNICITY +
                      MARITAL_STATUS +
                      FIRST_CAREUNIT +
                      (1 | CGID), 
                   data = temp,
                   family = binomial, 
                   control = glmerControl(optimizer = "Nelder_Mead"),
                   nAGQ = 10)

## View
summary(gm_2a)

## Generalized linear mixed model fit by maximum likelihood (Adaptive
##   Gauss-Hermite Quadrature, nAGQ = 10) [glmerMod]
##  Family: binomial  ( logit )
## Formula: 
## NQF ~ GENDER + AGE + ETHNICITY + MARITAL_STATUS + FIRST_CAREUNIT +  
##     (1 | CGID)
##    Data: temp
## Control: glmerControl(optimizer = "Nelder_Mead")
## 
##      AIC      BIC   logLik deviance df.resid 
##  13425.1  13520.2  -6699.5  13399.1    11146 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -5.3982 -0.7883  0.3440  0.7329  4.7918 
## 
## Random effects:
##  Groups Name        Variance Std.Dev.
##  CGID   (Intercept) 2.184    1.478   
## Number of obs: 11159, groups:  CGID, 493
## 
## Fixed effects:
##                        Estimate Std. Error z value Pr(>|z|)    
## (Intercept)           -0.089426   0.136183  -0.657 0.511397    
## GENDERM               -0.317127   0.049573  -6.397 1.58e-10 ***
## AGE                    0.259159   0.023635  10.965  < 2e-16 ***
## ETHNICITYOTHER         0.084333   0.108249   0.779 0.435941    
## ETHNICITYWHITE         0.006923   0.081463   0.085 0.932275    
## MARITAL_STATUSSINGLE   0.402554   0.063371   6.352 2.12e-10 ***
## MARITAL_STATUSUNKNOWN -0.189594   0.125578  -1.510 0.131102    
## MARITAL_STATUSWIDOWED  0.199190   0.058503   3.405 0.000662 ***
## FIRST_CAREUNITCSRU    -0.453720   0.171431  -2.647 0.008129 ** 
## FIRST_CAREUNITMICU     0.301164   0.078860   3.819 0.000134 ***
## FIRST_CAREUNITSICU     0.347701   0.116786   2.977 0.002909 ** 
## FIRST_CAREUNITTSICU    0.765025   0.142851   5.355 8.54e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##                 (Intr) GENDER AGE    ETHNICITYO ETHNICITYW MARITAL_STATUSS
## GENDERM         -0.242                                                    
## AGE              0.044 -0.050                                             
## ETHNICITYOT     -0.455 -0.033 -0.045                                      
## ETHNICITYWH     -0.577 -0.061 -0.062  0.673                               
## MARITAL_STATUSS -0.272  0.246  0.022  0.096      0.097                    
## MARITAL_STATUSU -0.083  0.064 -0.078 -0.069      0.043      0.158         
## MARITAL_STATUSW -0.310  0.367 -0.176  0.103      0.098      0.431         
## FIRST_CAREUNITC -0.205 -0.031  0.002  0.011      0.014      0.000         
## FIRST_CAREUNITM -0.475  0.031  0.083  0.070      0.035      0.027         
## FIRST_CAREUNITS -0.386  0.033  0.014  0.026      0.043      0.015         
## FIRST_CAREUNITT -0.359  0.009  0.011  0.045      0.066      0.027         
##                 MARITAL_STATUSU MARITAL_STATUSW FIRST_CAREUNITC
## GENDERM                                                        
## AGE                                                            
## ETHNICITYOT                                                    
## ETHNICITYWH                                                    
## MARITAL_STATUSS                                                
## MARITAL_STATUSU                                                
## MARITAL_STATUSW  0.202                                         
## FIRST_CAREUNITC -0.031          -0.049                         
## FIRST_CAREUNITM -0.033          -0.010           0.300         
## FIRST_CAREUNITS -0.041          -0.018           0.290         
## FIRST_CAREUNITT -0.076           0.021           0.266         
##                 FIRST_CAREUNITM FIRST_CAREUNITS
## GENDERM                                        
## AGE                                            
## ETHNICITYOT                                    
## ETHNICITYWH                                    
## MARITAL_STATUSS                                
## MARITAL_STATUSU                                
## MARITAL_STATUSW                                
## FIRST_CAREUNITC                                
## FIRST_CAREUNITM                                
## FIRST_CAREUNITS  0.540                         
## FIRST_CAREUNITT  0.441           0.556         
## convergence code: 0
## Model failed to converge with max|grad| = 0.00191109 (tol = 0.001, component 1)

sjt.glmer(gm_2a)

	NQF
	Odds Ratio	CI	p
Fixed Parts
(Intercept)	0.91	0.70 – 1.19	.511
GENDER (M)	0.73	0.66 – 0.80	<.001
AGE	1.30	1.24 – 1.36	<.001
ETHNICITY (OTHER)	1.09	0.88 – 1.35	.436
ETHNICITY (WHITE)	1.01	0.86 – 1.18	.932
MARITAL_STATUS (SINGLE)	1.50	1.32 – 1.69	<.001
MARITAL_STATUS (UNKNOWN)	0.83	0.65 – 1.06	.131
MARITAL_STATUS (WIDOWED)	1.22	1.09 – 1.37	<.001
FIRST_CAREUNIT (CSRU)	0.64	0.45 – 0.89	.008
FIRST_CAREUNIT (MICU)	1.35	1.16 – 1.58	<.001
FIRST_CAREUNIT (SICU)	1.42	1.13 – 1.78	.003
FIRST_CAREUNIT (TSICU)	2.15	1.62 – 2.84	<.001
Random Parts
τ_{00, CGID}	2.184
N_CGID	493
ICC_CGID	0.399
Observations	11159
Deviance	12180.256

Model 2b (Clinical Variables + ICU)

Clinical Variables (Model 1b)
First Intensive Care Unit
Care Provider (Random)

gm_2b <- glmer(NQF ~ SOFA +
                  FIRST_CAREUNIT +
                      (1 | CGID), 
                   data = temp,
                   family = binomial, 
                   control = glmerControl(optimizer = "Nelder_Mead"),
                   nAGQ = 10)

## View
summary(gm_2b)

## Generalized linear mixed model fit by maximum likelihood (Adaptive
##   Gauss-Hermite Quadrature, nAGQ = 10) [glmerMod]
##  Family: binomial  ( logit )
## Formula: NQF ~ SOFA + FIRST_CAREUNIT + (1 | CGID)
##    Data: temp
## Control: glmerControl(optimizer = "Nelder_Mead")
## 
##      AIC      BIC   logLik deviance df.resid 
##  13673.1  13724.3  -6829.5  13659.1    11152 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -5.3161 -0.8161  0.3780  0.7567  4.2014 
## 
## Random effects:
##  Groups Name        Variance Std.Dev.
##  CGID   (Intercept) 2.243    1.498   
## Number of obs: 11159, groups:  CGID, 493
## 
## Fixed effects:
##                      Estimate Std. Error z value Pr(>|z|)    
## (Intercept)         -0.077708   0.103960  -0.747 0.454772    
## SOFA                 0.003185   0.007625   0.418 0.676136    
## FIRST_CAREUNITCSRU  -0.466884   0.168643  -2.768 0.005632 ** 
## FIRST_CAREUNITMICU   0.241640   0.077318   3.125 0.001776 ** 
## FIRST_CAREUNITSICU   0.382924   0.115541   3.314 0.000919 ***
## FIRST_CAREUNITTSICU  0.734986   0.140761   5.222 1.77e-07 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##                 (Intr) SOFA   FIRST_CAREUNITC FIRST_CAREUNITM
## SOFA            -0.245                                       
## FIRST_CAREUNITC -0.270 -0.020                                
## FIRST_CAREUNITM -0.570 -0.018  0.302                         
## FIRST_CAREUNITS -0.474  0.051  0.290           0.536         
## FIRST_CAREUNITT -0.421  0.047  0.268           0.438         
##                 FIRST_CAREUNITS
## SOFA                           
## FIRST_CAREUNITC                
## FIRST_CAREUNITM                
## FIRST_CAREUNITS                
## FIRST_CAREUNITT  0.557

sjt.glmer(gm_2b)

	NQF
	Odds Ratio	CI	p
Fixed Parts
(Intercept)	0.93	0.75 – 1.13	.455
SOFA	1.00	0.99 – 1.02	.676
FIRST_CAREUNIT (CSRU)	0.63	0.45 – 0.87	.006
FIRST_CAREUNIT (MICU)	1.27	1.09 – 1.48	.002
FIRST_CAREUNIT (SICU)	1.47	1.17 – 1.84	<.001
FIRST_CAREUNIT (TSICU)	2.09	1.58 – 2.75	<.001
Random Parts
τ_{00, CGID}	2.243
N_CGID	493
ICC_CGID	0.405
Observations	11159
Deviance	12421.574

Model 3 (Demographics + Clinical Variables + ICU)

Demographics (as in Model 1a)
SOFA Score (As Model 1b)
First Intensive Care Unit
Care Provider (Random)

gm_icu <- glmer(NQF ~ 
                   GENDER +
                   AGE +
                   ETHNICITY +
                   MARITAL_STATUS +
                   SOFA +
                   FIRST_CAREUNIT +
                   (1 | CGID), 
                   data = temp,
                   family = binomial, 
                   control = glmerControl(optimizer = "Nelder_Mead"),
                   nAGQ = 10)

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
## control$checkConv, : Model failed to converge with max|grad| = 0.00109964
## (tol = 0.001, component 1)

## View
summary(gm_icu)

## Generalized linear mixed model fit by maximum likelihood (Adaptive
##   Gauss-Hermite Quadrature, nAGQ = 10) [glmerMod]
##  Family: binomial  ( logit )
## Formula: 
## NQF ~ GENDER + AGE + ETHNICITY + MARITAL_STATUS + SOFA + FIRST_CAREUNIT +  
##     (1 | CGID)
##    Data: temp
## Control: glmerControl(optimizer = "Nelder_Mead")
## 
##      AIC      BIC   logLik deviance df.resid 
##  13420.5  13523.0  -6696.3  13392.5    11145 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -5.2806 -0.7926  0.3433  0.7313  4.9852 
## 
## Random effects:
##  Groups Name        Variance Std.Dev.
##  CGID   (Intercept) 2.194    1.481   
## Number of obs: 11159, groups:  CGID, 493
## 
## Fixed effects:
##                        Estimate Std. Error z value Pr(>|z|)    
## (Intercept)           -0.158078   0.138989  -1.137 0.255395    
## GENDERM               -0.331301   0.049905  -6.639 3.17e-11 ***
## AGE                    0.263073   0.023705  11.098  < 2e-16 ***
## ETHNICITYOTHER         0.080328   0.108411   0.741 0.458715    
## ETHNICITYWHITE         0.014372   0.081614   0.176 0.860218    
## MARITAL_STATUSSINGLE   0.408109   0.063430   6.434 1.24e-10 ***
## MARITAL_STATUSUNKNOWN -0.179789   0.125682  -1.431 0.152572    
## MARITAL_STATUSWIDOWED  0.203482   0.058556   3.475 0.000511 ***
## SOFA                   0.020115   0.007848   2.563 0.010378 *  
## FIRST_CAREUNITCSRU    -0.462581   0.171437  -2.698 0.006970 ** 
## FIRST_CAREUNITMICU     0.297690   0.078919   3.772 0.000162 ***
## FIRST_CAREUNITSICU     0.362618   0.117035   3.098 0.001946 ** 
## FIRST_CAREUNITTSICU    0.783048   0.143071   5.473 4.42e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## 
## Correlation matrix not shown by default, as p = 13 > 12.
## Use print(x, correlation=TRUE)  or
##   vcov(x)     if you need it

## convergence code: 0
## Model failed to converge with max|grad| = 0.00109964 (tol = 0.001, component 1)

sjt.glmer(gm_icu)

	NQF
	Odds Ratio	CI	p
Fixed Parts
(Intercept)	0.85	0.65 – 1.12	.255
GENDER (M)	0.72	0.65 – 0.79	<.001
AGE	1.30	1.24 – 1.36	<.001
ETHNICITY (OTHER)	1.08	0.88 – 1.34	.459
ETHNICITY (WHITE)	1.01	0.86 – 1.19	.860
MARITAL_STATUS (SINGLE)	1.50	1.33 – 1.70	<.001
MARITAL_STATUS (UNKNOWN)	0.84	0.65 – 1.07	.153
MARITAL_STATUS (WIDOWED)	1.23	1.09 – 1.37	<.001
SOFA	1.02	1.00 – 1.04	.010
FIRST_CAREUNIT (CSRU)	0.63	0.45 – 0.88	.007
FIRST_CAREUNIT (MICU)	1.35	1.15 – 1.57	<.001
FIRST_CAREUNIT (SICU)	1.44	1.14 – 1.81	.002
FIRST_CAREUNIT (TSICU)	2.19	1.65 – 2.90	<.001
Random Parts
τ_{00, CGID}	2.194
N_CGID	493
ICC_CGID	0.400
Observations	11159
Deviance	12172.161

par(mfrow=c(2,3))

rocplot(as.numeric(predict(gm_initial, type="response")), temp$NQF, col="blue", main = "Initial Model")
rocplot(as.numeric(predict(gm_a, type="response")), temp$NQF, col="blue", main = "Model 1a (Demographics)")
rocplot(as.numeric(predict(gm_b, type="response")), temp$NQF, col="blue", main = "Model 1b (Clinical Variables)")
rocplot(as.numeric(predict(gm_2a, type="response")), temp$NQF, col="blue", main = "Model 2a (Demographics + ICU)")
rocplot(as.numeric(predict(gm_2b, type="response")), temp$NQF, col="blue", main = "Model 2b (Clinical Variables + ICU)")
rocplot(as.numeric(predict(gm_icu, type="response")), temp$NQF, col="blue", main = "Model 3 (All Covariates)")

Provider NQF Analysis

Background

Model 0 (No Covariates)

Model 1a (Patient Demographics)

Model 1b (Clinical Variables)

Model 2a (Demographics + ICU)

Model 2b (Clinical Variables + ICU)

Model 3 (Demographics + Clinical Variables + ICU)

Attending check

Provider Caremeasure Check

rocplot

Data Loading & Cleansing

Provider-Level Analysis

Data Exploration

Data Standardization and Factoring (for modeling)

Baseline Model (Provider Random Effects)

Model 1a (Patient Demographics)

Model 1b (Clinical Variables)

Model 2a (Demographics + ICU)

Model 2b (Clinical Variables + ICU)

Model 3 (Demographics + Clinical Variables + ICU)