Phenotype Evaluation with CohortDiagnostics

OHDSI SOS Challenge 2023

Jamie Gilbert

2023-04-18

Introduction

This tutorial has the following objectives:

  • Familiarize users with executing studies in CohortDiagnostics package
  • Explain iterative Phenotype evaluation
  • Provide links to resources and people
  • Code heavy but Phenotype Evaluation is iterative
  • A future SOS tutorial will focus on “just run this study” with Strategus

CohortDiagnostics overview

  • Developing Good Phenotypes is the most critical aspect of an observational study
  • The criteria for objectively determining if a phenotype is fit for purpose is left to investigators
  • Few (if any) standard tools exist to support this inevitably leads to bias
  • Solution: we developed CohortDiagnostics package
  • Phenotype Algorithms are never finished… (iterative process)

R Setup

  • Tutorial assumes some familiarity with R
  • Follow our R HADES setup guide for getting an R environment set up Here
  • We do not expect you to share results from this process
  • Almost all code blocks can be copy pasted

Find this guide:

https://rpubs.com/azimov/ohdsi_sos_2023_cd

Setting up study environment:

For this tutorial you will need:

install.packages("remotes")
remotes::install_github("OHDSI/CohortGenerator")
remotes::install_github("OHDSI/CohortDiagnostics")
remotes::install_github("OHDSI/ROhdsiWebApi")
remotes::install_github("OHDSI/OhdsiShinyModules")

This method ensures that you have the same package environment as a study author

install.packages("renv")

# Periodic stable releases of this file for hades wide work
download.file("https://raw.githubusercontent.com/OHDSI/Hades/main/hadesWideReleases/2023Mar30/renv.lock", 
              destfile = "hades_renv.lock")
renv::restore(lockfile =  = "hades_renv.lock")

Getting Study Cohort Definitions

First, go to Atlas and find your cohorts of interest.

We need the identifiers:

Demo atlas - view of cohort definitions

Here

ROhdsiWebApi usage

Use the ROhdsiWebApi package to download our cohorts by ID

baseUrl <- "https://atlas-demo.ohdsi.org/WebAPI"

atlasCohortIds <- c(1782164, 1782483, 1782482, 1782481)

cohortDefinitionSet <- ROhdsiWebApi::exportCohortDefinitionSet(
  baseUrl = baseUrl,
  cohortIds = atlasCohortIds,
  generateStats = TRUE
)

If you need to save your cohorts for offline use, use CohortGenerator

# Saving
CohortGenerator::saveCohortDefinitionSet(cohortDefinitionSet)

# Loading
cohortDefinitionSet <- CohortGenerator::getCohortDefinitionSet()

For non-public servers you will need to login.

This will require some communication with the atlas admin if regarding what security you’re using

ROhdsiWebApi::authorizeWebApi(baseUrl, authMethod = "windows", ...)

View your cohorts before execution!

View(cohortDefinitionSet)
Cohorts
cohortId cohortName
1782164 [SOS] End-stage renal disease
1782483 [SOS Phenotype Devt] aflibercept exposures after new use with 3 exposures in 21-70d windows
1782482 [SOS Phenotype Devt] bevacizumab exposures after new use with 3 exposures in 21-70d windows

…

Connection settings

connectionDetails <- DatabaseConnector::createConnectionDetails(
    dbms = "postgresql",
    server = "path_to_server/database",
    port = 5432,
    # We reccomend using the keyring package for secure credentials
    user = "my_user", 
    password = "blah"
)

databaseId <- "my_cdm_id"
cdmDatabaseSchema <- "mv_cdm_version_5_4"
cohortDatabaseSchema <- "where_results_live"

dataFolder <- "results"
incrementalFolder <- paste("incremental_", databaseId)

We strongly recommend using the keyring package

install.packages("keyring")
keyring::key_set("cdm-username") # Will prompt with secure input
keyring::key_set("cdm-password")
keyring::key_set("cdm-server")

# with DatabaseConnector we never want to save these passwords 
#in plain text form
connectionDetails <- DatabaseConnector::createConnectionDetails(
    dbms = "postgresql",
    port = 5432,
    server = keyring::key_set("cdm-server"),
    # We reccomend using the keyring package for secure credentials
    user = keyring::key_set("cdm-username"), 
    password = keyring::key_set("cdm-password")
)

Generating Cohorts

Call the cohort generator package

library(CohortGenerator)

cohortTableNames <- getCohortTableNames(cohortTable = "cohort")

generateCohortSet(
  connectionDetails = connectionDetails,
  cohortDefinitionSet = cohortDefinitionSet,
  cohortTableNames = cohortTableNames,
  incremental = TRUE,
  incrementalFolder = incrementalFolder
)

Executing Cohort Diagnostics

This can take a long time to run…

library(CohortDiagnostics)
executeDiagnostics(
  cohortDefinitionSet = cohortDefinitionSet,
  connectionDetails = connectionDetails,
  cdmDatabaseSchema = cdmDatabaseSchema,
  cohortDatabaseSchema = cohortDatabaseSchema,
  cohortTableNames = cohortTableNames,
  exportFolder = file.path(dataFolder,databaseId),
  databaseId = databaseId,
  incremental = TRUE,
  incrementalFolder = incrementalFolder,
  minCellCount = 5,
  runInclusionStatistics = TRUE,
  runIncludedSourceConcepts = TRUE,
  runOrphanConcepts = TRUE,
  runTimeSeries = TRUE,
  runVisitContext = TRUE,
  runBreakdownIndexEvents = TRUE,
  runIncidenceRate = TRUE,
  runCohortRelationship = TRUE,
  runTemporalCohortCharacterization = TRUE
  )

Viewing the Shiny App Results

Create an sqlite database:

# This is the same for any number of CDMs - will create file "MyStudyDatabase.sqlite"
createMergedResultsFile(
  dataFolder = dataFolder, 
  sqliteDbPath = "SOS_STUDY_2023.sqlite"
)

Launch the shiny app:

launchDiagnosticsExplorer(sqliteDbPath = "SOS_STUDY_2023.sqlite")

Resources

  • HADES setup guide https://ohdsi.github.io/Hades/rSetup.html
  • Github https://github.com/OHDSI/CohortDiagnostics
  • Documentation https://ohdsi.github.io/CohortDiagnostics/
  • OHDSI Teams
  • Chat OHDSI Forum
  • Github Issue tracker

Appendix: Sharing results

The files that should be of interest are:

<workspace_path>results/my_cdm_id/Results_my_cdm_id.zip

This requires pushing the shiny app to a new folder on

ShinyDeploy

Press the “publish” button in the top right and share to shinyapps.io

launchDiagnosticsExplorer(
  sqliteDbPath = "SOS_STUDY_2023.sqlite",
  makePublishable = TRUE,
  publishDir = tempdir(),
  overwritePublishDir = TRUE
)
createDiagnosticsExplorerZip(
  outputZipfile = "MyStudy.zip",
  sqliteDbPath = "SOS_STUDY_2023.sqlite"
)