Five Key Steps in Drug Development
Discovery & Development
Preclinical Research
Clinical Research
FDA Drug Review
Post-Marketing Safety Monitoring
FDA Programs to Accelerate Access
⏳ Typical Timeline: 10–15 years from discovery to approval. 🔍 Key Theme: Balancing speed, safety, and efficacy through regulated, stepwise evaluation.
Main Purpose of Regulatory Agencies
Key Regulatory Agencies Worldwide
FDA vs EMA
Similarities
Differences
Concordance Between FDA and EMA
Examples of Divergence
Role and Mission The U.S. Food and Drug Administration (FDA) plays a critical role in drug development and approval in the United States. Its mission is to protect public health by ensuring the safety, efficacy, and security of:
Scope and Economic Impact
Budget (2023)
Key Centers and Responsibilities
Drugs vs. Biologics
What the FDA Does Not Do
Summary of Key FDA Drug Regulations
Historical Background
Major Regulatory Milestones
Advertising Regulations
Prescription drug advertisements are regulated by the FDA; non-prescription drug advertisements fall under the Federal Trade Commission (FTC).
Types of FDA-regulated prescription drug ads:
Phase 1 Trials
Also called dose-finding studies, conducted to estimate safety, tolerability, and gather pharmacokinetic (PK) and pharmacodynamic (PD) data.
Usually involve 20–100 participants, often healthy volunteers, lasting several months.
Key concepts:
Common designs:
Sub-phases:
~52% success rate moving to Phase 2.
Phase 2 Trials
Goal: Assess biological activity/efficacy and monitor medium-term safety.
Enroll several hundred patients with the target disease; duration several months to 2 years.
Often include multiple doses or intervention arms.
Sub-phases:
Endpoints:
May use accelerated approval pathway for serious conditions with unmet needs, allowing approval based on surrogate endpoints from Phase 2 data.
~29% success rate moving to Phase 3.
Phase 3 Trials
Objective: Confirm efficacy and monitor safety in a large patient population (300–3000 patients), lasting 1–4 years.
Designs:
Randomization methods:
Masking/blinding types:
Data quality challenges: Missing data, incorrect data, high variability, delays.
Analysis approaches:
All trials must be registered at clinicaltrials.gov.
Phase 4 Studies (Post-Marketing Surveillance)
Conducted after FDA approval to monitor real-world safety and effectiveness.
Addresses limitations of Phase 3 (e.g., small sample size, short duration, selective populations).
Triggered regulatory changes such as the FDA Amendments Act (2007), introducing REMS (Risk Evaluation and Mitigation Strategies).
Common approach: Use real-world data (claims, electronic health records) for pharmacoepidemiological studies.
Outcomes:
Drug Life Cycle
Stages
Patent vs. Exclusivity
Pharmacoepidemiology
Definition: A scientific discipline that bridges pharmacology (study of drug effects) and epidemiology (study of disease distribution and determinants).
Scope: Uses epidemiologic methods to evaluate the use, safety, and effectiveness of medical products and interventions in human populations.
Pillars:
Real-World Data (RWD) & Real-World Evidence (RWE)
Role in Drug Development
Role of Pharmacoepidemiology (P’epi) in Drug Discovery and Phase 1–3 Trials
Natural History of Disease Studies
Unmet Need and Treatment Pattern Studies
Regulatory Approvals Based on RWE
Role of Pharmacoepidemiology in Phase 4 Studies
Effectiveness in Real-World Use
Adherence Patterns
Comparative Effectiveness
Real-World Safety
Role of Pharmacoepidemiology (P’epi) and Real-World Evidence in Policy and Clinical Decisions
Regulatory Decisions
Real-world evidence (RWE) increasingly supports FDA approvals.
Example: Prograf’s approval in 2021 for preventing organ rejection in lung transplant patients based on RWE.
Review of 2019–2021 FDA approvals:
Policy Decisions
Clinical Decisions
Randomized controlled trials (RCTs) are tightly controlled experiments where participants are randomized to treatment or placebo, considered the gold standard for assessing efficacy and required for drug approval. They have strict protocols, measure efficacy, and ensure internal validity but may have limited generalizability due to selective inclusion/exclusion criteria. They are often costly, long, and limited in studying harms, rare events, long-term effects, or diverse subgroups because of small sample sizes and short follow-up.
Real-world studies lack randomization and strict protocols, instead reflecting routine clinical practice. They measure effectiveness, can assess safety, harms, drug-drug interactions, and discontinuation in real-world populations, and can be larger and less expensive. They can answer multiple questions, include diverse patient populations, and provide data from usual care settings. However, high-quality real-world studies require robust data sources and analytic tools.
Real-world study data sources can be divided into primary and secondary categories. Primary sources include registry data, surveys, and prospective cohort studies. Registries collect standardized information on populations defined by disease or drug exposure, such as SEER for cancer incidence and survival, and are useful for natural history, biomarker discovery, and safety/effectiveness studies. Survey data from organizations like NCHS, CDC, and AHRQ can cover health, nutrition, and costs, with examples like the Medical Expenditure Panel Survey. Prospective cohort studies, such as the Framingham Study, follow participants over time to investigate disease causes and outcomes.
Secondary sources include administrative claims data and electronic health records (EHR). Claims data, generated through billing, contain enrollment, diagnoses, procedures, prescriptions, and costs, with strengths in completeness and size but lacking detailed clinical context, over-the-counter drug use, and lifestyle factors. Examples include Medicare, Medicaid, and private insurers. EHR data, created during clinical encounters, contain structured and unstructured information like clinical notes, labs, vitals, and prescription orders, offering rich clinical details but with variability in quality, missing data, and system heterogeneity.
Key differences include that claims capture dispensed prescriptions tied to reimbursement, while EHR captures prescriptions ordered and detailed clinical measures. Linking claims and EHR can provide a more complete dataset, combining breadth and depth for pharmacoepidemiologic research. Linked datasets can be fully or partially integrated, with examples including Medicare–registry linkages for anticoagulant use in cancer patients and CPRD-based studies on GLP-1 drugs and suicide risk in diabetes.
Study Designs
A study design is essentially the blueprint for research. A strong design ensures that results are valid and reliable—no amount of advanced statistical analysis can rescue a poorly designed study. In pharmacoepidemiology, three commonly used designs are:
Cross-Sectional Study
Cohort Study
Concept: Follows people forward in time.
Approach: Identify groups based on exposure (e.g., drug A vs. drug B) and track them to see who develops the outcome.
Example Questions:
Strengths: Good for studying incidence and temporal relationships.
Case-Control Study
Concept: Looks backward in time from the outcome to the exposure.
Approach:
Example: Did people with a rare adverse event take drug A more often than controls?
Strengths: Efficient for studying rare outcomes.
Statistical Analysis
In randomized controlled trials (RCTs), randomization reduces bias by balancing characteristics between groups, often minimizing the need for advanced adjustment methods. In real-world studies, treatment assignment is not random—it may depend on patient conditions, physician preferences, or other factors—so statistical adjustments are necessary.
Key points:
Study Design Level
Analysis Level
Common methods in pharmacoepidemiology:
Descriptive Analysis
Regression Models
Advanced Methods
Drug Development and Approval Process
This link provides information on drug approval steps. You can click on the each step to learn more about the process.
This paper compares and contrasts drug approval process in US vs. Europe.
This link provides chronology describing some of the milestones in the history of food and drug regulation in the United States.
This link describes four phases of clinical trials.
This paper describes traditional vs. accelerated development and approval process.
This link provides a case study on rofecoxib, illustrating how post-marketing pharmacoepidemiology research can uncover adverse events that ultimately led to the withdrawal of the drug from the market.
Role of Pharmacoepidemiology in Drug Approval Process
This is the official website of the International Society for Pharmacoepidemiology (ISPE). It offers comprehensive information about the field of pharmacoepidemiology—what it is, why it matters, and how it’s applied in practice. A valuable resource—bookmark this site!
This article outlines the stages of a drug’s life cycle and explores the market dynamics influencing drug pricing. It also provides insights into the roles of patents and market exclusivity in shaping the availability and cost of therapeutic products.
This article describes real-world evidence and its growing role in supporting FDA approvals for new drugs and biologics
Overview of Data and Methods Used in Pharmacoepidemiology
This paper provides a high-level overview of databases used in pharmacoepidemiology and describes types of research questions that can be answered using different databases.
This paper serves as an example of a pharmacoepidemiologic study that evaluates patterns of drug use—in this case, oral anticoagulants—among patients with both cancer and atrial fibrillation in a real-world setting.
This introductory paper explains the fundamental principles of cohort and case-control study designs, making it a useful resource for understanding key methodologies in pharmacoepidemiology.
This blog post from TriNetX illustrates how a patient’s healthcare journey is captured in both claims data and electronic health records (EHR). It emphasizes the value of using linked claims and EHR data to answer complex research questions that may not be fully addressed by either data source alone.
1. Introduction and Scope Caleb Alexander introduces drug utilization research (DUR) as a key component of pharmacoepidemiology and clinical care. The lecture covers:
2. The Promise and Perils of Medicines
Promise: When used correctly, medications are safe, cost-effective, and transformative (e.g., insulin, penicillin, TNF-α inhibitors).
Perils:
3. Scale of the Problem
Statistics (U.S.):
Medication Use Among Older Adults (2010–2011):
90% take ≥1 medication or dietary supplement daily.
4. Definition and Scope of Drug Utilization Research DUR is “an eclectic collection of descriptive and analytic methods for quantifying, understanding, and evaluating prescribing, dispensing, and consumption of medicines, and testing interventions to enhance quality.”
5. Drug Lifecycle and Its Implications
Use after market approval is dynamic.
Example: Diabetes therapies (1994–2007) show major shifts in use (e.g., decline in sulfonylureas, rise in biguanides).
Implications:
Drivers of change: Innovation, clinical guidelines, scientific evidence, regulatory changes, reimbursement policies, market forces.
6. Example Study – COVID-19 Therapeutics
Data Source: National COVID Cohort Collaborative.
Focus: Hydroxychloroquine, remdesivir, dexamethasone.
Findings:
7. Skills Required for DUR
8. DUR’s Relevance Across Sectors
1. Balancing Research Questions and Data Sources
Two approaches:
Often, this is an iterative process, moving back and forth between question and data source.
2. No Perfect Design
Every study has strengths and limitations; the “perfect” study does not exist.
The goal is to find the sweet spot where:
3. Descriptive Studies Can Be Important
4. Context Matters
5. Main Categories of Drug Utilization Study Designs
Descriptive Designs
Analytical Designs
Interventional Designs
Descriptive designs collect information without altering the environment, focusing on the who, what, when, and where of epidemiology. They help quantify and visualize drug use patterns and can serve as a foundation for later analytical work.
Purpose and Value
Cross-Sectional Analyses
Examples
Serial Cross-Sectional Analyses
Longitudinal Descriptive Analyses
Examples
Medicare Part D coverage adoption:
Opioid use trajectories over 5 years:
Part D utilization vs. ineligible control group:
Analytical designs focus on why drug utilization patterns occur and explore relationships between drivers (e.g., marketing, regulatory changes, scientific findings, policy shifts) and drug use outcomes. They are less common in DUR than descriptive designs but share many methodological considerations with safety and effectiveness studies.
Core Characteristics
Types of Analytical Designs
Ecological Studies
Explore associations at the population level.
Example: Analysis of SSRI prescriptions vs. suicide rates in children/adolescents after public health warnings.
Cohort Studies
Follow individuals over time to assess utilization patterns in relation to exposures or events.
Example: Danish registry-based study of statin use.
Case-Control Studies
Rare in DUR; compare those with a specific utilization-related outcome to those without.
Example: Association between physician–pharma industry interactions and requests for hospital formulary updates.
Interventional designs actively introduce a change or policy and measure its effect on drug utilization. They are less common than descriptive or analytical designs because they require more time and resources. Three main types are used: controlled before–after studies, interrupted time series (ITS), and randomized controlled trials (RCTs).
Controlled Before–After Studies
Compare outcomes before and after an intervention in both an intervention group and a control group.
Example: Policy change to reduce antibiotic use in children with acute otitis media.
Interrupted Time Series (ITS) Studies
A quasi-experimental design assessing trends before and after an intervention.
Key assumption: In the absence of the intervention, pre-intervention trends would have continued unchanged (counterfactual).
Measure:
Quality concerns:
Example: Reference-based pricing in British Columbia for ACE inhibitors.
Randomized Controlled Trials (RCTs)
Rare in DUR because many utilization questions cannot be randomized.
Advantages: Balances both measurable and unmeasurable confounders.
Example: Post–myocardial infarction patients randomized to usual insurance coverage vs. full prescription coverage for preventive medicines.
Other examples:
Other Analytical Approaches for DUR
Adherence and persistence are two related but distinct concepts in evaluating drug utilization and quality. Adherence refers to the extent to which a patient follows the prescribed dosing regimen — meaning the correct interval and dose — while persistence refers to how long a patient continues the therapy from initiation until discontinuation. Some discussions use “adherence” loosely to cover both concepts, but in research and clinical policy, distinguishing between them is important.
A related concept is primary nonadherence, which occurs when a patient never fills the first prescription for a prescribed therapy. Primary nonadherence can be significant, with rates reaching 25% in some settings, and a pooled estimate from a meta-analysis of several drug classes showing around 15%. More broadly, the WHO estimates that average adherence for chronic disease treatments in developed countries is around 50%, meaning half of patients do not take their medication as prescribed. Costs are a major driver — historically, about one in three U.S. adults have reported skipping or stretching medications due to expense.
Measuring adherence can be done through various methods, each with pros and cons. Common approaches include pharmacy refill records, electronic monitoring devices, and biochemical verification. In pharmacoepidemiology, the medication possession ratio (MPR) is widely used: it calculates the proportion of days in a given period that are “covered” by dispensed medication. An MPR threshold of 80% is often used to define adherence, although this is arbitrary. Other proportional measures include the proportion of days covered (PDC), continuous measures of medication availability, refill rates, and prescription possession ratios. These measures may overestimate true adherence, since filling a prescription doesn’t guarantee that the patient takes the medication.
Electronic monitoring systems — from pill bottles with sensors to devices linked with inhalers — can capture more granular patterns of medication use. Data often show different adherence trajectories across patients, from nearly perfect long-term adherence to rapid early discontinuation. Adherence is influenced by multiple interacting factors, including patient-level, socioeconomic, healthcare system, and provider-related elements.
Nonadherence can be grouped into three broad typologies:
No single intervention solves nonadherence; solutions must be tailored to the underlying reasons, patient context, and type of nonadherence present.
Off-label use refers to prescribing a drug in a way that differs from its FDA-approved label, which specifies the approved dose, target population, duration, and other usage parameters. This includes using a drug for an unapproved condition, in an unapproved patient group, at a different dosage, or for a longer/shorter duration than specified. It can range from guideline-recommended and evidence-based practice to unsafe, scientifically unsupported, or experimental use.
Off-label use is common for several reasons, including clinical innovation, unmet medical needs, and potential financial incentives for manufacturers. Potential pathways to off-label use include:
Advantages of off-label use can include fostering innovation, increasing available treatment options, and providing therapy for rare diseases with limited alternatives. Disadvantages include exposing patients to safety risks, increasing healthcare costs through ineffective treatment, and bypassing regulatory safeguards designed to protect patients.
Evidence shows that off-label prescribing is frequent and often unsupported:
The FDA’s policy is to regulate drug approval, not prescribing. Physicians can legally prescribe off-label based on clinical judgment. However, manufacturers face restrictions:
Potential policy improvements include:
Overall, off-label use is widespread, sometimes beneficial, but often unsupported by strong evidence, creating significant safety, effectiveness, and cost concerns. Efforts to manage its risks require collaboration between regulators, payers, researchers, and clinicians.
Here’s the side-by-side comparison table for Explicit vs. Implicit Quality Indicators:
| Aspect | Explicit Quality Indicators | Implicit Quality Indicators |
|---|---|---|
| Definition | Clearly defined, quantifiable measures of prescribing appropriateness, usually based on evidence or expert consensus, and applicable to large datasets (e.g., claims, EHRs). | Subjective measures that require expert clinical judgment to assess the appropriateness of individual prescriptions. |
| Examples | Beers criteria, START/STOPP criteria, DU90%, generic prescribing rates, polypharmacy thresholds (≥5 concurrent drugs). | Medication Appropriateness Index (MAI), Prescribing Appropriateness Indicator (PAI). |
| Data Requirements | Often derived from structured data (diagnosis codes, prescription records, lab results); can be automated. | Requires detailed patient-specific clinical information; often involves chart review or clinician interview. |
| Strengths | - Objective and reproducible - Suitable for large-scale monitoring - Easy to automate - Can be applied consistently across populations |
- Captures context-specific nuances - Can address issues not easily codified (effectiveness, directions, duplication, cost) - Flexible to complex patient scenarios |
| Weaknesses | - May oversimplify complex decisions - Limited to predefined criteria; may not reflect individual patient needs - Risk of “gaming” the system or unintended overuse |
- Resource-intensive (time, expertise) - Limited scalability - Subjective, potential for inter-rater variability (though tools like MAI improve reliability) |
| Reliability | Generally high if criteria and data are clear; less dependent on reviewer skill. | Dependent on assessor training and consistency; MAI and PAI show good inter- and intra-rater reliability with training. |
| Validity | Strong face and content validity if evidence-based; predictive validity depends on link to outcomes. | Can have strong content, predictive, and responsiveness validity when well-developed and tested. |
| Best Use Cases | Population-level monitoring, quality reporting, benchmarking, policy evaluation. | Individual patient medication review, complex cases, research on appropriateness in nuanced contexts. |
Explicit quality indicators are clearly defined, quantifiable measures used to evaluate prescribing quality and its concordance with scientific evidence or guidelines. They can help clinicians, patients, policymakers, and payers assess whether prescribing practices are appropriate, identify overuse, underuse, and misuse, and inform quality improvement, pay-for-performance programs, and accreditation.
Quality issues in prescribing are widespread. Ambulatory medication errors—especially prescribing and dosing errors—affect anywhere from one-quarter to over 90% of prescriptions depending on population and definitions used. Older adults are particularly vulnerable because of polypharmacy and the risks of both underuse and overuse.
Explicit indicators can be grouped into three main categories:
Examples of explicit measures include the Beers criteria, START/STOPP criteria (screening tools for appropriate and inappropriate prescribing in older people), DU90% (percentage of drug volume accounted for by the top drugs in defined daily doses), use of generics or preferred formulary drugs, and polypharmacy thresholds (often ≥5 concurrent medications).
However, explicit lists and criteria, while clear and automatable using claims or EHR data, still involve expert judgment in their development and can have limitations in applicability. Their validity is judged on dimensions like face validity (accepted by professionals), content and predictive validity (evidence-based, predictive of outcomes), concurrent validity (correlates with a gold standard), and feasibility (availability of necessary data and applicability to real-world patient care). Potential unintended effects—such as “gaming” the system or driving overuse—must be considered.
Implicit quality indicators differ in that they require expert judgment and subjective evaluation of individual prescribing decisions. They are not easily automated but can capture nuances that explicit measures might miss, such as appropriateness of directions, therapeutic duplication, or consideration of cost.
Two widely used implicit tools are:
Implicit measures have been applied in intervention studies, such as pharmacist- or geriatrician-led medication reviews, to document improvements in prescribing. For example, in one geriatric assessment trial, MAI scores improved significantly in the intervention group, with more drug withdrawals, dose reductions, and initiation of appropriate new therapies compared to controls.
Marketing & Promotion
Emerging Evidence
Information ecosystem & challenge: New data come from journal publications, news reports, regulatory advisories, professional meetings, peer exchanges, and drug labels/websites. Systematic reviews are often slow to update and too lengthy for rapid clinical uptake.
Impact pathways: Evidence can directly change prescribing or indirectly influence it via guideline updates, marketing shifts, payer policy changes, and formulary revisions.
Examples:
What moves the market most: Clear risk signals, large/rigorous trials, actionable conclusions, high media and professional visibility, and alignment with existing trends are most likely to shift prescribing quickly.
Research tips: Examine differential impacts (new users vs. continuers, by specialty or patient type). Consider interaction with marketing/regulation/reimbursement. Use interrupted time series, event studies, or difference-in-differences and include time-varying covariates like “media coverage,” “guideline release,” or “formulary changes.”
Regulation
FDA’s “does and does not”: The FDA approves drugs based on substantial evidence of safety and efficacy (often ≥2 rigorous RCTs, sometimes 1 plus supporting data) and may accept surrogate endpoints. It enforces promotional compliance and post-market safety. It does not assess cost-effectiveness, decide the “best” drug in a class, endorse all possible uses, or identify all rare AEs pre-approval.
Post-market tools: Risk Evaluation and Mitigation Strategies (REMS), required post-market studies, label changes, marketing restrictions, safety communications, and in rare cases, withdrawals.
Effectiveness of advisories: Few advisories have strong, immediate effects; most are delayed or minimal. Impact depends on the audience, clarity/specificity, communication channel, and whether actionable steps are provided.
Risk communication pitfalls: Poorly timed or vague warnings, lack of patient materials, and media amplification of small risks can harm public health (e.g., UK “pill scare,” thimerosal-vaccine panic).
Best practice: Use best available evidence, be specific and clear, tailor to audience, provide actionable guidance with supporting materials, and evaluate/iterate messaging.
Coverage & Reimbursement
Economic rationale: Cost sharing reduces moral hazard (overuse when someone else pays) but excessive cost sharing reduces access and adherence.
Formularies & P&T committees: Insurers/PBMs use Pharmacy & Therapeutics committees to assess efficacy, safety, and outcomes, tier drugs (generic/preferred/non-preferred), and set copays to steer use toward generics/preferred drugs. Restrictiveness varies widely, affecting patient out-of-pocket costs.
Utilization management tools:
Evidence: Cost sharing, tiering, and coinsurance have the largest effect on use; drug price elasticity is about -0.2 to -0.6 (10% copay ↑ → 2–6% drop in use/spending). Higher patient costs can worsen adherence and increase ER/hospital use for chronic conditions.
Natural experiments:
Policy balance: Combine value-based insurance design (low/no copay for high-value chronic meds), protections for vulnerable groups, and caps/catastrophic coverage to avoid both overuse and underuse.
Putting it all together
Introduction to Drug Utilization Research
This report introduces one of the most commonly used methods in drug utilization research – interrupted time series designs.
This report also discusses interrupted time series designs and considers design features includinglagged effects and multi-component interventions; corrections for first, second or higher level autocorrelation; and accounting for possible biases that may threaten study interpretation.
Evaluating Drug Utilization and Quality
The authors summarize, categorize, and estimate the effect size of interventions to improve medication adherence in chronic medical conditions, concluding that while several types of interventions are effective in improving medication adherence in chronic medical conditions, few significantly affected clinical outcomes.
This is a carefully developed and executed analysis that used a unique data source to quantify the frequency of off-label drug use in ambulatory practice in the United States, as well as the degree of scientific evidence supporting this practice.
This review, which focuses on polypharmacy and deprescribing in the elderly, provides a helpful framework for considering medication appropriateness and quality among this often-vulnerable population.
The authors present a revised and updated version of the STOPP/START criteria, which is one of the most widely used physiological systems-based explicit set of prescription indicators.
This editorial, by the developers of the Medication Appropriateness Index (MAI), contrasts this implicit tool with explicit quality indicators, and reviews information about the MAI’s reliability, validity, and responsiveness to change, as well as new applications of the MAI after 30 years of use.
Determinants of Drug Utilization
This satirical sketch from late night television examines the methods that pharmaceutical companies use to market their products to physicians.
This systematic review of the relationship between exposure to information from pharmaceutical companies and the quality, quantity, and cost of physicians’ prescribing concludes that with rare exceptions, studies of exposure to information provided directly by pharmaceutical companies have found associations with higher prescribing frequency, higher costs, or lower prescribing quality or have not found significant associations.
This eloquent and rigorous study used standardized patients to assess the effect of patient request on medication receipt, concluding that patients’ requests have a profound effect on physician prescribing in major depression and adjustment disorder.
This systematic review considered the totality of evidence regarding the impact of FDA regulatory communications regarding prescription drugs, concluding that although some FDA drug risk communications had immediate and strong impacts, many had either delayed or had no impact on health care utilization or health behaviors.
This is a widely cited study that examines the effect of prescription cost-sharing on drug and non-drug utilization and health, concluding that while pharmacy benefit design represents an important public health tool for improving patient treatment and adherence, the long-term consequences of benefit changes on health are still uncertain.
The overarching aim of drug effectiveness research is to improve human health by identifying which therapies work best for which patients in real-world conditions. This involves determining which treatments are most effective, safest, affordable, and tolerable. Effectiveness research often generates evidence on the benefits, harms, and costs of different treatment options, with the goal of guiding clinical decisions, regulatory policies, coverage determinations, and formulary listings.
Effectiveness research fits into the translational science pathway after efficacy studies. While basic science and early-phase clinical trials answer questions about mechanisms and initial safety/efficacy, effectiveness studies investigate whether interventions work when prescribed and used in everyday clinical practice. This includes assessing adherence, tolerance, barriers to access, affordability, drug–drug interactions, and safety in typical patient populations.
Framing a Research Question Well-framed research questions are crucial for designing meaningful studies. Two common approaches are:
Analytic Framework
PCOTES Framework (Population, Comparator, Outcomes, Timing, Setting)
Using such frameworks ensures that the research question is clear, specific, and directly informs study design.
The distinction is central to drug research:
Efficacy Studies
Effectiveness Studies
The design choice—efficacy or effectiveness—should be guided by the needs of the stakeholders who will use the evidence.
Here’s the combined Efficacy vs. Effectiveness Studies table with definitions based on both the transcript and the slide:
| Aspect | Efficacy Studies | Effectiveness Studies |
|---|---|---|
| Objective | Determines if an intervention works under optimal, controlled conditions. Answers “Can it work?” | Determines if an intervention works under usual, real-world conditions. Answers “Does it work?” |
| Motivation | Primarily for regulatory approval before market release. | Supports practice guidelines, formulary decisions, and healthcare policy after approval. |
| Intervention | Fixed regimen with strict dosing and administration protocols. | Flexible regimen as used in routine clinical practice. |
| Comparator | Often placebo, sometimes standard of care. | Usual care or active comparator reflecting real-world choices. |
| Design | Randomized controlled trials (RCTs) with tight control over variables. | Observational studies or pragmatic trials embedded in real-world practice. |
| Subjects | Highly selected participants meeting strict eligibility criteria. | Usual users of the intervention, often more heterogeneous in age, comorbidities, and demographics. |
| Outcomes | Condition-specific endpoints (e.g., biomarkers, surrogate outcomes) closely linked to the drug’s mechanism, often short-term. | Comprehensive, patient-centered outcomes (e.g., quality of life, clinical endpoints) that may have weaker link to mechanism but higher relevance, covering both short- and long-term horizons. |
1. Stakeholders Who Need Evidence
Stakeholders are individuals or groups with a vested interest in decisions informed by research. Their early involvement in shaping questions, influencing study design, and interpreting results improves both the relevance and uptake of findings. In practice, these stakeholders are often referred to as the “Seven P’s”:
| Stakeholder Group | Examples | Primary Evidence Needs |
|---|---|---|
| Patients / Public | Current patients, former patients who act as advocates, family members advocating on behalf of patients | Information on the most effective, safest, affordable, and tolerable treatments for their condition |
| Providers | Physicians, nurses, hospitals, community health centers, pharmacies | Evidence on the most effective and safest treatments, with minimal adverse effects or drug–drug interactions |
| Purchasers | Employers, government entities financing healthcare | Cost-effectiveness data, including real-world treatment performance |
| Payers | Insurers, Medicare/Medicaid, national health services | Safety, effectiveness, and cost-effectiveness of covered products |
| Policymakers | Regulatory agencies, legislators | Data on safety and public health impact to inform regulations and policy |
| Product Makers | Pharmaceutical and device manufacturers | Post-marketing safety monitoring and evidence across the product lifecycle |
| Principal Investigators / Researchers | Academic and industry scientists, research funders | High-quality, credible evidence to meet scientific and regulatory standards |
2. Real-World Evidence for Decision-Making
Real-world evidence (RWE) refers to clinical evidence on the use, benefits, and risks of a medical product, derived from real-world data (RWD). Real-world data includes information on patient health status and healthcare delivery that is routinely collected outside of controlled trials. Sources include electronic health records, administrative claims or billing data, disease or product registries, and increasingly, digital health technologies and wearables.
3. Regulatory Context
The 21st Century Cures Act (2016) accelerated the integration of RWE into U.S. regulatory decision-making. The FDA’s 2018 RWE Framework focuses on two main uses:
When evaluating an RWE study, the FDA asks whether the data are fit for use—meaning they have adequate sample size, relevant endpoints, completeness, and consistency—whether the study design is appropriate, and whether the analysis plan is robust. Key considerations include the use of active comparators, addressing unmeasured confounding, choosing the right hypothesis framework (superiority or non-inferiority), and pre-specifying sensitivity analyses to avoid selective reporting.
4. Study Designs Generating RWE
RWE can come from purely observational studies, such as retrospective cohort studies or registry-based research, or from hybrid/pragmatic randomized controlled trials that integrate elements of real-world practice.
Two examples of hybrid approaches are:
5. Linking Stakeholders to RWE Uses
| Stakeholder | Use of RWE |
|---|---|
| Patients | To make informed treatment choices based on safety and effectiveness in real-world settings |
| Providers | To guide clinical decision-making aligned with actual practice patterns |
| Purchasers | To evaluate cost-effectiveness of treatments |
| Payers | To inform coverage and reimbursement decisions |
| Policymakers | To support regulations and update clinical guidelines |
| Product Makers | To monitor safety and support market positioning throughout the product lifecycle |
| Researchers | To develop methods, test hypotheses, and confirm findings in real-world populations |
Trials vs Observational Studies
Goal: Establish causality, not just association, between an intervention and an outcome.
Randomized Trials
Observational Studies
Retrospective Cohort Studies
Definition: Identify exposed and unexposed individuals from past data, follow forward for outcomes.
Population Definition
Comparison Groups: Usually different treatments or no treatment (no placebo in routine care).
Challenges
Exposure Definition
Outcome Definition
Target Trial Emulation
Concept: Design observational study as if it were a randomized trial.
Steps
Analytic Methods
Bias Avoidance
Case-Control Studies
Definition: Select individuals with the outcome (cases) and without the outcome (controls), look back for exposure history.
Advantages
Limitations
Measure of Association: Odds ratio (approximates risk ratio if outcome is rare).
Analysis: Conditional logistic regression to adjust for confounders.
Example: Pediatric SSTI treatment effectiveness — adjusted for illness severity and seasonal changes in practice.
Self-Controlled Designs (Case-Crossover)
Definition: Case-only design comparing exposure during hazard period before outcome with exposure during control periods for same person.
Advantages
Requirements
Key Concepts
Analysis
Example: Oxaliplatin and anaphylactic shock — short hazard/control periods, elevated risk observed.
Key Takeaways
Purpose of Addressing Bias
Selection Bias
Prevalence Bias
Protopathic Bias
Information Bias
Misclassification
Non-differential: Errors similar across groups; usually bias toward the null.
Differential: Errors differ by group; systematic bias away from or toward null.
Causes:
Recall bias: cases may recall exposures differently from controls (common in case-control studies).
Detection Bias
Differences in monitoring or follow-up frequency lead to unequal outcome detection.
Examples:
Mitigation Strategies
When collecting new data:
When using existing data:
Definition
Mechanism of Bias
Common Examples
Illustrative Problem
Solutions
Time-dependent exposure modeling
Redefine time zero
When to Suspect Immortal Time Bias in a Study
Prevention
Definition
Key Challenge
Analytical Approaches to Control Confounding
Purpose
Concept
Propensity Score (PS) = probability of receiving the treatment of interest given baseline covariates
Why Use
Workflow
Estimate the PS
Apply the PS to Control for Confounding
Matching: Pair treated and control patients with similar PS values.
Stratification/Subclassification: Group into PS strata (often quintiles) and adjust analyses accordingly.
Regression Adjustment: Include PS as a covariate in the outcome regression model.
Weighting: Use Inverse Probability of Treatment Weighting (IPTW).
Example
Considerations & Limitations
Large sample sizes are generally needed for effective balance.
PS methods do not address unmeasured confounding.
Treatment effects may vary across PS strata (heterogeneity of treatment effect), which may need explicit evaluation.
Proper variable selection remains debated:
Study Context
Stakeholders
Target Trial Emulation Approach
Purpose: Force explicit trial-like design choices in an observational setting.
Trial-like Specifications:
Eligibility Criteria:
Treatment Strategies:
Follow-up Start (Time Zero): Date of treatment assignment (baseline).
Outcome: First diagnosis of breast, colorectal, lung, or prostate cancer.
Analysis Strategies:
Statistical Design & Methods
Sequential Emulation:
Adjustment for Confounding:
Reasoning: Mimics randomization; reduces time-related biases by careful specification of time zero.
Baseline Imbalances Before Weighting
Strengths
Limitations
Implications
Study Context
Stakeholders
Design Overview
Exposure & Time Periods
Object Drug: Insulin secretagogue (primary) or metformin (negative control).
Precipitant Drug: ACE inhibitor.
Time Classification:
Baseline Observation: ≥180 days prior to object drug start without exposure.
Analytic Approach
Main Analysis: Conditional Poisson regression → Rate Ratios (RR) for hypoglycemia during ACE inhibitor exposure vs. non-exposure.
Strength of SCCS:
Time-Varying Covariates Adjusted For:
Results
Strengths
Limitations
Principles of Drug Effectiveness Research
This white paper describes the real-world evidence program that was launched soon after passage of the 21st Century Cures Act in the U.S. in 2016. It includes many definitions relevant to studies that use real-world data to generate real-world evidence about medical products.
Methods in Drug Effectiveness Research
This is an example of a case-control study designed to evaluate a potential harm of the first glucagon-like peptide 1-based therapies.
This is an example of a case-crossover study designed to evaluate a potential harm of antidepressants.
This paper describes the target trial methodology by the authors who were instrumental in developing these methods.
This is an example of the prevalent new-user cohort design.
This is a rich article about matching methods for use with propensity scores.
Please see this article for more advanced reading about propensity score methodology.
This is the PRECIS-2 Tool that is valuable when designing trials to generate real world evidence.
Comparative Effectiveness and Real-World Evidence
Dickerman BA, García-Albéniz X, Logan RW, Denaxas S, Hernán MA. Evaluating Metformin Strategies for Cancer Prevention: A Target Trial Emulation Using Electronic Health Records. Epidemiology. 2023 Sep 1;34(5):690-699. doi: 10.1097/EDE.0000000000001626. Epub 2023 May 23. PMID: 37227368; PMCID: PMC10524586.
Hee Nam Y, Brensinger CM, Bilker WB, Flory JH, Leonard CE, Hennessy S. Angiotensin-Converting Enzyme Inhibitors Used Concomitantly with Insulin Secretagogues and the Risk of Serious Hypoglycemia. Clin Pharmacol Ther. 2022 Jan;111(1):218-226. doi: 10.1002/cpt.2377. Epub 2021 Aug 23. PMID: 34312836; PMCID: PMC8678147.
These are the two studies discussed in depth in the lecture that illustrate the use of pharmacoepidemiology methods to study the effectiveness and/or safety of products.
Definition & Importance
Origin: From pharmakon (Greek: medical substance) + vigilia (Latin: to keep watch).
WHO Definition: Science and activities related to detection, assessment, understanding, and prevention of adverse effects or other drug-related problems.
FDA Definition: Scientific and data-gathering activities related to detection, assessment, and understanding of adverse events.
Post-Marketing Surveillance (FDA): Monitoring the safety of a drug or device after market release.
Why It Matters:
Adverse Event (AE) vs. Adverse Drug Reaction (ADR)
Limitations of Pre-Approval Studies (Phases I–III)
Designed for efficacy, not safety signal detection.
Small exposure: Usually only thousands of patients.
Homogeneous participants: Strict inclusion/exclusion criteria; excludes complex patients with comorbidities or polypharmacy.
Short follow-up: Limits detection of long-term risks.
Controlled settings: Close monitoring doesn’t reflect real-world use.
Post-Approval Changes:
Pharmacovigilance Process
Surveillance Types
Passive Surveillance
Relies on spontaneous/voluntary reporting.
Examples:
Advantages: Low cost, broad coverage.
Limitations: Underreporting, reporting bias.
Active Surveillance
Impact of Post-Marketing Safety Surveillance
Study of 222 novel therapeutics (FDA approvals 2001–2010):
32% had a post-marketing safety event (e.g., safety communication, boxed warning, market withdrawal).
Higher event frequency in:
Background & IOM 2006 Report
Key Findings
Institute of Medicine (IOM) Recommendations
Improve Safety Signal Generation & Hypothesis Development
Active Surveillance
Expertise in Advisory Committees
Regulatory Authority Enhancement
Legislative Impact: 2007 FDA Amendments Act (FDAAA)
Post-Approval Study Authority: FDA can require manufacturers to conduct post-marketing studies.
Enforcement: Authority to impose monetary penalties for non-compliance.
Risk Evaluation and Mitigation Strategies (REMS):
Post-Marketing Safety Reporting Requirements
Expedited Reports:
Non-Expedited Reports (domestic):
Serious & expected.
Non-serious & unexpected.
Non-serious & expected.
Reporting schedule:
Ongoing Responsibility
Definition (WHO)
Examples of Safety Signals
Purpose of Safety Signal Detection
Potential Sources of Safety Signals
Clinical Trials
Pharmacovigilance Databases
Medical Literature
Media Reports
Manufacturer’s Global Safety Database
Foreign Regulatory Agencies
Observational Studies
Active Surveillance Programs
Detection Methods
1. Manual Review
Review of individual case safety reports (ICSRs), e.g., all deaths or all serious cases for a specific product.
Advantages: allows detailed clinical evaluation.
Limitations:
2. Data Mining / Statistical Signal Detection
Automated methods to identify disproportionality between drug–event pair reporting frequency and expected frequency.
Common approach: compare the proportion of a specific event among all reports for a given drug against the proportion for all other drugs in the database.
Advantages:
Limitations:
Key Characteristics of Data Mining in Pharmacovigilance
Disproportionality Measures
Thresholds for signal detection vary (e.g., PRR ≥ 2, chi-square ≥ 4, ≥ 3 cases).
Dynamic nature:
Important Notes on Interpretation
Why True Incidence Cannot Be Calculated
Workaround: Reporting Ratio
Reporting Ratio =
\[ \frac{\text{Number of cases of an event for a drug}}{\text{Number of dispensed prescriptions (utilization data)}} \]
Limitations:
Overview After a safety signal is detected, the next step is to evaluate whether the relationship between the drug and the adverse event is causal or coincidental. This assessment is performed at two levels:
Five Key Factors for Individual Case Assessment
1. Temporal Relationship
Does the adverse event occur after the drug was administered, within a biologically plausible time frame?
Dechallenge (D-challenge):
Rechallenge:
2. Precedence (Class Effect)
3. Biological / Pharmacological Plausibility
Is there a mechanistic explanation consistent with current medical or pharmacological knowledge?
Examples:
4. Alternative Etiologies
Could the event be explained by other causes?
5. Information Quality
Are the case details complete and reliable?
WHO–UMC Causality Categories
The World Health Organization – Uppsala Monitoring Centre classification:
| Category | Description |
|---|---|
| Certain | Clear temporal relationship, positive dechallenge/rechallenge, no alternative explanation. |
| Probable / Likely | Reasonable temporal relationship, positive dechallenge, unlikely alternative explanation. |
| Possible | Reasonable temporal relationship, but alternative explanations are possible. |
| Unlikely | Temporal relationship improbable, other explanations more likely. |
| Conditional / Unclassified | More data needed to assess. |
| Unassessable / Unclassifiable | Insufficient or contradictory information that cannot be resolved. |
Aggregated (Product-Level) Causality Assessment
When moving beyond individual cases, regulators and safety scientists consider:
Totality of Evidence from:
Strength and Consistency across sources.
Precedence from related drugs/classes.
Mechanistic plausibility supported by biology or pharmacology.
Key Takeaways
What are Boxed Warnings?
Process for Adding a Boxed Warning
Key Takeaways
Definition
A spontaneous or voluntary reporting system for adverse events (AEs) or adverse drug reactions (ADRs).
Reports are submitted directly to:
Most common form of pharmacovigilance worldwide.
Key Characteristics
Reporter-initiated:
Passive Surveillance Systems – U.S. vs. Global
| Aspect | United States | Worldwide (WHO PIDM) |
|---|---|---|
| Main Program | MedWatch (FDA medical product safety reporting program, est. 1993) | WHO Program for International Drug Monitoring (PIDM) (est. 1968) |
| Core Database | FAERS – FDA Adverse Event Reporting System | VigiBase – WHO global database of suspected ADRs |
| Report Sources | - Voluntary: Healthcare professionals &
consumers (~6%) - Mandatory: Manufacturers (~94%) |
- National pharmacovigilance centers in 170+ member countries (≈99% world population) |
| Reporting Methods | - Online forms (HCP, consumer, industry) - Paper forms |
- National centers submit ADR data via VigiFlow |
| What to Report | - Unexpected side effects/adverse events (mild → death) - Product quality issues - Medication errors - Therapeutic failures |
- Suspected adverse effects from medicines & vaccines |
| Products Covered | - Drugs (Rx & OTC) - Biologics (blood products, gene therapy, transplants) - Medical devices - Certain combination products - Cosmetics & food (safety-related events) |
- Medicines & vaccines from all therapeutic areas |
| Manufacturer Obligations | - Serious & unexpected AEs: Report within 15 days (domestic
& foreign) - Others: Quarterly (first 3 yrs), then annually |
- Varies by national law; member countries commit to regular data submission to WHO |
| Public Access | - FAERS Dashboard (interactive) - FAERS data files (raw) - Case-level data upon request |
- VigiBase access via WHO-UMC (for members); aggregated signal info shared globally |
| Signal Detection Process | 1. AE identified → reported to MedWatch → entered into FAERS. 2. FDA analysts monitor for signals. 3. Confirmed signal → safety communications/regulatory action. |
1. National centers collect ADR reports → VigiFlow → VigiBase. 2. WHO-UMC analyzes global data for signals. 3. Findings shared with members & WHO for action. |
| Other Reporting Mechanisms | - Joint Commission Sentinel Event System - Maryland Dept. of Health ADE reporting - ISMP error reporting - USP MedMarks - VA Patient Safety Reporting |
- Integration with other major databases: U.S. FAERS & VAERS, EU EudraVigilance, national databases |
Analysing Passively Reported Events
Safety Signal (WHO):
Possible causal relationship between AE and drug, previously unknown/incompletely documented.
Examples:
Disproportionality Methods
Proportional Reporting Ratio (PRR)
\[ PRR = \frac{\frac{A}{A+B}}{\frac{C}{C+D}} \]
Reporting Odds Ratio (ROR)
\[ ROR = \frac{\frac{A}{B}}{\frac{C}{D}} \]
Strengths & Limitations of Passive Methods
Strengths
Limitations
Post-marketing surveillance: Monitoring the safety of a drug or device after market release.
Limitations of Passive Surveillance Addressed by Active Surveillance
Definition (EMA)
Active surveillance seeks to ascertain completely the number of adverse events via a continuous, pre-organized process.
Active approach: Investigator actively pursues answers to safety questions.
Contrast:
Types of Active Surveillance
Sentinel sites
Drug event monitoring
Registries
Large data activities
International Active Surveillance Systems
Use of large datasets and organized systems to actively monitor adverse events (AEs) after drug or biologic product approval.
Overcomes passive surveillance limitations by:
| Region / Country | System Name | Lead Organization | Focus / Notes |
|---|---|---|---|
| United States | Sentinel Initiative | FDA (CDER) | Uses claims & EHR data for signal detection, strengthening, and validation. |
| Vaccine Safety Datalink (VSD) | CDC | Long-standing vaccine AE monitoring (pre-dates Sentinel). | |
| Biologics Effectiveness and Safety (BEST) Initiative | FDA (CBER) | Part of Sentinel; monitors biologics (vaccines, blood, tissues, gene therapies, devices) but not biologic drugs regulated by CDER. | |
| Europe (EU) | Darwin EU | EMA | Centralized EU data network for drug safety and effectiveness research. |
| Canada | CNODES | Canadian Network for Observational Drug Effect Studies | Active drug safety monitoring. |
| CAEFISS | Public Health Agency of Canada | Vaccine AE monitoring. | |
| United Kingdom | Vigilance and Risk Management of Medicines Division | MHRA | Active AE surveillance for medicines. |
| Australia & Asia | Asian Pharmacoepidemiology Network (AsPEN) | Multinational Collaboration | Shared pharmacoepidemiology and safety monitoring infrastructure across Asia, some Europe, and Australia. |
| Japan | MID-NET (Medical Information Database Network) | Ministry of Health, Labour and Welfare & PMDA | Uses EHR and administrative data for AE surveillance. |
| China | National ADR Monitoring Sentinel Alliance | National Medical Products Administration | EHR-based active AE monitoring program (launched 2016). |
Introduction to Drug Safety
This document describes principals for the FDA’s conduct of ongoing postmarketing safety surveillance for human drug products and human biological products. It includes safety identification and an assessment of the causal association between the product and the identified adverse event.
This paper describes the frequency of postmarket safety events, including withdrawal, boxed warnings, and safety communications, among novel therapeutics approved by the US FDA between 2001 and 2010.
This paper provides the background history of the FDA Amendments Acts of 2007 and its impact on FDA’s capacity to track medication effects and mitigate risk.
Pharmacovigilance: Passive Surveillance
This article provides a concise and insightful perspective on the importance of post-marketing drug safety surveillance, particularly through passive surveillance systems like spontaneous adverse event reporting. It also discusses signal detection methods, FDA and international adverse event reporting systems, and limitations and strengths of passive surveillance methods.
Pharmacovigilance: Active Surveillance
This paper describes the US FDA’s Sentinel Initiative (active surveillance system using large database in the US) to monitor the safety of medical products.
This presentation slides describe DARWIN EU (data network for collecting and analyzing real-world data on the safety and effectiveness of medicines, established by the European Medicines Agency and European Medicines Regulatory Network), including its objectives and ongoing implementation efforts.