rtichoke for time-to-events and competing risks framework

Agenda:

  • Competing-Risks and Time-to-Events in Prediction Models

    • Why we should care about time-to-events in prediction models?

    • Why we should care about competing-risks in prediction models?

  • rtichoke generic Framework for Performance Metrics

    • Development: Binary Model / Time-to-Event Model / Competing Risks.
    • Validation: Competing-Risks + Time-to-Events framework.

Competing-Risks and Time-to-Events in Prediction Models

Why Competing-Events are important for prediction models? 📅

p̂

0.11

0.15

0.18

0.29

0.31

0.33

0.45

0.47

0.63

0.72

Ŷ

0

0

0

0

0

0

0

1

1

1

Y

0

0

0

0

1

0

1

0

1

1


🤨


🤨


🤨


🤨


🤢


🤨


🤢

💊
🤨

💊
🤢

💊
🤢

What’s the PPV?

p̂

0.11

0.15

0.18

0.29

0.31

0.33

0.45

0.47

0.63

0.72

Ŷ

0

0

0

0

0

0

0

1

1

1

Y

0

0

0

0

1

0

1

0

1

1

Time

0.5

3

0.01

4.9


🤨


🤨


🤨


🤨


🤢


🤨


🤢

💊
🤨

💊
🤢

💊
🤢

What can go wrong?

p̂

0.11

0.15

0.18

0.29

0.31

0.33

0.45

0.47

0.63

0.72

Ŷ

0

0

0

0

0

0

0

1

1

1

Y

0

0

0

0

1

0

1

0

1

1

Time

0.5

3

0.01

4.9


🤨


🤨


🤨


🤨


🤢


🤨


🤢

💊
🤨

💊
🤢

💊
🤢

First patient to be treated was observed with a late event.
Was it the best choice? Common implied assumption - we would like to prioritize the most urgent cases.

Second patient to be treated was observed with a very early event.
Was it the best choice? Maybe it’s too good early to be true.

p̂

0.11

0.15

0.18

0.29

0.31

0.33

0.45

0.47

0.63

0.72

Ŷ

0

0

0

0

0

0

0

1

1

1

Y

0

0

0

0

1

C

1

C

1

1

Time

7

0.5

8

3

9

0.01

4.9


🤨


🤢


🤬


🤨


🤢


🤬


🤢

💊
🤬

💊
🤢

💊
🤢

What about longer time-horizons?
10 Years prediction instead of 5 years prediction -> more censored/primary events.

p̂

0.11

0.15

0.18

0.29

0.31

0.33

0.45

0.47

0.63

0.72

Ŷ

0

0

0

0

0

0

0

1

1

1

Y

0

0

0

0

1

C

1

C

1

1

Time

7

0.5

8

3

9

0.01

4.9


🤨


🤢


🤬


🤨


🤢


🤬


🤢

💊
🤬

💊
🤢

💊
🤢

  • Heuristic: Exclusion.
    Assumption: Number of censored events is negligible. Implication: Overestimation.

  • Heuristic: KM-Estimate.
    Assumption: Non-Informative Censoring. Implication: Depends on censoring mechanism.

  • Heuristic: Weighted KM-Estimated.
    Assumption: Informative Censoring. Implication: requires additional model.

Why Competing-Events are important for prediction models? 💀

How should we consider unrelated death cases?

p̂

0.11

0.15

0.18

0.29

0.31

0.33

0.45

0.47

0.63

0.72

Ŷ

0

0

0

0

0

0

0

1

1

1

Y

0

0

0

0

1

0

1

0

1

1


🤨


🤨


🤨


🤨


🤢


💀


🤢

💊
💀

💊
🤢

💊
💀

: How should we consider unrelated death cases?

How should we consider unrelated death cases?

p̂

0.11

0.15

0.18

0.29

0.31

0.33

0.45

0.47

0.63

0.72

Ŷ

0

0

0

0

0

0

0

1

1

1

Y

0

0

0

0

1

0

1

0

1

1


🤨


🤨


🤨


🤨


🤢


💀


🤢

💊
💀

💊
🤢

💊
💀

: How should we consider unrelated death cases?

  • Heuristic: Exclusion.
    Assumption: Number of competing events is negligible. Implication: Overestimation.

  • Heuristic: Competing-Event = Non-Event.
    Assumption: Irrelevance of the primary outcome and possible related treatment.

rtichoke to compare different types of models

Solution: Fancier Model?

  • Time-to-Event models for dealing with censoring.

  • Competing-Risks models for dealing with death.

    Drawbacks:

    • Common for statisticians, not in ML community.

    • Less established technological api’s.

    • Difficult to interpret.

    • Demand more complicated data structures.

Is it worth the trouble?

  • Estimate the ‘trouble’.

  • What is the benefit?

Can we compare Calibration for different types of models?

Can we compare Discrimination for different types of models?

Can we compare Utility for different types of models?