R Simulation for CEACs

Example No. 1

Strategy 2 is approximately twice as expensive as strategy 1 and 25% more effective.

Create 10000 gamma distributed values for the costs of strategy 1.

Histogram of the costs of strategy 1 (theoretical distribution):

Create 10000 gamma distributed values for the costs of strategy 2, which is around twice as expensive:

Theoretical distribution of costs of strategy 2:

Calculate the difference among costs of strategy 2 and 1 for each of the 10000 simulated values:

As for many distributions, the difference of two gamma distributions is approximately normally distributed.

Normal distribution for effect estimates (QALYs):

Create 10000 normally distributed values for the utilities of strategy 1:

Theoretical distribution of utilities:

Please pay attention that mu and se of calc_alpha = mu and se of calc_beta.

Create 10000 normally distributed values for the utilities of strategy 2, which is 25% more effective than strategy 1:

Theoretical distribution of utilities:

Calculate the difference among effectiveness (QALYs) of strategy 2 and 1 for each of the 10000 simulated values:

Put QALYs and costs together in a data set and calculate ICER:

The ICER is calculated as (costs2 - costs1) / (qaly2 - qaly1). ICER for the first 6 observations:

[1] -18947  18283   1919  43258    993 -46521

We have 10000 observations with 4 variables:

[1] 10000     4

This is our theoretical distribution with 10000 observations of costs and QALYs for two groups each. Now we randomly draw 100 participants (=hypothetical study) from each of the two samples (1 and 2) (code not shown).

Let’s have a look at the distributions of QALYs and costs of the sample and compare them to the theoretical distributions:

Difference in costs (first 10 observations):

 [1] 1299  669  595  277 3025 1312 -134  568  244  -11

Difference in QALYs (first 10 observations):

 [1] -0.151 -0.013 -0.131  0.368  0.198  0.102  0.227  0.130  0.044 -0.050

Mean difference in QALYs (all studies):

[1] 0.04

ICER:

[1] 19036

Results of bootstrapping those results 1000 times (only rep. 1-5 shown):

$diff_qalys
[1] 0.045

$diff_costs
[1] 643

$IKER
[1] 14315

           [,1]  [,2]  [,3]  [,4]  [,5] 
diff_qalys 0.048 0.03  0.053 0.057 0.058
diff_costs 590   528   538   532   576  
IKER       12302 17507 10199 9320  9891 

    2.5%    97.5% 
 7053.80 27877.62 

Plot:

Calculation of the CEAC

Calculate the differences in QALYs and the differences in costs (not shown):

Calculate the CEAC as a function of the willingness-to-pay (diff_costs / diff_qalys < wtp) with wtp rfom 1 to 50000 in steps of 100 (not shown):

Plot:

Example No. 2

Strategy 2 is approximately twice as effective as strategy 1 and 14% more expensive.

Theoretical distribution of costs and utilities:

ICER (first 10 observations):

 [1]      765    -5810    10817     3876      -73    -8857 -1907159     4636
 [9]    -3258    -5116

Randomly draw 100 participants from each of the two samples (code not shown).

Histograms of the distributions:

Mean difference in costs (first 10 observations):

 [1]  910 -308 1841  -17  626   38 1895 3106  -65 1974

Mean difference in costs (all observations):

[1] 250

Mean difference in QALYs (first 10 observations):

 [1]  0.02896322  0.09741600  0.17084721  0.35093832  0.07707042  0.11079517
 [7]  0.18217476 -0.08235339  0.16867609  0.26622551

ICER:

[1] 1165

Repeat 1000 times (bootstrapping, not shown):

95% confidence interval for the ICER:

   2.5%   97.5% 
-522.05 1575.25 

Plot:

CEAC:

Example No. 3

Strategy 2 is 25% more effective than strategy 1 and equally expensive.

Theoretical distribution of costs and utilities:

ICER (first 10 observations):

 [1]   781  3427  -102 -1126 15743 -3536  -328  5209  2674  -814

Randomly draw 100 participants from each of the two samples (code not shown).

Histograms of the distributions:

Mean difference in costs (first 10 observations):

 [1]  -9   1  -3   9  -5 -10  -5  -9   0  -3

Mean difference in costs (all observations):

[1] -208

ICER:

[1] -2543

Repeat 1000 times (bootstrapping, not shown):

95% confidence interval for the ICER:

     2.5%     97.5% 
-10502.30  10656.85 

Plot:

CEAC (calculation not shown):

Plot:

Example No. 4

Strategy 2 is more effective than strategy 1 and cheaper (strict dominance).

Theoretical distribution of costs and utilities:

ICER (first 10 observations):

 [1]   6662    218    384 -47607   -570    254  -1983  -2808   -138     39

Randomly draw 100 participants from each of the two samples (code not shown).

Histograms of the distributions:

Mean difference in costs (first 10 observations):

 [1]   4  -4   3 -14 -24   5   0  -9   2 -16

Mean difference in costs (all observations):

[1] -272

Mean difference in QALYs (first 10 observations):

 [1] 1 0 0 0 0 0 0 0 0 0

Mean difference in QALYs (all observations):

[1] 0.055

ICER:

[1] -4981

Repeat 1000 times (bootstrapping, not shown):

95% confidence interval for the ICER:

     2.5%     97.5% 
-35057.00  27756.65 

Plot:

CEAC (calculation not shown):

Plot:

Note that the ICER will be negative in most replications (lower costs divided by higher number of QALYs). The CEAC is thus not meaningful!

Example No. 5

Strategy 2 is equally effective as strategy 1 and cheaper (weak dominance).

Theoretical distribution of costs and utilities:

ICER (first 10 observations):

 [1] -2892 -2284  -138 -3360 -2474 -1818 -1722   868  1624  -401

Randomly draw 100 participants from each of the two samples (code not shown).

Histograms of the distributions:

Mean difference in costs (first 10 observations):

 [1]  -2 -11 -13   1 -10  -7   1   1   3  -7

Mean difference in costs (all observations):

[1] -173

Mean difference in QALYs (first 10 observations):

 [1]  0  0  0  0  0 -1  0  0  0  0

Mean difference in QALYs (all observations):

[1] -0.059

ICER:

[1] 2906

Repeat 1000 times (bootstrapping, not shown):

95% confidence interval for the ICER:

     2.5%     97.5% 
-96898.00  90355.25 

Plot:

CEAC (calculation not shown):

Plot:

Example No. 6

Strategy 2 is slightly less effective than strategy 1 but considerably cheaper.

Theoretical distribution of costs and utilities:

ICER (first 10 observations):

 [1] -4005  8202 13109  -454  -914  -178  -659  3738  1778 -5690

Randomly draw 100 participants from each of the two samples (code not shown).

Histograms of the distributions:

Mean difference in costs (first 10 observations):

 [1]   1   2   4 -23  -7 -33  -4  -8  -8 -16

Mean difference in costs (all observations):

[1] -282

Mean difference in QALYs (first 10 observations):

 [1] 0 0 0 0 0 0 0 0 0 0

Mean difference in QALYs (all observations):

[1] -0.069

ICER:

[1] 4090

Repeat 1000 times (bootstrapping, not shown):

95% confidence interval for the ICER:

     2.5%     97.5% 
-31538.42  45392.75 

Plot:

CEAC (calculation not shown):

Plot:

Real-world example

Lan et al. 2022, PMID: 35345479, Cost-Effectiveness of Umeclidinium/Vilanterol versus Salmeterol/Fluticasone in Elderly Patients with Chronic Obstructive Pulmonary Diseases in China

Costs and QALYs directly taken from Table 4.

Standard deviations estimated from the scatter plot and 95% confidence intervals.

Randomly draw 100 participants from each of the two samples (code not shown).

Histograms of the distributions:

Mean difference in costs (first 10 observations):

 [1]   77    7  167   50  -65  111  128  -41   19 -122

Mean difference in costs (all observations):

[1] 2263

Mean difference in QALYs (first 10 observations):

 [1] 0 0 0 0 0 0 0 0 0 0

Mean difference in QALYs (all observations):

[1] -0.067

ICER:

[1] -33880

$diff_qalys
[1] 0.01500783

$diff_costs
[1] -2558.13

$IKER
[1] -170453

           [,1]       [,2]         [,3]       [,4]       [,5]      
diff_qalys 0.03943024 -0.005146858 0.03545826 0.03748577 0.04541466
diff_costs -3193.531  -1363.265    -2633.76   -1316.672  -1998.507 
IKER       -80991.92  264873.2     -74277.77  -35124.58  -44005.76 

Repeat 1000 times (bootstrapping, not shown):

95% confidence interval for the ICER:

     2.5%     97.5% 
-424119.8  542523.8 

Plot:

CEAC (calculation not shown):

Plot: