Probability-Calibrated Tempered MCLE After Positive Phase 2 Selection
Han Zhang
2026-07-31
Overview
A treatment program often advances from Phase 2 only when the observed treatment effect exceeds a prespecified Go threshold. The estimate available after that decision is drawn from a selected subset of trials and therefore tends to overstate the true effect. This matters when the Phase 2 estimate is used to size a Phase 3 trial, value a program, or make another development decision.
Maximum conditional likelihood estimation (MCLE) addresses this bias by conditioning on the event that triggered selection. The adjustment is principled, but the unconstrained estimator is ill-posed when a positive Phase 2 result only barely exceeds the threshold: its estimated treatment effect diverges to negative infinity. Constraining the estimate to be nonnegative hides the divergent tail but can still move many selected estimates below the original decision threshold.
This manuscript develops a tempered MCLE that applies only a fraction of the full conditional-likelihood adjustment. Any fixed amount of tempering makes the estimate finite at the selection boundary. The tuning parameter is calibrated through a minimum fitted pre-selection Go probability, which can be interpreted directly as a lower bound on how probable the observed Go event may be under the fitted model. Theoretical results and simulations under two Phase 2 designs characterize the resulting bias and threshold-crossing behavior. A downstream design evaluation compares retaining versus withdrawing the original Phase 2 Go decision, with the corrected estimate used to size the subsequent Phase 3 trial under both strategies.
1. Positive Phase 2 selection
1.1 Two-sample model
Consider a randomized two-arm Phase 2 trial with a continuous endpoint:
\[ X_{Ti}\overset{iid}{\sim}N(\mu_T,\sigma^2), \qquad i=1,\ldots,n_T, \]
\[ X_{Cj}\overset{iid}{\sim}N(\mu_C,\sigma^2), \qquad j=1,\ldots,n_C. \]
The treatment effect is
\[ \delta=\mu_T-\mu_C. \]
Let
\[ Y=\bar X_T-\bar X_C \]
be the usual estimate and let
\[ S_p^2 = \frac{(n_T-1)S_T^2+(n_C-1)S_C^2}{n_T+n_C-2} \]
be the pooled variance. Write the observed values as \(y\) and \(s_p^2\), and define
\[ \kappa=\frac{1}{n_T}+\frac{1}{n_C}, \qquad \nu=n_T+n_C-2. \]
Under the common-variance normal model,
\[ Y\sim N(\delta,\sigma^2\kappa), \qquad \frac{\nu S_p^2}{\sigma^2}\sim\chi^2_\nu, \]
and \(Y\) is independent of \(S_p^2\).
1.2 Selection rule and inferential objective
Suppose the program advances only when
\[ Y>c, \]
where \(c\) is a prespecified Go threshold. The analysis is therefore conducted after observing the event
\[ A=\{Y>c\}. \]
Throughout, \(y>c\), \(s_p^2>0\), and \(\nu>0\). The objective is to estimate \((\delta,\sigma^2)\) for a selected trial rather than for an unselected sample from the original design.
This distinction is operationally important. Conditional on Go, favorable sampling variation is overrepresented. The unadjusted estimate \(Y\) therefore has positive conditional bias and can yield an optimistic Phase 3 effect assumption. A useful correction must reduce this selection bias without turning a marginally positive result into an unstable point estimate.
2. MCLE as the starting point
2.1 Conditional likelihood
Because the selection event depends on \(Y\) but not on \(S_p^2\), the likelihood after selection is
\[ L_c(\delta,\sigma\mid y,s_p^2,Y>c) = \frac{ f_Y(y\mid\delta,\sigma) f_{S_p^2}(s_p^2\mid\sigma) }{ P_{\delta,\sigma}(Y>c) }. \]
Define
\[ a=\frac{c-\delta}{\sigma\sqrt\kappa}, \qquad \bar\Phi(a)=1-\Phi(a). \]
Then \(P_{\delta,\sigma}(Y>c)=\bar\Phi(a)\), and, up to a parameter-free constant,
\[ L_c(\delta,\sigma) \propto \frac{ \sigma^{-(\nu+1)} \exp\left\{-\dfrac{\nu s_p^2+(y-\delta)^2/\kappa}{2\sigma^2}\right\} }{ \bar\Phi(a) }. \]
The unconstrained MCLE is the maximizer
\[ (\widehat\delta_{\rm MCLE},\widehat\sigma_{\rm MCLE}) = \underset{\delta\in\mathbb R,\,\sigma>0}{\arg\max}\; L_c(\delta,\sigma). \]
The denominator is the entire selection adjustment. It rewards parameter values under which the observed Go event would have been rare. That mechanism corrects selection bias, but it can dominate the data-fitting terms when \(y\) is close to \(c\).
2.2 Two established properties
The following results motivate the modification developed below. They are stated without repeating their earlier proofs. Formal proofs of both results are available in Two-Sample MCLE Adjustment After Positive Phase 2 Selection.
Boundary singularity. For fixed \(s_p^2>0\), as the observed effect approaches the Go threshold from above,
\[ \widehat\delta_{\rm MCLE} = c-\frac{s_p^2\kappa}{y-c} +o\left((y-c)^{-1}\right) \longrightarrow-\infty. \]
This is a property of the conditional-likelihood problem, not a root-finding failure. A result arbitrarily close to \(c\) can be explained by an arbitrarily small fitted Go probability
\[ \widehat p_{\rm Go} =1-\Phi(\widehat a) =1-\Phi\left( \frac{c-\widehat\delta_{ \rm MCLE}} {\widehat\sigma_{\rm MCLE}\sqrt\kappa} \right)\longrightarrow 0, \]
producing an unbounded downward correction. Thus the fitted Go probability is the pre-selection probability of crossing \(c\), evaluated at the fitted \((\widehat\delta,\widehat\sigma)\). The same inverse-gap tail implies that the unconstrained MCLE has no finite conditional mean, although its conditional median remains finite.
The flatness responsible for this behavior can be seen using Example 1. Set
\[ n_T=n_C=50, \quad c=0.33, \quad s_p=1, \quad y=0.331. \]
The observation exceeds the Go threshold by only \(0.001\). For each candidate \(\delta\), define the profile conditional log likelihood
\[ \ell_c^{\rm prof}(\delta) =\max_{\sigma>0}\ell_c(\delta,\sigma). \]
Figure 1 reports this quantity relative to its maximum.
Figure 1. Profile conditional log likelihood for a barely selected result in Example 1.
The maximizer is \(\widehat\delta_{\rm MCLE}\approx-39.67\) with \(\widehat\sigma\approx1\). The profile remains within 0.1 log-likelihood unit of its maximum for candidate effects from approximately \(-60.6\) to \(-24.2\). The fitted standardized selection distance is about \(\widehat a=200\), giving \(\log_{10}(\widehat p_{\rm Go})\approx-8688\). The model is therefore using an effectively impossible fitted Go event to explain an observation that did in fact pass the threshold.
As \(y-c\) is made still smaller, the flat high-likelihood region and its maximizer move farther to the left without limit.
Where near-threshold observations arise. The Example 1 design was simulated again with 10,000 selected trials at each true effect on \(0,0.05,\ldots,1\). An MCLE was classified as ill-posed for this diagnostic when \(\widehat\delta_{\rm MCLE}<-10\). Figure 2 shows the resulting estimated probability across the effect grid.
Â
The estimated probability is 4.33% at \(\delta=0\), decreases to 1.29% at \(\delta=0.35\), and is close to zero at the upper end of the effect grid. We next explain why ill-posed estimates occur more frequently when the true treatment effect is small.
Proposition 1 (conditional density ordering after selection). Fix \(n_T\), \(n_C\), \(\sigma\), and \(c\). For any \(\delta>0\), define
\[ R(y) = \frac{f_0(y\mid Y>c)}{f_\delta(y\mid Y>c)}, \qquad y>c. \]
Then \(R(y)\) is strictly decreasing in \(y\), \(R(c^+)>1\), and \(R(y)\to0\) as \(y\to\infty\). Consequently, there is a unique \(y^\star>c\) such that
\[ f_0(y\mid Y>c)>f_\delta(y\mid Y>c), \qquad c<y<y^\star, \]
whereas
\[ f_0(y\mid Y>c)<f_\delta(y\mid Y>c), \qquad y>y^\star. \]
This is the Main conclusion in Two-Sample MCLE Adjustment After Positive Phase 2 Selection; the proof is provided there and is not repeated here.
Interpretation. When \(\delta=0\), crossing \(Y>c\) is relatively surprising. Conditional on this rare event, the observed \(Y\) is disproportionately likely to lie just above \(c\). These are precisely the observations for which the inverse-gap correction can become extreme, so the probability of an ill-posed MCLE is highest near the null.
When \(\delta>0\), crossing the threshold is less surprising. After selection, the distribution places relatively more mass farther above \(c\), where the MCLE correction is better identified and less sensitive to the observed gap.
The proposition therefore gives a direct selection-bias interpretation: a Phase 2 result that barely exceeds the Go threshold is more consistent with a small true effect plus a favorable selection fluctuation than with a genuinely large effect. This explains the declining pattern in the simulation and why the full-MCLE pathology is concentrated in the low-effect region.
2.3 The nonnegative MCLE used by Kirby et al.
Kirby, Li, and Chuang-Stein (2020) assume a positive treatment effect in their numerical comparison and replace a nonpositive solution by zero (Section 3, p. 684). Let \(\widetilde\delta_{\rm MCLE}\) denote the unconstrained conditional-likelihood solution in their fixed-standard-error calculation. Their plotted estimator is therefore
\[ \widehat\delta_{\rm Kirby} = \max(0,\widetilde\delta_{\rm MCLE}), \]
not the unconstrained MCLE above. This post-estimation truncation makes the reported mean bias finite by replacing the negative tail with a point mass at zero. It should not, however, be interpreted as resolving the underlying instability.
For the positive thresholds considered here,
\[ \max(0,\widetilde\delta_{\rm MCLE})\leq c \quad\Longleftrightarrow\quad \widetilde\delta_{\rm MCLE}\leq c. \]
Truncation at zero therefore does not itself change whether the corrected estimate lies below \(c\). The high probability of this threshold crossing under the alternative \(\delta>c\) is caused by the aggressive MCLE correction before truncation. Kirby et al. similarly found MCLE to be among the methods with the largest probability of moving an initially selected estimate below the threshold under the alternative.
3. Tempered conditional likelihood
3.1 Modified objective
Up to a parameter-free constant, the full conditional log likelihood from Section 2.1 is
\[ \boxed{ \ell_c(\delta,\sigma) = -(\nu+1)\log\sigma -\frac{\nu s_p^2+(y-\delta)^2/\kappa}{2\sigma^2} -\log\bar\Phi(a) }. \]
The first two terms measure agreement with the observed treatment contrast and pooled variance. The final term is the adjustment for observing the data only after the Go event. The instability described in Section 2.2 arises when this selection term dominates the data-fitting terms for \(y\) just above \(c\), i.e., \(\bar\Phi(a)\), the Go probability, is close to zero.
For a prespecified \(0\leq\gamma<1\), temper only that selection term:
\[ \boxed{ \ell_\gamma(\delta,\sigma) = -(\nu+1)\log\sigma -\frac{\nu s_p^2+(y-\delta)^2/\kappa}{2\sigma^2} -\gamma\log\bar\Phi(a) }. \]
The endpoints clarify the role of \(\gamma\):
- \(\gamma=0\) gives the unadjusted likelihood;
- \(\gamma=1\) gives the full conditional likelihood;
- \(0<\gamma<1\) retains a prespecified fraction of the selection adjustment.
Tempering changes only the term responsible for selection correction. It is not a post hoc truncation of the resulting estimate.
3.2 Estimation
Let
\[ \lambda(a)=\frac{\phi(a)}{\bar\Phi(a)} \]
denote the inverse Mills ratio. The score equations are
\[ \frac{y-\delta}{\sigma\sqrt\kappa} =\gamma\lambda(a) \]
and
\[ \sigma^2 = \frac{\nu s_p^2} {\nu+1+\gamma a\lambda(a)-\gamma^2\lambda(a)^2}. \]
Define
\[ m_\gamma(a)=\gamma\lambda(a)-a = \frac{y-c}{\sigma\sqrt\kappa}. \]
The two-parameter problem reduces to the one-dimensional equation
\[ \frac{(y-c)^2}{\kappa} \{\nu+1+\gamma a\lambda(a)-\gamma^2\lambda(a)^2\} -\nu s_p^2m_\gamma(a)^2=0, \]
subject to \(m_\gamma(a)>0\). Once \(\widehat a\) is obtained,
\[ \widehat\sigma_\gamma =\frac{y-c}{\sqrt\kappa\,m_\gamma(\widehat a)}, \qquad \widehat\delta_\gamma =c-\widehat a\widehat\sigma_\gamma\sqrt\kappa. \]
Numerically, \(\sigma\) can be profiled out and \(\ell_\gamma\) maximized over \(a\). This avoids a two-dimensional search and does not require an arbitrary fixed grid.
3.3 Finite behavior at the Go threshold
The central regularization result is that every fixed \(\gamma<1\) produces a finite estimate as \(y\) approaches \(c\). Let
\[ h=y-c, \qquad r=\frac{\nu s_p^2\kappa}{h^2}, \]
and
\[ D_\gamma(a) =\nu+1+\gamma a\lambda(a)-\gamma^2\lambda(a)^2. \]
The score equation becomes
\[ D_\gamma(a)=r\,m_\gamma(a)^2. \]
For a standard normal inverse Mills ratio,
\[ \lambda'(a)=\lambda(a)\{\lambda(a)-a\}<1. \]
The strict upper bound follows from the positive variance of a truncated standard normal;
\[ 1 + a\lambda(a) - \lambda(a)^2 = 1 - \lambda'(a) > 0. \]
Therefore
\[ m_\gamma'(a)=\gamma\lambda'(a)-1<0 \]
for every \(0\leq\gamma<1\). In addition, \(m_\gamma(a)\to+\infty\) as \(a\to-\infty\). At the other end, the Mills expansion gives
\[ \begin{aligned} m_\gamma(a) &=\gamma\left(a+\frac{1}{a}+O(a^{-3})\right)-a\\ &=-(1-\gamma)a+\frac{\gamma}{a}+O(a^{-3}) \longrightarrow-\infty. \end{aligned} \]
as \(a\longrightarrow \infty\). Hence there is a unique finite \(a_\gamma\) satisfying \(m_\gamma(a) = 0\), i.e.,
\[ a_\gamma=\gamma\lambda(a_\gamma). \]
At a feasible fitted solution, \(m_\gamma(\widehat a)>0\). The identity
\[ D_\gamma(a) =\nu+1-\gamma\lambda(a)m_\gamma(a) \]
then yields
\[ 0<D_\gamma(\widehat a)\leq\nu+1. \]
As \(y\downarrow c\), \(r\to\infty\), and consequently
\[ 0<m_\gamma(\widehat a)^2 =\frac{D_\gamma(\widehat a)}{r} \leq\frac{\nu+1}{r} \longrightarrow0. \]
Because \(m_\gamma\) is continuous and has the unique zero \(a_\gamma\),
\[ \widehat a\longrightarrow a_\gamma. \]
Continuity also gives
\[ r\,m_\gamma(\widehat a)^2\longrightarrow\nu+1. \]
Substitution into the recovery equations produces the finite limits
\[ \widehat\sigma_\gamma^2 \longrightarrow \frac{\nu}{\nu+1}s_p^2 \]
and
\[ \boxed{ \widehat\delta_\gamma \longrightarrow c-a_\gamma s_p\sqrt{\frac{\kappa\nu}{\nu+1}} }. \]
Main conclusion (finite boundary correction). For every fixed \(\gamma<1\), an observed effect approaching the Go threshold no longer drives the treatment-effect estimate to \(-\infty\). Instead, \(\widehat\delta_\gamma\) converges to the finite limit displayed above. Thus tempering removes the boundary singularity at the level of the objective function; no post-estimation truncation or numerical fallback is required.
For \(\gamma=0\), \(a_\gamma=0\) and the limiting treatment-effect estimate is \(c\). For every fixed \(0<\gamma<1\), the limit is below \(c\) but remains finite.
3.4 Why full conditioning is different
At \(\gamma=1\),
\[ m_1(a)=\lambda(a)-a>0 \]
for every finite \(a\), so the finite root that bounds the tempered estimator does not exist. The previously established full-MCLE result is
\[ \widehat a\sim\frac{s_p\sqrt\kappa}{y-c}, \qquad \widehat\sigma^2\longrightarrow s_p^2, \]
and
\[ \widehat\delta_1 \sim c-\frac{s_p^2\kappa}{y-c} \longrightarrow-\infty. \]
The distinction is therefore structural: \(m_\gamma\) has a finite zero for \(\gamma<1\) and no finite zero for \(\gamma=1\).
The same contrast is visible directly in the likelihood. At \(y=c\) and fixed \(\sigma\), its \(a\)-dependent component is proportional to
\[ \frac{\phi(a)}{\bar\Phi(a)^\gamma}. \]
Using \(\bar\Phi(a)\sim\phi(a)/a\) as \(a\to\infty\) gives
\[ \frac{\phi(a)}{\bar\Phi(a)^\gamma} \sim a^\gamma\phi(a)^{1-\gamma}. \]
For \(\gamma<1\), the remaining Gaussian factor \(\phi(a)^{1-\gamma}\) drives the objective to zero and prevents escape to the boundary. Full conditioning cancels that factor exactly.
This regularization is not uniform as \(\gamma\uparrow1\). The large-\(a\) approximation
\[ a_\gamma^2\approx\frac{\gamma}{1-\gamma} \]
shows that the finite root, and therefore the magnitude of the maximum possible correction, diverges as tempering disappears.
3.5 Recovery far above the threshold
The \(\delta\) score equation can be written as
\[ \widehat\delta_\gamma = y-\gamma\widehat\sigma_\gamma\sqrt\kappa\, \lambda(\widehat a). \]
When the fitted effect is far above the Go threshold, \(\widehat a\to-\infty\) and therefore \(\lambda(\widehat a)\to0\). The selection correction vanishes, giving
\[ \boxed{ \widehat\delta_\gamma-y\longrightarrow0 }. \]
Thus, the tempered estimator approaches the naive estimator in the region where selection is negligible. If \(Y\) is consistent for the true treatment effect, \(Y\overset{p}{\to}\delta\), it follows immediately that
\[ \boxed{ \widehat\delta_\gamma \overset{p}{\longrightarrow} \delta }. \]
4. Probability calibration for selecting \(\gamma\)
4.1 Why calibrate on a probability scale?
Selecting \(\gamma\) directly is possible but difficult to justify clinically. The instability has a more interpretable representation. Full MCLE permits
\[ P_{\widehat\delta,\widehat\sigma}(Y>c) =\bar\Phi(\widehat a) \longrightarrow0 \]
as \(y\downarrow c\). In other words, it can explain the observed Go decision as an event that was arbitrarily improbable under its own fitted model.
Tempered MCLE is therefore calibrated by specifying the smallest fitted pre-selection Go probability the estimator may imply:
\[ P_{\widehat\delta,\widehat\sigma}(Y>c) \geq p_{\min}. \]
This is an estimator constraint, not a claim about the true data-generating probability. It is not a p-value, a posterior probability that \(\delta>0\), or necessarily a regulatory type-I error rate.
4.2 The implicit probability floor and its calibration
The tempered score equation can be written as
\[ \frac{y-c}{\sigma\sqrt\kappa} =m_\gamma(a) =\gamma\lambda(a)-a. \]
For a fixed \(0\leq\gamma<1\), let \(a_\gamma\) denote the unique zero of \(m_\gamma\). Because \(y>c\), every fitted interior solution satisfies
\[ m_\gamma(\widehat a)>0 \quad\Longrightarrow\quad \widehat a<a_\gamma. \]
Since \(\bar\Phi\) is strictly decreasing,
\[ P_{\widehat\delta,\widehat\sigma}(Y>c) =\bar\Phi(\widehat a) >\bar\Phi(a_\gamma) \equiv p_\gamma. \]
As \(y\downarrow c\), \(\widehat a\uparrow a_\gamma\), so this lower bound is sharp. Every fixed \(\gamma<1\) therefore already defines an implicit fitted probability floor. To calibrate on this scale, reverse the relationship: choose \(p_{\min}\in(0,0.5]\) and set the implicit boundary equal to the corresponding upper-tail normal quantile,
\[ a_\gamma =\bar\Phi^{-1}(p_{\min}) =\Phi^{-1}(1-p_{\min}). \]
Because \(a_\gamma\) is the zero of the tempered score,
\[ m_\gamma(a_\gamma) =\gamma\lambda(a_\gamma)-a_\gamma=0. \]
The resulting likelihood power is
\[ \boxed{ \gamma =\frac{a_\gamma}{\lambda(a_\gamma)} =\frac{a_\gamma p_{\min}}{\phi(a_\gamma)} }. \]
Since \(\lambda(a)>a\) for \(a\geq0\), this mapping automatically gives \(0\leq\gamma<1\). The calibration is built into the score equation rather than imposed as an unrelated numerical restriction.
4.3 Effect-scale interpretation
The calibrated boundary also determines the limiting fitted effect established in section 3.3:
\[ \boxed{ \delta_{\rm lim}(s_p) = c-a_\gamma s_p\sqrt{\frac{\kappa\nu}{\nu+1}} }. \]
Hence a smaller \(p_{\min}\) permits a larger standardized correction, produces a \(\gamma\) closer to one, and moves the procedure toward full MCLE. A larger \(p_{\min}\) gives stronger regularization and moves it toward the unadjusted estimator.
For \(n_T=n_C=50\), \(c=0.33\), \(s_p=1\), and \(\nu=98\), the mapping is shown in Table 1.
Table 1. Probability-floor calibration and boundary limits for Example 1.
| \(p_{\min}\) | \(a_\gamma\) | \(\gamma\) | \(\delta_{\rm lim}(1)\) |
|---|---|---|---|
| 0.010 | 2.326 | 0.873 | -0.133 |
| 0.050 | 1.645 | 0.797 | 0.003 |
| 0.100 | 1.282 | 0.730 | 0.075 |
4.4 Prespecification and boundary cases
The calibration should be fixed before observing \(y\). The direct design question is: what is the smallest fitted pre-selection probability of Go that the estimator should be allowed to assign? After choosing this probability floor, calculate \(\gamma\) from Section 4.2, evaluate operating characteristics over plausible \((\delta,\sigma)\), and report sensitivity analyses at nearby values of \(p_{\min}\).
The endpoints are coherent:
- \(p_{\min}=0.5\) gives \(a_\gamma=0\) and \(\gamma=0\);
- \(p_{\min}\downarrow0\) gives \(a_\gamma\to\infty\) and \(\gamma\uparrow1\);
- \(p_{\min}>0.5\) would require negative \(\gamma\) and lies outside the proposed family.
Choosing \(p_{\min}\) after inspecting the selected estimate would make the regularization data-adaptive and undermine its design-stage interpretation.
5. Simulation framework
Table 2 lists the three simulation questions, which also organize the results in Section 6.
Table 2. Simulation questions, comparisons, and main summaries.
| Question | Comparison | Main summaries |
|---|---|---|
| 1. Does tempering remove the full-MCLE boundary singularity, and how does its behavior change with \(\gamma\)? | Naive, full MCLE, and tempered MCLE at \(p_{\min}=0.01\), 0.05, and 0.10. Larger \(p_{\min}\) gives stronger tempering and smaller \(\gamma\). | Conditional mean and median bias; adjusted estimate below the Phase 2 Go threshold. |
| 2. How does tempered MCLE compare with Kirby’s corrections? | Primary tempered MCLE (\(p_{\min}=0.05\)) versus LWB, percentile bias, CLWB, Kirby MCLE, and full MCLE. | Conditional mean and median bias; adjusted estimate below the Go threshold; RMSE. |
| 3. How should an adjusted estimate be used in practice for Phase 2 and Phase 3? | Both strategies use the adjusted estimate as the Phase 3 design effect. Strategy A withdraws the original Go decision when the adjusted estimate is at or below \(c\); Strategy B retains the original Go decision regardless of the adjusted estimate. | Phase 3 sample size and average achieved power of the combined Phase 2–Phase 3 program under the two strategies. |
5.1 Designs and data generation
Following Kirby et al., we evaluated two two-arm Phase 2 designs that differed only in the Go threshold. Both designs assigned 50 patients to treatment and 50 to control and assumed a common outcome standard deviation of \(\sigma=1\), giving a standard error of 0.20 for \(Y\) and \(\nu=98\) residual degrees of freedom. Example 1 used \(c=0.33\), whereas Example 2 used \(c=0.50\). In each example, the true effect ranged from 0 to 1 in increments of 0.05, with 10,000 selected replicates evaluated at each value.
Both examples evaluate the same three probability floors, \(p_{\min}=0.01\), 0.05, and 0.10. The middle value is used as the primary tempered estimator in comparisons with the Kirby methods.
5.2 Estimators
The primary comparison includes:
- the naive estimator \(Y\);
- unconstrained full MCLE, with \(\delta\in\mathbb R\) and \(\sigma\) jointly estimated;
- probability-calibrated tempered MCLE at \(p_{\min}=0.01\), 0.05, and 0.10, also with \(\sigma\) jointly estimated.
Four methods evaluated by Kirby et al. are included as external comparators:
- Likelihood weighted bias (LWB): subtracts the selection bias averaged over candidate treatment effects using the ordinary likelihood as weights;
- Percentile bias (PB): subtracts a conservative percentile of the bias distribution obtained under a vague uniform prior;
- Conditional likelihood weighted bias (CLWB): averages the selection bias using the conditional likelihood as weights;
- Kirby MCLE: computes the unconstrained conditional-likelihood solution with known \(\sigma\) and then truncates it at zero, \(\widehat\delta_{\rm Kirby}=\max(0,\widetilde\delta_{\rm MCLE})\).
The three bias-subtraction estimators can be written as
\[ \widehat\delta_{\rm LWB}=Y-\operatorname{abias}(Y), \qquad \widehat\delta_{\rm PB}=Y-\operatorname{pbias}(Y), \qquad \widehat\delta_{\rm CLWB}=Y-\operatorname{cbias}(Y). \]
Implicit threshold-crossing boundaries for LWB and CLWB
Both likelihood-weighted corrections have a useful threshold-crossing interpretation in the known-standard-error normal model used by Kirby et al. Write
\[ \tau=\sigma\sqrt\kappa, \qquad Y\mid\delta\sim N(\delta,\tau^2), \qquad t=\frac{y-c}{\tau}. \]
The resulting threshold-crossing boundaries describe when an adjusted estimate falls below the original threshold. Detailed derivations are given in Appendix A: the shared selection-bias calculation in Section A.1, LWB in Section A.2, and CLWB in Sections A.3–A.4.
For LWB, which uses ordinary-likelihood weighting over the unrestricted treatment-effect scale, the correction has a unique standardized boundary. The main results are
\[ \boxed{ \widehat\delta_{\rm LWB}<c \quad\Longleftrightarrow\quad c<Y<c+0.5888455\,\tau } \]
and
\[ \boxed{ \widehat\delta_{\rm LWB}(y) \longrightarrow c-0.9031973\,\tau } \qquad\text{as }y\downarrow c. \]
Thus LWB remains finite for a barely selected observation, with a maximum near-threshold downward correction of approximately \(0.903\tau\).
For CLWB, a parallel continuous calculation gives
\[ \boxed{ \widehat\delta_{\rm CLWB}<c \quad\Longleftrightarrow\quad c<Y<c+0.4081854\,\tau } \]
and
\[ \boxed{ \widehat\delta_{\rm CLWB}(y) \longrightarrow c-0.4621662\,\tau } \qquad\text{as }y\downarrow c. \]
Both methods therefore remain finite at the selection boundary under the formulations used here. CLWB has both the narrower threshold-crossing region and the smaller near-threshold downward correction. These statements concern only whether an estimate lies below \(c\); neither estimator is equivalent to subtracting its boundary constant from every observed \(y\). The narrower CLWB region helps explain why it falls below \(c\) less often while retaining more positive bias when the true effect is weak.
The constants are universal only on the standardized scale and under their stated normal-model, weighting, and integration assumptions. With \(\tau=0.20\), the LWB boundaries are 0.448 and 0.618 for Examples 1 and 2, respectively; the CLWB boundaries are 0.412 and 0.582. A plug-in standard error would make these boundaries random, and changing the integration range, sampling model, or selection rule would generally change the constants.
These built-in operating points help explain the strong empirical performance of LWB and CLWB but are not selected from an externally justified risk tolerance. Tempered MCLE instead allows the correction strength to be prespecified through \(p_{\min}\) and evaluated against design-specific operating characteristics. This flexibility, rather than a claim of uniform dominance over either method, is the principal practical justification for tempering.
All four Kirby methods treat \(\sigma=1\) as known and use a standard error of 0.20. In contrast, the full and tempered MCLEs use the simulated pooled variance and estimate \(\sigma\) jointly; the full MCLE remains unconstrained so that its boundary pathology is visible. All methods are applied to the same selected Monte Carlo trials.
5.3 Simulation summaries and Phase 3 strategies
All summaries are conditional on the original Phase 2 Go event, \(Y>c\). Bias is defined as estimate minus truth and summarized by its mean and median. We also report the probability that an adjusted estimate is below \(c\) and, for the comparison with Kirby’s corrections, RMSE. The full MCLE has neither finite mean bias nor finite RMSE; its median remains well defined. Grid-averaged absolute mean and median biases summarize centering across the 21 equally weighted true effects.
For the Phase 3 comparison, let \(d\) be the method-specific adjusted estimate. A two-arm Phase 3 trial is sized for 80% power at a two-sided 5% level with
\[ n(d) = \left\lceil \frac{2\sigma_{\rm design}^2 \{z_{1-\alpha/2}+z_{1-\beta}\}^2}{d^2} \right\rceil, \]
where \(\sigma_{\rm design}=1\). Both practical strategies use \(d\) as the design effect:
- Strategy A—withdraw Go: cancel Phase 3 when \(d\leq c\); such a replicate has achieved power zero.
- Strategy B—retain Go: keep the original \(Y>c\) decision regardless of \(d\), and use \(d\) only for Phase 3 sizing.
Achieved power is evaluated at the true effect and averaged over all replicates satisfying the original \(Y>c\) rule. We report median sample size per group and average achieved power.
6. Simulation results
Throughout this section, tables and figures intentionally present overlapping results. Tables provide exact values at selected true effects, whereas figures show behavior across the full effect grid.
6.1 Question 1: Does tempering remove the boundary singularity?
The naive estimator, full MCLE, and the three tempered MCLEs are compared first. The probability floors \(p_{\min}=0.01\), 0.05, and 0.10 correspond to \(\gamma=0.873\), 0.797, and 0.730, respectively, in these designs. Tables 3–10 and Figures 3–8 report the results for both examples.
Example 1: \(c=0.33\)
The first comparison isolates the effect of \(p_{\min}\) using the naive, tempered, and unconstrained full-MCLE estimators.
Mean bias
Table 3. Conditional mean bias of the naive, full-MCLE, and tempered estimators in Example 1.
| True effect | Naive | Full MCLE | Tempered 0.01 | Tempered 0.05 | Tempered 0.10 |
|---|---|---|---|---|---|
| 0.00 | 0.414 | Not finite | 0.150 | 0.215 | 0.253 |
| 0.20 | 0.251 | Not finite | 0.037 | 0.087 | 0.118 |
| 0.35 | 0.146 | Not finite | -0.020 | 0.017 | 0.040 |
| 0.50 | 0.068 | Not finite | -0.045 | -0.022 | -0.006 |
| 0.80 | 0.005 | Not finite | -0.018 | -0.014 | -0.012 |
| 1.00 | -0.001 | Not finite | -0.005 | -0.005 | -0.004 |
The unadjusted estimator retains substantial positive bias below the Go threshold. All three tempered estimators reduce it, with smaller \(p_{\min}\) producing stronger correction. Their mean biases approach zero as the true effect moves well above \(c\). Full-MCLE mean bias is marked as not finite rather than represented by an unstable Monte Carlo average.
Median bias
Table 4. Conditional median bias of the naive, full-MCLE, and tempered estimators in Example 1.
| True effect | Naive | Full MCLE | Tempered 0.01 | Tempered 0.05 | Tempered 0.10 |
|---|---|---|---|---|---|
| 0.00 | 0.393 | -0.182 | 0.122 | 0.185 | 0.224 |
| 0.20 | 0.226 | -0.114 | 0.024 | 0.065 | 0.092 |
| 0.35 | 0.122 | -0.070 | -0.012 | 0.011 | 0.027 |
| 0.50 | 0.047 | -0.038 | -0.020 | -0.011 | -0.004 |
| 0.80 | 0.004 | -0.001 | -0.001 | -0.000 | 0.000 |
| 1.00 | -0.000 | -0.000 | -0.000 | -0.000 | -0.000 |
Unlike its mean, the full MCLE median is finite. It overcorrects downward in the low-effect region, while the naive estimator remains positively biased. The tempered estimators interpolate between them. The \(p_{\min}=0.05\) calibration is close to median-unbiased around the design threshold and has only small negative median bias above it.
Adjusted estimate below the Go threshold
Table 5. Conditional probability that an adjusted estimate is below the Go threshold in Example 1.
| True effect | Full MCLE | Tempered 0.01 | Tempered 0.05 | Tempered 0.10 |
|---|---|---|---|---|
| 0.00 | 85.1% | 80.5% | 77.4% | 74.0% |
| 0.20 | 71.0% | 65.4% | 61.4% | 57.6% |
| 0.35 | 55.2% | 48.8% | 45.2% | 41.9% |
| 0.50 | 35.6% | 30.7% | 27.9% | 25.2% |
| 0.80 | 5.1% | 3.8% | 3.3% | 2.9% |
| 1.00 | 0.4% | 0.3% | 0.2% | 0.2% |
The ordering is monotone in the regularization strength. Full MCLE moves the largest percentage of estimates below \(c\); among tempered estimators, \(p_{\min}=0.01\) does so more often than 0.05 or 0.10. Stronger correction is beneficial when the true effect is small but increases the chance that a corrected estimate falls below \(c\) when \(\delta>c\).
Aggregate bias summaries
Table 6. Grid-averaged absolute mean and median bias for the tempered estimators in Example 1.
| \(p_{\min}\) | Mean absolute mean bias | Mean absolute median bias |
|---|---|---|
| 0.01 | 0.040 | 0.023 |
| 0.05 | 0.050 | 0.035 |
| 0.10 | 0.060 | 0.046 |
Across the equally weighted grid, the smaller probability floor gives the best overall centering because it corrects more strongly in the low-effect region. The pointwise tables remain important: this gain is accompanied by more negative bias near and above the Go threshold and more frequent crossings below \(c\).
Example 2: \(c=0.50\)
The second example uses the same \(p_{\min}=0.01\), 0.05, and 0.10 values as Example 1. Only the Go threshold changes, from 0.33 to 0.50. Selection is therefore more stringent and conditional bias is larger at small true effects.
Mean bias
Table 7. Conditional mean bias of the naive, full-MCLE, and tempered estimators in Example 2.
| True effect | Naive | Full MCLE | Tempered 0.01 | Tempered 0.05 | Tempered 0.10 |
|---|---|---|---|---|---|
| 0.00 | 0.564 | Not finite | 0.268 | 0.343 | 0.387 |
| 0.20 | 0.389 | Not finite | 0.133 | 0.196 | 0.233 |
| 0.35 | 0.265 | Not finite | 0.045 | 0.097 | 0.129 |
| 0.50 | 0.161 | Not finite | -0.011 | 0.028 | 0.052 |
| 0.80 | 0.030 | Not finite | -0.034 | -0.021 | -0.013 |
| 1.00 | 0.006 | Not finite | -0.014 | -0.011 | -0.008 |
All three tempered estimators reduce the positive mean bias. A smaller \(p_{\min}\) produces stronger correction and crosses zero earlier, while a larger floor retains more positive bias below the threshold and less negative bias above it. The full-MCLE mean remains undefined.
Median bias
Table 8. Conditional median bias of the naive, full-MCLE, and tempered estimators in Example 2.
| True effect | Naive | Full MCLE | Tempered 0.01 | Tempered 0.05 | Tempered 0.10 |
|---|---|---|---|---|---|
| 0.00 | 0.547 | -0.246 | 0.239 | 0.315 | 0.360 |
| 0.20 | 0.368 | -0.156 | 0.110 | 0.169 | 0.205 |
| 0.35 | 0.243 | -0.117 | 0.033 | 0.076 | 0.105 |
| 0.50 | 0.136 | -0.072 | -0.007 | 0.018 | 0.036 |
| 0.80 | 0.021 | -0.007 | -0.003 | 0.000 | 0.002 |
| 1.00 | 0.003 | -0.000 | 0.000 | 0.001 | 0.001 |
The full MCLE again overcorrects the median at small effects. The tempered estimators interpolate between that behavior and the positively biased naive estimator. The \(p_{\min}=0.05\) estimator has small median bias around and above the Go threshold.
Adjusted estimate below the Go threshold
Table 9. Conditional probability that an adjusted estimate is below the Go threshold in Example 2.
| True effect | Full MCLE | Tempered 0.01 | Tempered 0.05 | Tempered 0.10 |
|---|---|---|---|---|
| 0.00 | 92.2% | 88.4% | 86.0% | 83.0% |
| 0.20 | 83.5% | 78.5% | 74.6% | 71.2% |
| 0.35 | 72.8% | 67.2% | 63.3% | 59.2% |
| 0.50 | 57.1% | 51.1% | 46.9% | 43.2% |
| 0.80 | 18.1% | 14.9% | 13.0% | 11.3% |
| 1.00 | 3.9% | 3.0% | 2.5% | 2.2% |
The ordering is the same as in Example 1: stronger correction moves more estimates below \(c\). Every method crosses the threshold more often here because the higher threshold creates more severe selection.
Aggregate bias summaries
Table 10. Grid-averaged absolute mean and median bias for the tempered estimators in Example 2.
| \(p_{\min}\) | Mean absolute mean bias | Mean absolute median bias |
|---|---|---|
| 0.01 | 0.074 | 0.054 |
| 0.05 | 0.097 | 0.078 |
| 0.10 | 0.113 | 0.096 |
As in Example 1, \(p_{\min}=0.01\) gives the smallest aggregate bias over the equally weighted grid, at the cost of more negative bias above the threshold and more frequent crossings below \(c\).
Across both examples, every tempered estimator has finite mean and median bias, unlike the singular full MCLE. Smaller \(p_{\min}\), and hence larger \(\gamma\), gives a stronger selection adjustment: it reduces positive bias when the true effect is weak but moves more adjusted estimates below \(c\).
6.2 Question 2: How does tempered MCLE compare with Kirby’s corrections?
The primary tempered estimator, \(p_{\min}=0.05\), is compared with LWB, percentile bias, CLWB, Kirby MCLE, and full MCLE on the same selected trials. Tables 11–20 and Figures 9–16 report the results for both examples.
Example 1: \(c=0.33\)
The next tables compare the primary \(p_{\min}=0.05\) tempered estimator with LWB, percentile bias, CLWB, Kirby MCLE, and unconstrained full MCLE on the same selected trials.
Mean bias
Table 11. Conditional mean-bias comparison with Kirby’s corrections in Example 1.
| True effect | LWB | Percentile | Conditional LWB | Kirby MCLE | Tempered 0.05 | Full MCLE |
|---|---|---|---|---|---|---|
| 0.00 | 0.276 | 0.194 | 0.331 | 0.109 | 0.215 | Not finite |
| 0.20 | 0.129 | 0.054 | 0.174 | -0.010 | 0.087 | Not finite |
| 0.35 | 0.042 | -0.023 | 0.077 | -0.063 | 0.017 | Not finite |
| 0.50 | -0.014 | -0.066 | 0.010 | -0.078 | -0.022 | Not finite |
| 0.80 | -0.024 | -0.042 | -0.020 | -0.026 | -0.014 | Not finite |
| 1.00 | -0.011 | -0.015 | -0.011 | -0.007 | -0.005 | Not finite |
Kirby MCLE is the strongest finite correction near the lower end: its mean bias is 0.109 at the null but becomes negative by \(\delta=0.20\). Its finite mean depends on truncation at zero. Tempered MCLE corrects less abruptly, retains positive mean bias below the threshold, and remains close to zero above it. LWB and CLWB correct less at small effects, whereas percentile bias crosses zero earlier than tempered MCLE.
Median bias
Table 12. Conditional median-bias comparison with Kirby’s corrections in Example 1.
| True effect | LWB | Percentile | Conditional LWB | Kirby MCLE | Tempered 0.05 | Full MCLE |
|---|---|---|---|---|---|---|
| 0.00 | 0.248 | 0.161 | 0.307 | 0.000 | 0.185 | -0.182 |
| 0.20 | 0.098 | 0.018 | 0.145 | -0.118 | 0.065 | -0.114 |
| 0.35 | 0.015 | -0.055 | 0.048 | -0.073 | 0.011 | -0.070 |
| 0.50 | -0.029 | -0.083 | -0.014 | -0.040 | -0.011 | -0.038 |
| 0.80 | -0.013 | -0.022 | -0.015 | -0.001 | -0.000 | -0.001 |
| 1.00 | -0.004 | -0.004 | -0.004 | -0.000 | -0.000 | -0.000 |
Truncation at zero gives Kirby MCLE median bias exactly zero at the null, but its median is strongly negatively biased at positive effects below the threshold. Its curve then closely follows the full-MCLE median. Tempered MCLE remains positively biased at small effects but is better centered around and above the threshold.
Adjusted estimate below the Go threshold
Table 13. Conditional probability that an adjusted estimate is below the Go threshold for the Kirby comparison in Example 1.
| True effect | LWB | Percentile | Conditional LWB | Kirby MCLE | Tempered 0.05 | Full MCLE |
|---|---|---|---|---|---|---|
| 0.00 | 74.7% | 86.3% | 60.6% | 85.4% | 77.4% | 85.1% |
| 0.20 | 58.4% | 72.3% | 44.3% | 71.5% | 61.4% | 71.0% |
| 0.35 | 42.4% | 56.6% | 30.2% | 55.4% | 45.2% | 55.2% |
| 0.50 | 25.4% | 36.8% | 17.3% | 35.8% | 27.9% | 35.6% |
| 0.80 | 3.0% | 5.4% | 1.7% | 5.1% | 3.3% | 5.1% |
| 1.00 | 0.2% | 0.5% | 0.1% | 0.4% | 0.2% | 0.4% |
Kirby MCLE, percentile bias, and full MCLE have nearly identical threshold-crossing probabilities. Tempered MCLE falls below \(c\) slightly more often than LWB but materially less often than those three methods. CLWB has the lowest probability, consistent with its weaker correction and larger residual bias below the threshold.
Aggregate bias comparison
Table 14. Grid-averaged absolute mean and median bias for the Kirby comparison in Example 1.
| Method | Mean absolute mean bias | Mean absolute median bias |
|---|---|---|
| Naive | 0.125 | 0.111 |
| Likelihood weighted bias | 0.070 | 0.057 |
| Percentile bias | 0.060 | 0.052 |
| Conditional likelihood weighted bias | 0.087 | 0.075 |
| Kirby MCLE | 0.045 | 0.041 |
| Tempered MCLE (\(p_{\min}=0.05\)) | 0.050 | 0.035 |
| Full MCLE | Not finite | 0.056 |
Kirby MCLE has the smallest aggregate absolute mean bias, while tempered MCLE has the smallest aggregate absolute median bias. The aggregate result for Kirby MCLE reflects a mixture of positive bias at the null and negative bias over much of the positive-effect range, together with truncation at zero; it should not be interpreted as uniform pointwise superiority.
Conditional treatment-effect RMSE
| True effect | Naive | Tempered 0.05 | LWB | Percentile | CLWB | Kirby MCLE | Full MCLE |
|---|---|---|---|---|---|---|---|
| 0.00 | 0.420 | 0.266 | 0.296 | 0.230 | 0.342 | 0.204 | Not finite |
| 0.05 | 0.379 | 0.243 | 0.262 | 0.203 | 0.304 | 0.198 | Not finite |
| 0.10 | 0.340 | 0.225 | 0.232 | 0.183 | 0.268 | 0.201 | Not finite |
| 0.15 | 0.303 | 0.213 | 0.208 | 0.172 | 0.236 | 0.211 | Not finite |
| 0.20 | 0.270 | 0.206 | 0.189 | 0.168 | 0.209 | 0.224 | Not finite |
| 0.25 | 0.239 | 0.206 | 0.178 | 0.173 | 0.187 | 0.238 | Not finite |
| 0.30 | 0.212 | 0.206 | 0.171 | 0.180 | 0.171 | 0.252 | Not finite |
| 0.35 | 0.192 | 0.212 | 0.173 | 0.194 | 0.163 | 0.267 | Not finite |
| 0.40 | 0.177 | 0.221 | 0.181 | 0.210 | 0.164 | 0.279 | Not finite |
| 0.45 | 0.172 | 0.227 | 0.190 | 0.223 | 0.170 | 0.286 | Not finite |
| 0.50 | 0.168 | 0.233 | 0.200 | 0.237 | 0.178 | 0.294 | Not finite |
| 0.55 | 0.170 | 0.235 | 0.207 | 0.244 | 0.186 | 0.289 | Not finite |
| 0.60 | 0.174 | 0.236 | 0.215 | 0.251 | 0.196 | 0.286 | Not finite |
| 0.65 | 0.181 | 0.236 | 0.221 | 0.254 | 0.205 | 0.279 | Not finite |
| 0.70 | 0.185 | 0.233 | 0.224 | 0.254 | 0.210 | 0.270 | Not finite |
| 0.75 | 0.192 | 0.230 | 0.226 | 0.252 | 0.215 | 0.259 | Not finite |
| 0.80 | 0.194 | 0.225 | 0.225 | 0.248 | 0.216 | 0.247 | Not finite |
| 0.85 | 0.194 | 0.217 | 0.219 | 0.238 | 0.213 | 0.233 | Not finite |
| 0.90 | 0.199 | 0.216 | 0.220 | 0.236 | 0.216 | 0.228 | Not finite |
| 0.95 | 0.199 | 0.210 | 0.215 | 0.227 | 0.213 | 0.218 | Not finite |
| 1.00 | 0.200 | 0.208 | 0.213 | 0.222 | 0.212 | 0.212 | Not finite |
At the first grid point above the Go threshold, \(\delta=0.35\), tempered MCLE has RMSE 0.212 compared with 0.173 for LWB and 0.163 for CLWB. Tempered MCLE has less mean bias than either method at this point, but its larger standard deviation makes its RMSE larger. Relative to LWB, tempered MCLE has lower RMSE at the weakest effects (\(\delta=0,0.05,0.10\)), higher RMSE through most of the middle of the grid, and similar RMSE at the strongest effects. It has greater RMSE than LWB at 14 of the 21 equally weighted grid points; this count is a description of the chosen grid, not a claim of uniform dominance.
Example 2: \(c=0.50\)
The primary \(p_{\min}=0.05\) estimator is next compared with the same external methods used in Example 1.
Mean bias
Table 16. Conditional mean-bias comparison with Kirby’s corrections in Example 2.
| True effect | LWB | Percentile | Conditional LWB | Kirby MCLE | Tempered 0.05 | Full MCLE |
|---|---|---|---|---|---|---|
| 0.00 | 0.418 | 0.331 | 0.479 | 0.123 | 0.343 | Not finite |
| 0.20 | 0.254 | 0.172 | 0.307 | 0.001 | 0.196 | Not finite |
| 0.35 | 0.141 | 0.066 | 0.188 | -0.071 | 0.097 | Not finite |
| 0.50 | 0.054 | -0.012 | 0.091 | -0.101 | 0.028 | Not finite |
| 0.80 | -0.026 | -0.063 | -0.014 | -0.064 | -0.021 | Not finite |
| 1.00 | -0.020 | -0.035 | -0.017 | -0.021 | -0.011 | Not finite |
Kirby MCLE gives the strongest finite correction: it has the least positive bias at the null, is essentially unbiased at \(\delta=0.20\), and is negatively biased at larger effects. Tempered MCLE retains more positive bias below the threshold but is better centered around and above it. LWB and CLWB are less aggressive, while percentile bias lies between tempered and Kirby MCLE near the threshold.
Median bias
Table 17. Conditional median-bias comparison with Kirby’s corrections in Example 2.
| True effect | LWB | Percentile | Conditional LWB | Kirby MCLE | Tempered 0.05 | Full MCLE |
|---|---|---|---|---|---|---|
| 0.00 | 0.394 | 0.303 | 0.459 | 0.000 | 0.315 | -0.246 |
| 0.20 | 0.226 | 0.140 | 0.283 | -0.160 | 0.169 | -0.156 |
| 0.35 | 0.113 | 0.032 | 0.161 | -0.117 | 0.076 | -0.117 |
| 0.50 | 0.027 | -0.045 | 0.061 | -0.072 | 0.018 | -0.072 |
| 0.80 | -0.024 | -0.055 | -0.021 | -0.009 | 0.000 | -0.007 |
| 1.00 | -0.011 | -0.017 | -0.012 | -0.000 | 0.001 | -0.000 |
At the null, the median Kirby estimate is zero because of the constraint. At \(\delta=0.20\), its median bias is \(-0.160\), reflecting the continuing influence of the point mass at zero. Tempered MCLE avoids this point mass and has substantially less negative median bias around and above the threshold.
Adjusted estimate below the Go threshold
Table 18. Conditional probability that an adjusted estimate is below the Go threshold for the Kirby comparison in Example 2.
| True effect | LWB | Percentile | Conditional LWB | Kirby MCLE | Tempered 0.05 | Full MCLE |
|---|---|---|---|---|---|---|
| 0.00 | 83.7% | 92.8% | 71.5% | 92.3% | 86.0% | 92.2% |
| 0.20 | 71.8% | 84.5% | 57.7% | 83.6% | 74.6% | 83.5% |
| 0.35 | 60.0% | 74.1% | 45.9% | 73.1% | 63.3% | 72.8% |
| 0.50 | 44.0% | 58.0% | 31.6% | 57.2% | 46.9% | 57.1% |
| 0.80 | 11.8% | 18.8% | 7.2% | 18.2% | 13.0% | 18.1% |
| 1.00 | 2.2% | 4.0% | 1.3% | 3.9% | 2.5% | 3.9% |
Kirby MCLE, percentile bias, and full MCLE again have nearly identical threshold-crossing probabilities. Tempered MCLE is close to LWB and falls below \(c\) materially less often than those three methods. CLWB has the lowest probability but retains the greatest positive bias below the threshold.
Aggregate bias comparison
Table 19. Grid-averaged absolute mean and median bias for the Kirby comparison in Example 2.
| Method | Mean absolute mean bias | Mean absolute median bias |
|---|---|---|
| Naive | 0.207 | 0.190 |
| Likelihood weighted bias | 0.127 | 0.111 |
| Percentile bias | 0.102 | 0.091 |
| Conditional likelihood weighted bias | 0.153 | 0.138 |
| Kirby MCLE | 0.069 | 0.066 |
| Tempered MCLE (\(p_{\min}=0.05\)) | 0.097 | 0.078 |
| Full MCLE | Not finite | 0.092 |
Kirby MCLE has the smallest aggregate bias summaries in Example 2, but its pointwise behavior is asymmetric: positive bias at the null is exchanged for negative bias at positive effects, and the zero constraint determines its low-effect median. Tempered MCLE is less aggressive and better centered near the Go threshold, while preserving a smooth, unconstrained estimate.
Conditional treatment-effect RMSE
| True effect | Naive | Tempered 0.05 | LWB | Percentile | CLWB | Kirby MCLE | Full MCLE |
|---|---|---|---|---|---|---|---|
| 0.00 | 0.567 | 0.369 | 0.427 | 0.346 | 0.483 | 0.233 | Not finite |
| 0.05 | 0.524 | 0.338 | 0.388 | 0.311 | 0.442 | 0.232 | Not finite |
| 0.10 | 0.479 | 0.304 | 0.347 | 0.273 | 0.398 | 0.229 | Not finite |
| 0.15 | 0.437 | 0.277 | 0.311 | 0.242 | 0.359 | 0.236 | Not finite |
| 0.20 | 0.397 | 0.254 | 0.277 | 0.215 | 0.320 | 0.247 | Not finite |
| 0.25 | 0.355 | 0.230 | 0.243 | 0.189 | 0.281 | 0.258 | Not finite |
| 0.30 | 0.316 | 0.215 | 0.215 | 0.173 | 0.247 | 0.274 | Not finite |
| 0.35 | 0.282 | 0.207 | 0.195 | 0.167 | 0.218 | 0.290 | Not finite |
| 0.40 | 0.251 | 0.205 | 0.181 | 0.169 | 0.194 | 0.305 | Not finite |
| 0.45 | 0.221 | 0.205 | 0.172 | 0.176 | 0.175 | 0.319 | Not finite |
| 0.50 | 0.201 | 0.211 | 0.173 | 0.189 | 0.167 | 0.328 | Not finite |
| 0.55 | 0.185 | 0.219 | 0.179 | 0.206 | 0.165 | 0.339 | Not finite |
| 0.60 | 0.170 | 0.223 | 0.184 | 0.217 | 0.164 | 0.341 | Not finite |
| 0.65 | 0.164 | 0.227 | 0.192 | 0.229 | 0.170 | 0.339 | Not finite |
| 0.70 | 0.168 | 0.233 | 0.204 | 0.240 | 0.182 | 0.332 | Not finite |
| 0.75 | 0.175 | 0.239 | 0.215 | 0.251 | 0.195 | 0.325 | Not finite |
| 0.80 | 0.178 | 0.235 | 0.218 | 0.251 | 0.200 | 0.306 | Not finite |
| 0.85 | 0.185 | 0.236 | 0.225 | 0.256 | 0.210 | 0.297 | Not finite |
| 0.90 | 0.190 | 0.231 | 0.226 | 0.253 | 0.214 | 0.277 | Not finite |
| 0.95 | 0.194 | 0.228 | 0.226 | 0.250 | 0.217 | 0.261 | Not finite |
| 1.00 | 0.196 | 0.223 | 0.224 | 0.244 | 0.217 | 0.248 | Not finite |
At the Go threshold, \(\delta=0.50\), tempered MCLE has RMSE 0.211 compared with 0.173 for LWB and 0.167 for CLWB. As in Example 1, the smaller tempered mean bias is offset by a larger standard deviation. Tempered MCLE has lower RMSE than LWB from \(\delta=0\) through 0.30, whereas LWB generally has lower RMSE over the middle and upper portions of the grid; the two are nearly identical at \(\delta=1\). Tempered MCLE has greater RMSE than LWB at 13 of the 21 equally weighted grid points. Which method has lower RMSE over an application-specific effect distribution therefore depends on how the plausible true effects are weighted.
The comparison shows a bias–variance tradeoff rather than uniform dominance. Kirby MCLE gives the smallest grid-averaged absolute mean bias in both examples, while tempered MCLE gives the smallest grid-averaged absolute median bias in Example 1. Around both Go thresholds, LWB and CLWB have smaller RMSE than tempered MCLE.
6.3 Question 3: Should the original Go decision be withdrawn after adjustment?
Both strategies use the adjusted estimate as the Phase 3 design effect. Under Strategy A, Phase 3 is cancelled when the adjusted estimate is at or below \(c\); its achieved power is then set to zero. Under Strategy B, the original \(Y>c\) decision is retained regardless of the adjusted estimate, which is used only for Phase 3 sizing. Tables 21–24 and Figure 17 summarize Phase 3 sample size and achieved power.
Phase 3 sample size
The table reports median sample size per group. Strategy B is shown because it retains every original Go program; whenever Strategy A launches, it uses the same method-specific sample-size calculation.
| True effect | Oracle | Naive | Tempered 0.05 | LWB | Percentile | CLWB | Kirby MCLE | Full MCLE |
|---|---|---|---|---|---|---|---|---|
| 0.00 | Not finite | 102 | 457 | 255 | 603.5 | 167 | Not finite | Not finite |
| 0.20 | 393 | 87 | 224 | 177.5 | 331.5 | 132 | 2334 | 2140.5 |
| 0.35 | 129 | 71 | 121 | 118 | 181 | 100 | 204.5 | 200 |
| 0.50 | 63 | 53 | 66 | 71 | 91 | 67 | 75 | 74 |
| 0.80 | 25 | 25 | 25 | 26 | 26 | 26 | 25 | 25 |
| 1.00 | 16 | 16 | 16 | 16 | 16 | 16 | 16 | 16 |
| True effect | Oracle | Naive | Tempered 0.05 | LWB | Percentile | CLWB | Kirby MCLE | Full MCLE |
|---|---|---|---|---|---|---|---|---|
| 0.00 | Not finite | 53 | 159 | 102 | 171 | 75 | Not finite | Not finite |
| 0.20 | 393 | 49 | 116 | 87 | 136 | 68 | 9738.5 | 8157.5 |
| 0.35 | 129 | 45 | 87 | 74 | 108 | 61 | 290.5 | 289.5 |
| 0.50 | 63 | 39 | 59 | 57 | 76 | 50 | 86 | 86 |
| 0.80 | 25 | 24 | 25 | 27 | 29 | 26 | 26 | 25 |
| 1.00 | 16 | 16 | 16 | 17 | 17 | 17 | 16 | 16 |
At the threshold-focused grid points, percentile correction recommends larger median trials than the oracle: 181 versus 129 patients per group in Example 1, and 76 versus 63 in Example 2.
Average achieved Phase 3 power
The following tables give average achieved power under Strategy B.
| True effect | Oracle | Naive | Tempered 0.05 | LWB | Percentile | CLWB | Kirby MCLE | Full MCLE |
|---|---|---|---|---|---|---|---|---|
| 0.00 | Not defined | 2.5% | 2.4% | 2.5% | 2.5% | 2.5% | 1.0% | 1.0% |
| 0.20 | 80.1% | 26.1% | 58.0% | 50.4% | 66.7% | 37.9% | 27.9% | 28.0% |
| 0.35 | 80.3% | 54.9% | 71.1% | 72.0% | 78.6% | 67.2% | 46.2% | 46.2% |
| 0.50 | 80.1% | 71.3% | 76.7% | 78.6% | 81.7% | 77.4% | 61.3% | 61.1% |
| 0.80 | 80.7% | 79.0% | 79.5% | 80.3% | 80.9% | 80.3% | 78.0% | 78.0% |
| 1.00 | 80.7% | 80.3% | 80.4% | 80.7% | 80.8% | 80.7% | 80.3% | 80.3% |
| True effect | Oracle | Naive | Tempered 0.05 | LWB | Percentile | CLWB | Kirby MCLE | Full MCLE |
|---|---|---|---|---|---|---|---|---|
| 0.00 | Not defined | 2.5% | 2.5% | 2.5% | 2.5% | 2.5% | 1.0% | 1.0% |
| 0.20 | 80.1% | 16.3% | 39.7% | 26.4% | 39.6% | 20.9% | 23.4% | 23.5% |
| 0.35 | 80.3% | 37.7% | 64.3% | 56.3% | 69.4% | 47.7% | 39.1% | 39.0% |
| 0.50 | 80.1% | 58.9% | 74.1% | 73.1% | 79.5% | 68.5% | 54.0% | 54.0% |
| 0.80 | 80.7% | 77.6% | 79.7% | 80.9% | 82.4% | 80.5% | 75.0% | 75.0% |
| 1.00 | 80.7% | 80.0% | 80.4% | 81.0% | 81.5% | 81.0% | 79.6% | 79.5% |
At \(\delta=0.35\) in Example 1, average achieved power is 54.9% for naive sizing, 71.1% for tempered MCLE, 72.0% for LWB, and 78.6% for percentile correction, compared with 80.3% for the oracle. At \(\delta=0.50\) in Example 2, the corresponding values are 58.9%, 74.1%, 73.1%, 79.5%, and 80.1%.
The figure directly compares Strategy B on the horizontal axis with Strategy A on the vertical axis. Each path follows the true-effect grid from 0 to 1. The enlarged dot marks \(\delta=0.35\) in Example 1, the grid point nearest \(c=0.33\), and \(\delta=0.50=c\) in Example 2.
Near the thresholds, withdrawing Go substantially reduces average achieved power. The Strategy B versus Strategy A values are 71.1% versus 28.0% for tempered MCLE, 72.0% versus 31.2% for LWB, and 67.2% versus 39.2% for CLWB in Example 1. In Example 2 they are 74.1% versus 29.6%, 73.1% versus 32.6%, and 68.5% versus 40.8%, respectively.
Retaining the original Go decision does not obligate a sponsor to launch a Phase 3 trial of any size produced mechanically by the adjusted estimate. A very small adjusted design effect implies a very large trial that may be economically infeasible and therefore may never launch after the usual feasibility assessment. Such extreme downward adjustments occur most often when the true effect is weak, but they can also occur by chance when the true effect is clinically meaningful. Strategy A would automatically discard the latter programs. Under Strategy B, a small adjusted estimate can instead be handled by planning at a prespecified minimum clinically meaningful or commercially relevant effect, or by incorporating a Phase 3 sample-size reassessment. These options address planning uncertainty without turning the adjusted estimate into a second Phase 2 Go/No-go rule.
We therefore recommend Strategy B in practice: retain the prespecified Phase 2 decision based on the observed, unadjusted \(Y\), and use the adjusted estimate as a candidate Phase 3 design effect. An adjusted estimate below \(c\) is useful as a diagnostic of correction strength, but not as a second Go/No-go rule.
7. Interpretation and practical use
7.1 What tempering accomplishes
Tempering addresses the specific failure of full conditional likelihood. It prevents the fitted pre-selection Go probability from collapsing to zero and thereby replaces an inverse-gap singularity with a finite, interpretable threshold limit. The estimate remains unconstrained on the treatment-effect scale; finiteness is achieved through the objective rather than through truncation at zero.
The method also preserves the desirable large-signal behavior of MCLE. When the fitted Go probability is close to one, the selection term contributes little to either the objective or its score, so the correction vanishes. The simulation confirms that tempered mean and median biases become small as the true effect moves well above the threshold.
LWB provides a strong benchmark because likelihood averaging regularizes its correction and, in the canonical known-variance normal model, implicitly places its threshold-crossing boundary 0.5888455 standard errors above \(c\). Tempering does not rely on this single built-in operating point. Its probability floor allows correction strength to be prespecified using the intended design, stakeholder tolerance for downward correction, and externally supported plausibility judgments, with sensitivity analyses expressed on the same probability scale.
7.2 What tempering does not guarantee
Probability calibration does not make the estimator exactly unbiased at every treatment effect. A larger \(p_{\min}\) deliberately accepts more residual positive bias in exchange for fewer crossings below \(c\). A smaller \(p_{\min}\) improves centering near weak effects but moves the estimator closer to full MCLE.
Nor does the simulation establish uniform superiority over LWB, percentile bias, CLWB, or Kirby MCLE. In the primary \(p_{\min}=0.05\) comparison, Kirby MCLE has the smallest aggregate absolute mean bias in both examples, whereas tempered MCLE has the smallest aggregate absolute median bias in Example 1. Kirby MCLE achieves this performance through post-estimation truncation at zero, which creates a point mass at zero and negative bias across much of the positive effect range. Tempered MCLE instead remains untruncated and falls below \(c\) less often than Kirby MCLE, percentile bias, or full MCLE. Method choice must therefore reflect the desired pointwise centering, tolerance for downward correction, and acceptability of truncating the estimator at zero.
RMSE adds a variance criterion to this comparison. Around both Go thresholds, the primary tempered estimator has less mean bias than LWB and CLWB but greater variance, producing larger RMSE. Tempered MCLE has a relative RMSE advantage over LWB mainly in the weakest-effect region, where reducing selection bias is most consequential. This bias-variance tradeoff is distinct from the full-MCLE instability: tempered MCLE has a finite second moment, whereas full MCLE does not.
7.3 Separating the Phase 3 launch and planning roles
The downstream analysis supports assigning different roles to the two estimates. Strategy A, which withdraws the original Go decision when the adjusted estimate is below the same threshold, creates method-dependent launch populations. Near both Go thresholds, assigning zero power to cancelled trials substantially reduced average achieved power for every correction method.
In practice, we therefore recommend that the prespecified rule based on the observed, unadjusted \(Y\) determine the Phase 2 Go/No-go decision. Among trials that pass that rule, the adjusted estimate may instead inform the Phase 3 design effect. An adjusted estimate below \(c\) is a diagnostic of adjustment strength, not a second decision. This separation maintains a common selected population and directly targets the post-selection planning use case. The sample-size and power results are descriptive; they are not used to rank methods through a single oracle-distance loss.
8. Conclusion
Positive Phase 2 selection creates a genuine need for bias correction, but the unconstrained full MCLE is too unstable near the Go threshold to provide a point estimator with a well-defined mean bias. Truncating negative estimates at zero changes the reported distribution without changing whether the corrected estimate lies below a positive threshold.
Probability-calibrated tempered MCLE modifies the source of the instability. For every fixed \(\gamma<1\), the estimator has a finite threshold limit and a well-defined mean bias. Calibrating \(\gamma\) through \(p_{\min}\) makes the regularization directly interpretable as a lower bound on the fitted pre-selection probability of a Go result.
The simulations support tempered MCLE as a stable compromise rather than a universally dominant estimator. It substantially reduces positive selection bias, avoids the full-MCLE singularity, and permits explicit control of the bias-correction tradeoff. When correction is used for Phase 3 planning, the original Go rule should remain fixed and all selected trials should be compared on a common population. Re-screening by the corrected estimate answers a different portfolio question and should include the zero contribution of programs that no longer launch, rather than report only conditional power among launches.
Appendix A. Derivations of the LWB and CLWB threshold-crossing boundaries
A.2 LWB correction, threshold-crossing boundary, and threshold limit
For a candidate effect \(u\), the ordinary likelihood is proportional to
\[ L(u;y)\propto\phi\left(\frac{y-u}{\tau}\right). \]
With unrestricted integration over the treatment-effect scale, its normalized form is the \(N(y,\tau^2)\) density for \(u\). The LWB correction is therefore
\[ A_L(y) =E\left[ \tau\lambda\left(\frac{c-U}{\tau}\right) \right], \qquad U\sim N(y,\tau^2), \]
and \(\widehat\delta_{\rm LWB}(y)=y-A_L(y)\). Set
\[ t=\frac{y-c}{\tau} \]
and write \(U=y+\tau Z\), where \(Z\sim N(0,1)\). Then
\[ A_L(y)=\tau H(t), \qquad H(t) =E\{\lambda(-t-Z)\} =\int_{-\infty}^{\infty}\lambda(-t-z)\phi(z)\,dz, \]
so that
\[ \boxed{ \widehat\delta_{\rm LWB}(y) =c+\tau\{t-H(t)\} }. \]
The inverse Mills ratio satisfies
\[ \lambda'(a)=\lambda(a)\{\lambda(a)-a\}>0. \]
Differentiating under the expectation therefore gives
\[ H'(t)=-E\{\lambda'(-t-Z)\}<0. \]
Thus \(H\) is strictly decreasing. Moreover, \(H(0)>0\) and \(H(t)\to0\) as \(t\to\infty\). The LWB estimate is below \(c\) exactly when
\[ t<H(t), \]
so the strictly increasing identity function and strictly decreasing \(H\) have a unique crossing. Numerical evaluation gives
\[ t_*=H(t_*), \qquad t_*=0.5888455. \]
It follows, apart from the probability-zero equality event, that
\[ \widehat\delta_{\rm LWB}<c \quad\Longleftrightarrow\quad c<Y<c+0.5888455\,\tau. \]
Finally,
\[ H(0) =\int_{-\infty}^{\infty}\lambda(-z)\phi(z)\,dz =0.9031973. \]
Continuity of \(H\) at zero yields
\[ \widehat\delta_{\rm LWB}(y) =c+\tau\{t-H(t)\} \longrightarrow c-0.9031973\,\tau \qquad\text{as }y\downarrow c. \]
A.3 Continuous CLWB correction
For a candidate effect \(u\), the conditional likelihood after selection is
\[ L_c(u;y) \propto \frac{ \phi\{(y-u)/\tau\} }{ \bar\Phi\{(c-u)/\tau\} }, \qquad y>c. \]
Let \(\Omega=[c,\infty)\) denote the candidate-effect domain. The CLWB correction and estimator are
\[ A_C(y) = \frac{ \int_{\Omega}B(u)L_c(u;y)\,du }{ \int_{\Omega}L_c(u;y)\,du }, \qquad \widehat\delta_{\rm CLWB}(y)=y-A_C(y). \]
Define the standardized variables
\[ t=\frac{y-c}{\tau}, \qquad x=\frac{u-c}{\tau}. \]
Then \(x\geq0\) and
\[ \bar\Phi\left(\frac{c-u}{\tau}\right)=\Phi(x), \qquad \frac{B(u)}{\tau} =\frac{\phi(x)}{\Phi(x)} =\lambda(-x). \]
Substitution into the weighted average gives
\[ A_C(y)=\tau G_{+}(t), \]
where
\[ G_{+}(t) = \frac{ \displaystyle \int_0^\infty \frac{\phi(x)\phi(t-x)}{\Phi(x)^2}\,dx }{ \displaystyle \int_0^\infty \frac{\phi(t-x)}{\Phi(x)}\,dx }. \]
Since \(y=c+\tau t\),
\[ \boxed{ \widehat\delta_{\rm CLWB}(y) =c+\tau\{t-G_{+}(t)\} }. \]
A.4 CLWB monotonicity, threshold-crossing boundary, and threshold limit
For fixed \(t\), normalize
\[ w_t(x) \propto \frac{\phi(t-x)}{\Phi(x)}, \qquad x\geq0, \]
to obtain a probability density. Because
\[ \phi(t-x)=\phi(x)\exp(tx-t^2/2), \]
increasing \(t\) exponentially tilts \(w_t\) toward larger \(x\). Meanwhile,
\[ g(x)=\frac{\phi(x)}{\Phi(x)}=\lambda(-x) \]
is strictly decreasing. Since \(G_{+}(t)=E_t\{g(X)\}\), differentiation under the integral gives
\[ G_{+}'(t) =\operatorname{Cov}_t\{g(X),X\}<0. \]
Thus \(G_{+}\) is strictly decreasing. The CLWB estimate is below \(c\) exactly when
\[ c+\tau\{t-G_{+}(t)\}<c \quad\Longleftrightarrow\quad t<G_{+}(t). \]
Here \(G_{+}(0)>0\), while \(G_{+}(t)\to0\) as \(t\to\infty\) because the tilted weight moves toward large \(x\), where \(g(x)\to0\). Because the left side of the boundary equation increases while \(G_{+}(t)\) decreases, there is a unique crossing. Numerical evaluation gives
\[ t_*=G_{+}(t_*), \qquad t_*=0.4081854. \]
It follows that
\[ \widehat\delta_{\rm CLWB}<c \quad\Longleftrightarrow\quad c<Y<c+0.4081854\,\tau. \]
Finally, numerical evaluation at the selection boundary gives
\[ G_{+}(0)=0.4621662. \]
As \(y\downarrow c\), \(t\downarrow0\), and continuity of \(G_{+}\) yields
\[ \widehat\delta_{\rm CLWB}(y) =c+\tau\{t-G_{+}(t)\} \longrightarrow c-0.4621662\,\tau. \]
Reference
Kirby S, Li J, Chuang-Stein C. Selection bias for treatments with positive Phase 2 results. Pharmaceutical Statistics. 2020;19:679-691. doi:10.1002/pst.2024.
Zhang H. Two-Sample MCLE Adjustment After Positive Phase 2 Selection. 2026. https://rpubs.com/zhangh12/TSMCLE.