in Oncology Phase I Dose Escalation Trials
September 30, 2026
The opinions expressed in this presentation and on the following slides are solely of the presenter, and not necessarily those of Novartis.
Initially, we have limited knowledge on toxicity of a new treatment.
I.e., we need to limit the risk to patients in trial by using small cohorts (1-6 patients) and an appropriate starting dose.
We then systematically increase the dosing
as quickly as possible, to reach a biologically active and (hopefully) efficacious dose,
as safely as possible, such that Dose-Limiting Toxicity (DLT) events are controlled
Goal: determine a safe dosing range for further optimization and development of the therapy (in an iterative manner, and ideally collect early efficacy data)
Safety model should limit risk of overdosing within trial and future patients at an acceptable DLT rate of below 33%
Y_i | \pi(d_i), n_i \sim \text{Binomial}(\pi(d_i), n_i) \text{logit}(\pi(d_i)) = \log(\alpha) + \beta \, \log(d_i / d_\text{ref})
p(\pi(d)|Y,n)
\pi(d) \in \begin{cases} [0, 0.16] & \text{under dose / low toxicity} \\ (0.16, 0.33] & \text{target dose / moderate toxicity} \\ (0.33, 1] & \text{\textcolor{red}{over dose / high toxicity}} \end{cases}
\text{EWOC criterion:} \; \; \text{Pr}{(\pi(d) \in \text{\textcolor{red}{over dose}})} < 1/4
d is limited if we are certain about overdosing or have large posterior uncertainty in p(\pi(d)|Y,n)
The starting dose and the planned escalation grid already encode a strong, asymmetric belief that implies a prior on the dose-DLT response curve:
The sequence of doses we want to study implies a prior!
In addition, we have some a priori beliefs that the prior should reflect, such as what reasonable behavior for the initial cohort(s) might be.
If the first cohort is of size 3, the 3+3 rules probably make some sense, i.e., at the starting dose:
2/3 and then 0/3 should allow escalating back to the starting dose.
Furthermore, for cohorts of size one, at the starting dose, we might want to consider that:
Repeatedly observing 0/3 DLTs should allow for escalating freely to the next (e.g., 2 \times) current dose level… we came up with a total of 18 data scenarios (talk to me in the break).
\begin{align*} Y_i | \pi(d_i), n_i &\sim \mathrm{Binomial}\left(\pi(d_i),n_i\right) \\ \mathrm{logit}(\pi(d)) &= \log(\alpha) + \beta \log(d_i/d_\text{ref}) \end{align*}
Prior for the intercept \log(\alpha) & slope \log(\beta):
(\log(\alpha), \log(\beta)) \sim \text{MultiNormal}(\mathbf{m}, \mathbf{S})
I.e., we need to choose the
The distribution of \alpha determines the distribution of the odds for a DLT at the reference dose d_\text{ref}:
\mathrm{logit}(\pi(d_\mathrm{ref})) = \log(\alpha) + \underbrace{\beta \log(d_\mathrm{ref}/d_\mathrm{ref})}_{=0}
Here, we chose a starting dose of d_\mathrm{start} = 1 and reference dose d_\text{ref}= 4, where
\log(\alpha) \sim \mathcal{N}(\mathrm{logit}(0.17), 1.7^2)
as shown by the violin plot.
If we vary the mean slope prior from 1/2 to 1 to 2, all curves meet at the reference dose (x axis) & intercept prior (y axis).
Extreme slopes such as 1/10 or 10 are not biologically plausible and should thus have low prior mass!
Rewriting the BLRM to \pi(d) illustrates why:
\pi(d) = \frac{\alpha (d/d_\text{ref})^\beta}{1 + \alpha(d/d_\text{ref})^\beta}
Flat curve limit
For \log(\beta) \rightarrow -\infty, \beta \rightarrow 0, the dose-response curve becomes flat at the intercept: \pi(d) \rightarrow \alpha/(1 + \alpha)
Step function limit
For \log(\beta) \rightarrow \infty, \beta \rightarrow \infty, the dose-response curve becomes a step function at the reference dose:
\pi(d < d_\text{ref}) \rightarrow 0, \pi(d > d_\text{ref}) \rightarrow 1
Rule of thumb
Biologically-feasible dose-response curve slopes should be between 1/4 and 4 \Rightarrow reasonable default prior
E.g., if we put 95% of the mass there with
\log(\beta) \sim \mathcal{N}(\log(1), (\underbrace{\log(4)/1.96}_{\approx 0.7})^2) \\
Then we get1:
95.1% reasonable slopes
(between 1/4 and 4)
2.5% flat (slope < 1/4)
2.5% step functions (slope > 4)
\log(\beta) \sim \mathcal{N}(\log(1), \textcolor{red}{3}^2)
34.7% reasonable slopes
(between 1/4 and 4)
31.4% flat (slope < 1/4)
33.9% step functions (slope > 4)
\log(\beta) \sim \mathcal{N}(\log(1), \textcolor{red}{3}^2)
30.6% reasonable slopes
(between 1/4 and 4)
26.2% flat (slope < 1/4)
43.2% step functions (slope > 4)
\begin{align*} d_\text{ref}&= 4\\ \log(\alpha) &\sim \mathcal{N}(\mathrm{logit}(0.17), 1.7^2) \\ \log(\beta) &\sim \mathcal{N}(\log(1), (\log(4)/1.96)^2) \\ \rho &= 0.6 \end{align*}
93.2% reasonable slopes
(between 1/4 and 4)
3.6% flat (slope < 1/4)
3.2% step functions (slope > 4)
{OncoBayes2}Setting up the prior for log-intercept, log-slope and correlation is simple using {OncoBayes2}:
\begin{align*} d_\text{ref}&= 4\\ \log(\alpha) &\sim \mathcal{N}(\mathrm{logit}(0.17), 1.7^2) \\ \log(\beta) &\sim \mathcal{N}(\log(1), (\log(4)/1.96)^2) \\ \rho &= 0.6 \end{align*}
with the starting dose d_\mathrm{start} = 1.
Note
If you have information that is incompatible with our choices after observing initial data, you need to change your prior to reflect that!
library("OncoBayes2"); library("tibble")
dose_info <- tibble(
group_id = "single_group", Dose = c(0.5, 1, 2, 4, 8, 12, 16)
)
drug_info <- tibble(
drug_name = "Dose", dose_ref = 4, dose_unit = "mg/day"
)
trial <- blrm_trial(data = NULL, dose_info, drug_info)
d <- trial |> summary("dimensionality")
# Prior on (log-intercept, log-slope)
m <- c(logit(0.17), log(1)) # Mean
s <- c(1.7, log(4)/1.96) # SDs
r <- 0.6 # Correlation
trial_fit <- trial |> update(
prior_EX_mu_comp = mixmvnorm(c(1, m, s, r), param = "msr"),
prior_EX_tau_comp = mixmvnorm(c(1, 0, 0, diag(c(1, 1)))),
prior_tau_dist = 0, # Tau fixed to 0
prior_EX_prob_comp =
matrix(1, nrow = d$num_groups, ncol = d$num_components)
)
plot_toxicity_curve(trial_fit)Idea: rather than hand-tuning \mathbf{m} and \mathbf{S}, state what you believe at the two ends of the planned grid and let the rest follow (Neuenschwander et al. 2008; Wakayama et al. 2025):
\begin{align*} \text{Pr}(\pi(d_1) > 0.3) &= 0.05 \\ \text{Pr}(\pi(d_J) \le 0.3) &= 0.21 \end{align*}
\Rightarrow This prior is design-informed: it is the dosing grid itself that supplies the two anchors.
For our grid, with d_\text{ref} at the starting dose1:
\begin{align*} \log(\alpha) &\sim \mathcal{N}(-2.14, 1.02^2) \\ \log(\beta) &\sim \mathcal{N}(-0.68, 0.27^2) \\ \rho &= 0.76 \end{align*}
99.4% reasonable slopes
(between 1/4 and 4)
0.6% flat (slope < 1/4)
0% step functions (slope > 4)
The Emax model with E_\text{max} = 1, E_0 = 0 makes the MTD \theta a model parameter — the dose with \pi(\theta) = \pi_t, here \pi_t = 0.33:
\pi(d) = \frac{d^{\,\beta}}{d^{\,\beta} + \theta^{\,\beta}\,(1/\pi_t - 1)}
Equivalently, on the logit scale, with a prior on the log-MTD and the log-slope:
\begin{align*} \text{logit}(\pi(d)) &= \text{logit}(\pi_t) + \beta\,(\log d - \log\theta) \\ (\log(\theta), \log(\beta)) &\sim \text{MultiNormal}(\mathbf{m}, \mathbf{S}) \end{align*}
No reference dose is needed — \theta takes its place.
Both parameterizations describe the same family of curves. Substituting
\log(\alpha) = \text{logit}(\pi_t) - \beta\,(\log\theta - \log d_\text{ref})
turns one into the other exactly.
But that map is non-linear in \beta, so
(\log(\theta), \log(\beta)) \sim \text{MultiNormal}\;\;\not\Rightarrow\;\; (\log(\alpha), \log(\beta)) \sim \text{MultiNormal}
\Rightarrow These are different priors!
Which one should we elicit?
The intercept asks “how toxic is d_\text{ref}?”. The MTD asks “which dose is the MTD?”
Implementation
{OncoBayes2} hard-codes the bivariate normal on (\log(\alpha), \log(\beta)), so this prior needs {brms}.
\begin{align*} \log(\theta) &\sim \mathcal{N}(2.58, 1.6^2) \\ \log(\beta) &\sim \mathcal{N}(-0.22, 0.45^2) \\ \rho &= 0.3 \end{align*}
i.e. a median MTD of 13.2 mg, with
99.4% reasonable slopes
(between 1/4 and 4)
0.5% flat (slope < 1/4)
0.1% step functions (slope > 4)
The data are compatible with the prior, so the curve distribution simply tightens — no mass is pushed into the implausible corners.
99.9% reasonable slopes
(between 1/4 and 4)
0.1% flat (slope < 1/4)
0% step functions (slope > 4)
Replicated full trial simulation based on hypothetical truths
Simulations assume that we aim for an MTD (safety \approx efficacy), and we then compare the priors.
Results:
P(MTD), %
|
E[N] | ||||
|---|---|---|---|---|---|
| under | target | over | stop | ||
| All toxic | 0 / 0 / 0 / 0 / 0 | 0 / 0 / 0 / 0 / 0 | 3 / 0 / 0 / 4 / 2 | 97 / 100 / 100 / 96 / 98 | 8 / 7 / 3 / 8 / 8 |
| Flat 50% | 0 / 0 / 0 / 0 / 0 | 0 / 0 / 0 / 0 / 0 | 18 / 10 / 5 / 17 / 16 | 82 / 90 / 95 / 83 / 84 | 13 / 10 / 6 / 12 / 13 |
| More toxic | 76 / 61 / 54 / 51 / 50 | 24 / 37 / 36 / 45 / 48 | 0 / 1 / 0 / 3 / 2 | 0 / 1 / 10 / 0 / 0 | 18 / 18 / 17 / 25 / 22 |
| Prior median | 46 / 37 / 34 / 18 / 18 | 47 / 60 / 59 / 58 / 63 | 7 / 3 / 1 / 24 / 19 | 0 / 0 / 6 / 0 / 0 | 19 / 17 / 15 / 26 / 23 |
| Not toxic | 100 / 100 / 98 / 100 / 100 | 0 / 0 / 0 / 0 / 0 | 0 / 0 / 0 / 0 / 0 | 0 / 0 / 2 / 0 / 0 | 21 / 14 / 13 / 20 / 20 |
| Not toxic, 1st pat. DLT | 100 / 100 / 0 / 100 / 100 | 0 / 0 / 0 / 0 / 0 | 0 / 0 / 0 / 0 / 0 | 0 / 0 / 100 / 0 / 0 | 32 / 31 / 1 / 34 / 33 |
| Zero tox | 100 / 100 / 100 / 100 / 100 | 0 / 0 / 0 / 0 / 0 | 0 / 0 / 0 / 0 / 0 | 0 / 0 / 0 / 0 / 0 | 15 / 12 / 12 / 13 / 14 |
| N(0, 0.7) / N(0, 3) / N(0, 3), ρ=0 / calibrated, d*=1 / model the MTD — bold = “best”? | |||||
\Rightarrow Operating characteristics tables do not tell the full story.
\Rightarrow Carefully consider them together with initial data scenarios!
Resources:
{OncoBayes2} on GitHub: opensource.nibr.com/OncoBayes2Contact: Lukas Widmer / lukas_andreas.widmer@novartis.com
But: safety model must still safely guide dose escalation!
Rule based
3+3 cohorts
2 cohorts with 1/3 \Rightarrow stop
Model assisted
Bayesian Optimal Interval Design
Model based
BLRM, TITE-CRM, CRM, …
Trial start
| Dose | num_patients | num_toxicities |
|---|---|---|
| 0.5 | 0 | 0 |
| 1.0 | 0 | 0 |
| 2.0 | 0 | 0 |
| 4.0 | 0 | 0 |
| 8.0 | 0 | 0 |
| 12.0 | 0 | 0 |
| 16.0 | 0 | 0 |
Cohort 1
| Dose | num_patients | num_toxicities |
|---|---|---|
| 0.5 | 0 | 0 |
| 1.0 | 3 | 0 |
| 2.0 | 0 | 0 |
| 4.0 | 0 | 0 |
| 8.0 | 0 | 0 |
| 12.0 | 0 | 0 |
| 16.0 | 0 | 0 |
… Cohort 4
| Dose | num_patients | num_toxicities |
|---|---|---|
| 0.5 | 0 | 0 |
| 1.0 | 3 | 0 |
| 2.0 | 3 | 0 |
| 4.0 | 3 | 0 |
| 8.0 | 3 | 0 |
| 12.0 | 0 | 0 |
| 16.0 | 0 | 0 |
plot_curve_dist <- function(newdata, model) {
data_unnested <- newdata |> mutate(risk = rvar(posterior_linpred(
model$blrmfit, newdata=newdata, allow_new_levels=FALSE, transform = T, permuted = FALSE))
) |> unnest_rvars() |> select(-group_id, -.chain, -.iteration)
mu_log_beta <- as_draws_df(model$blrmfit) |> as_tibble() |> select(contains("log_slope") & contains("mu_log_beta"))
colnames(mu_log_beta) <- "log_beta"
mu_log_beta <- mu_log_beta |> mutate(beta = exp(log_beta), n = 1:n())
data_unnested <- data_unnested |> inner_join(mu_log_beta, join_by(.draw == n)) |>
mutate(curve_type = case_when(
beta < 1/4 ~ "Flat", beta > 4 ~ "Step function", .default = "Reasonable"
) |> factor(levels = c("Step function", "Flat", "Reasonable")))
data_unnested |> filter(.draw <= 1000) |> mutate("Curve Type" = curve_type) |>
ggplot(aes(x = Dose, y = risk, group = .draw, color=`Curve Type`)) +
geom_line( alpha=0.1, size = 0.8) +
scale_x_continuous(breaks = summary(model, "dose_info")$Dose) +
scale_y_continuous(breaks = c(seq(0, 1, by = 0.1), 0.16, 0.33), labels = scales::percent, limits = c(0, 1)) +
scale_colour_manual(values = c(
"Step function" = "red", "Flat" = "orange", "Reasonable" = "darkgreen"
), drop = FALSE) +
hline_at(c(0.16, 0.33), linetype = I(2)) +
xlab("Dose") +
ylab("P(DLT)") +
guides(color = guide_legend(override.aes = list(alpha = 1))) + theme(legend.position="none")
}
tibble(
summary(demo_blrm_trial, "dose_info")[1, ] |> select(group_id, stratum_id),
Dose = seq(from = 0, to = max(dose_info$Dose), length.out = 105)
) |> plot_curve_dist(trial_fit)q_{75\%}(\pi(d_i)|Y,n) < 1/3
OncoBayes2 using routines from (Vehtari et al. 2021) and automatically flagged if of concernEvery prior on the dose-DLT curve implies a prior on the MTD — the two just make different statements about it:
Rule of thumb
Plot the implied MTD prior against the planned dose grid. If most of its mass sits outside the grid, the grid and the prior disagree.
Rule of thumb
Biologically-feasible dose-response curve slopes should be between 1/4 and 4 \Rightarrow reasonable default prior
The calibrated prior places its slope mass well inside that range,
\log(\beta) \sim \mathcal{N}(-0.68, 0.27^2)
i.e., 95% of the slopes lie in [0.30, 0.87].
Then we get1:
99.6% reasonable slopes
(between 1/4 and 4)
0.4% flat (slope < 1/4)
0% step functions (slope > 4)
\log(\beta) \sim \mathcal{N}(-0.68, \textcolor{red}{3}^2)
33.8% reasonable slopes
(between 1/4 and 4)
40.9% flat (slope < 1/4)
25.3% step functions (slope > 4)
\log(\beta) \sim \mathcal{N}(-0.68, \textcolor{red}{3}^2)
31.9% reasonable slopes
(between 1/4 and 4)
35.6% flat (slope < 1/4)
32.5% step functions (slope > 4)
\log(\beta) \sim \mathcal{N}(-0.68, \textcolor{red}{3}^2)
0% reasonable slopes
(between 1/4 and 4)
0% flat (slope < 1/4)
100% step functions (slope > 4)
\begin{align*} d_\text{ref}&= 4\\ \log(\alpha) &\sim \mathcal{N}(-2.14, 1.02^2) \\ \log(\beta) &\sim \mathcal{N}(-0.68, 0.27^2) \\ \rho &= 0.76 \end{align*}
99.4% reasonable slopes
(between 1/4 and 4)
0.6% flat (slope < 1/4)
0% step functions (slope > 4)
\begin{align*} d_\text{ref}&= 4\\ \log(\alpha) &\sim \mathcal{N}(-2.14, 1.02^2) \\ \log(\beta) &\sim \mathcal{N}(-0.68, 0.27^2) \\ \rho &= 0.76 \end{align*}
31.8% reasonable slopes
(between 1/4 and 4)
68.2% flat (slope < 1/4)
0% step functions (slope > 4)
\begin{align*} d_\text{ref}&= 4\\ \log(\alpha) &\sim \mathcal{N}(-2.14, 1.02^2) \\ \log(\beta) &\sim \mathcal{N}(-0.68, 0.27^2) \\ \rho &= 0.76 \end{align*}
Every dose now looks toxic, so the posterior is pushed up as far as the prior allows: the median runs from 58% at 0.5 to 98% at 16.
100% reasonable slopes
(between 1/4 and 4)
0% flat (slope < 1/4)
0% step functions (slope > 4)
{brms}\begin{align*} \text{logit}(\pi(d)) &= \text{logit}(0.33) \\ &\quad + \beta\,(\log d - \log \theta) \\[4pt] \log(\theta) &\sim \mathcal{N}(2.58, 1.6^2) \\ \log(\beta) &\sim \mathcal{N}(-0.22, 0.45^2) \\ \rho &= 0.3 \end{align*}
Note
The intercept is fixed at \text{logit}(0.33): it is the target toxicity that defines the MTD, not a free parameter.
library("brms")
model <- bf(
num_toxicities | trials(num_patients) ~
inter + exp(logb) * (log(Dose) - logmtd),
nl = TRUE, family = binomial()
) + lf(logmtd ~ 1) + lf(logb ~ 1) + lf(inter ~ 1)
# bivariate normal on (log-MTD, log-slope)
model_prior <- prior(
"target += multi_normal_cholesky_lpdf(
[b_logmtd[1], b_logb[1]]'| M, C)",
class = "b", check = FALSE
) + prior(constant(logit(0.33)), nlpar = "inter")
M <- c(2.58, -0.22) # Mean
S <- c(1.6, 0.45) # SDs
rho <- 0.3 # Correlation
stanvars <- stanvar(rho) + stanvar(M) + stanvar(S) +
stanvar(block = "tdata", scode = "matrix[2,2] C =
[ [S[1], 0],
[rho * S[2], sqrt(1.0 - rho*rho) * S[2]] ];")
fit <- brm(model, data, prior = model_prior,
stanvars = stanvars)\begin{align*} \log(\theta) &\sim \mathcal{N}(2.58, 1.6^2) \\ \log(\beta) &\sim \mathcal{N}(-0.22, 0.45^2) \\ \rho &= 0.3 \end{align*}
The slope prior is narrow, so even overwhelming amounts of safe data leave the shape of the curve alone:
99.6% reasonable slopes
(between 1/4 and 4)
0% flat (slope < 1/4)
0.4% step functions (slope > 4)
\begin{align*} \log(\theta) &\sim \mathcal{N}(2.58, 1.6^2) \\ \log(\beta) &\sim \mathcal{N}(-0.22, 0.45^2) \\ \rho &= 0.3 \end{align*}
Toxicity above the target at every dose drags the MTD from a prior median of 13.2 down to 0.08 — 100% of the posterior puts it below the lowest planned dose of 0.5.
100% reasonable slopes
(between 1/4 and 4)
0% flat (slope < 1/4)
0% step functions (slope > 4)