adaptr: an R package for simulating and comparing adaptive clinical trials

The adaptr R package facilitates simulation and comparison of adaptive clinical trial designs using Bayesian statistical methods. The package supports a flexible number of arms, use of a common control arm, pre-specified and user-defined outcome-and posterior probability distribution-generating functions, fixed-and response-adaptive randomisation (RAR), various adaptation rules for arm dropping and stopping, calculation of trial design performance metrics, and visualisation of results. Simulations are relatively fast, can run in parallel, and can be reloaded from previous sessions.


General
Allows for two or more intervention arms and comparison of all arms or pairwise comparisons against a common control arms.Simulations use Bayesian statistics.

Outcomes and posteriors
Supports multiple outcome types specified by user-defined functions, with built-in convenience functions for easy use with binary, binomially distributed and continuous, normally distributed outcomes.
For more flexibility, user-defined functions for generating outcomes and sampling from posterior distributions are supported; these may use more complex functions provided by other packages (e.g., Markov chain Monte Carlo or variational inference).

Random allocation
Supports fixed and response-adaptive randomisation, and combinations thereof, including special handling of control group allocation in trials with a common control (e.g., matching highest non-control group allocation ratio and square-root-ratio-based allocation (Park et al., 2020;Viele, Broglio, et al., 2020)).Supports minimum/maximum allocation limits and softening non-fixed allocation probabilities by raising them to a power (Ryan et al., 2020) (which may vary throughout the trial) and normalising probabilities to sum to 100%.

Adaptation rules
Supports probabilistic trial-stopping/arm-dropping rules for superiority, inferiority, and practical equivalence between all arms in trial designs without a common control.Supports probabilistic trial-stopping/arm-dropping rules for superiority, inferiority, practical equivalence, and futility compared to a common control in such designs.
In trial designs with a common control, a superior arm becomes the new control.In these cases equivalence/futility testing may be continued (using the new control) or disabled.Performance metrics Supports the calculation of various trial design performance metrics (Viele, Broglio, et al., 2020;Viele, Saville, et al., 2020) according to different arm selection strategies in trials not ending with superiority.

Visualisation
Offers graphical summaries of the overall or arm-specific statuses over time across multiple simulations (such as still recruting or trial stopped/arm dropped for various reasons), and relevant arm-specific metrics (e.g., allocation probabilities) over time across single/multiple trial simulations.

Technical
Supports parallel execution on multiple cores using R's built-in parallel package.Allows reproducible results with seeding.Supports saving/loading large trial objects and growing previous simulation objects.
Uses base R with no required external dependencies; optional plotting functionality does, however, require the ggplot2-package (Wickham et al., 2019).
Adaptive analyses are conducted at specified recruitment numbers, after random allocation of simulated patients to currently active arms.Both fixed and response-adaptive randomisation (RAR) as well as combinations (including several restrictions of RAR) are supported (Pallmann et al., 2018;Park et al., 2020;Ryan et al., 2020;The Adaptive Platform Trials Coalition, 2019).
Outcomes are then generated, followed by drawing samples from the posterior distributions; the packages comes with built-in functions for generating outcomes and posteriors using fast models with conjugate, flat priors (Ryan et al., 2019), but more advanced estimation are allowed using user-defined functions.
Posterior draws are used to calculate probabilities for (i) enforcing adaptive trial-stopping/arm-dropping rules with respect to superiority, inferiority, equivalence and/or futility, (ii) assessing the trial's final status (superiority, inferiority, equivalence, futility, or stopped at a maximum pre-specified sample size), and (iii) adjusting allocation probabilities if RAR is used.Single or multiple trial simulations can be run, saved and grown, and summarised using multiple relevant performance metrics (Viele, Broglio, et al., 2020;Viele, Saville, et al., 2020).Performance metrics can be calculated under different assumptions about the preferable intervention in clinical practice should the trials not end with superiority.

Functions and workflow
An overview of the principal functions in adaptr is given in Table 2:

Function name Action setup_trial, setup_trial_binom, setup_trial_norm
Defines the adaptive trial specification including arms, true outcome rates, stopping rules, outcome/posterior generating functions, and summary settings.Full flexibility is provided by setup_trial; the convenience functions setup_trial_binom and setup_trial_norm provide sensible defaults for binary, binomially distributed and continuous, normally distributed outcomes, respectively.
run_trial Runs a single simulation using a trial specification defined with

setup_trial. run_trials
Runs multiple trial simulations using a trial specification defined with setup_trial.Can save simulation objects to external files and load/grow previous simulations.

extract_results
Extracts key results and (partial) performance metrics from multiple simulations conducted by run_trials according to a specified arm selection strategy for simulations not ending in superiority.Returns a tidy data.framewith one simulation per row (Wickham et al., 2019).summary Summarises, in a human-friendly way, key results and calculates overall performance metrics for multiple simulations according to a specified arm selection strategy for simulations not ending in superiority.plot_status Plots the overall statuses or arm-specific statuses for multiple simulations over the course of the simulated trials.plot_history Plots the history of relevant metrics in each arm for one or multiple simulations over the course of the simulated trial(s).
print Print methods for printing the outputs of most of the included functions in a human-friendly way.

Numerical results
We summarise the results, assuming the control arm would be used in clinical practice in case of inconclusive trials (unless dropped early): Alternatively, we can get tidy results (Wickham et al., 2019) for each simulation using the same selection strategy as above:

Visual summaries
We can plot the overall trial statuses at each adaptive analysis using: plot_status(sims) Arm-specific statuses and additional metrics over time may be plotted as well.

Discussion
We have developed a flexible, extensible and comprehensive R package that allows relatively easy and efficient simulation of adaptive clinical trials using various features with few, welldocumented user-facing functions.
The package facilitates planning of adaptive clinical trials and comparison of trial designs using combinations of the most important and common adaptive features.In addition to the efficient convenience functions provided for common outcome types, more complex models are supported but may increase simulation time substantially and require (highly) parallel execution on multiple cores locally and/or remotely using cloud computing.
Compared to other open-source solutions (Meyer et al., 2021), adaptr comprises a completely functional and well-documented R package with support for multiple complex randomisation strategies, different stopping rules with detailed control, full support for user-defined outcome-/posterior-generating functions, parallel computation, easy calculation of various trial design performance metrics according to different arm selection strategies for trials not ending in superiority, and visualisation of relevant metrics across single or multiple simulations.While relatively feature-rich, the package currently has some limitations.These primarily include the lack of a 'flooring' option (stopping allocation to arms when their probabilities of being the best fall below a certain threshold, possibly with subsequent resumption of allocation (Viele, Broglio, et al., 2020)), inability to add arms during the conduct (as done in adaptive platform trials (The Adaptive Platform Trials Coalition, 2019); this is complex and not supported in most other software (Meyer et al., 2021)) and the lack of separate stopping rules/allocation ratios in subgroups or enrichment (which is similarly complex (The Adaptive Platform Trials Coalition, 2019)).Finally, the package uses Bayesian statistical methods as these are well-suited for adaptive trials and easily extended; frequentist statistical methods are not supported.
In conclusion, the adaptr R package provides a feature-rich and extensible open-source, scripting-based solution for planning, simulating and comparing adaptive clinical trials.

Table 1
gives an overview of the functionality of adaptr:

Table 1 :
Overview of functionality in the adaptr package.

Table 2 :
Overview of user-facing functions in adaptr.
so is running multiple trials, for example 1000.If cores > 1, simulations are runs in parallel on the number of cores specified.Setting sparse = FALSE keeps all intermediate simulation results; this is required for plot_history (see below) but makes the object a lot larger (in this case, by a factor 15). -