Clinical Trials

A clinical trial is a prospective study that is used to assess the safety or efficacy of an intervention, such as a pharmaceutical drug. At Precision Analytics, we frequently assist our clients with the many statistical components of clinical trial design and analysis, from Phase I trials to post-marketing studies.

Our client, a hospital-affiliated research institute, was preparing to conduct a cluster randomized trial of a rare neurological disease in order to examine the efficacy of a drug that had already received regulatory approval for another indication. The goal of the trial was to determine whether the institute should encourage a label expansion to a rare disease. One of the challenges of working with rare diseases is that recruitment is challenging. Our client was concerned whether the small number of potentially recruitable patients would result in a sufficiently powered study.

Conventional statistical approaches base sample size and power calculations on hypothesis testing and thresholds of type I and type II error. For example, the null hypothesis of no treatment effect is tested against the alternative hypothesis that there is a positive effect of treatment for one or more measured endpoints. The goal of a traditional sample size calculation is to determine the number of patients required to maximize the chance of making a “correct” conclusion: given an expected clinically meaningful effect size, many researchers and regulators desire at least 80% power and a significance level of 5%. The challenge in clinical trials of rare diseases is that they are often underpowered, leading to a null result that is of very limited utility for clinical decision-making.

Our client wanted to assess the feasibility of the proposed clinical trial, given that recruitment was extremely limited due to the rare nature of the disease. Published observational studies from off-label use of the drug were available for sample size and power calculations, but there was large variation in the estimated effectiveness of the drug for this disease.

The question we sought to answer was, given a fixed sample size for this rare disease, what was the probability that the study would be sufficiently powered?

We proposed exploring the trial design from a Bayesian perspective. Bayesian statistical approaches have been gaining popularity over the last two decades, and rare diseases are a particularly appropriate application. It allows for “prior beliefs”, in the form of external data or expert clinical opinion, about the intervention to be integrated with the clinical trial data to inform final decisions about whether a trial should be undertaken or continued. Bayesian approaches can potentially increase statistical power , either making large trials more efficient or making smaller studies less likely to be underpowered.

We used Bayesian models to combine the available clinical data and simulate the different scenarios that could occur given different effect sizes, and propagate the uncertainty in each estimate through to the final conclusions. This allowed us to provide a fuller picture, and estimate the probability of the researchers ending up with an underpowered study. We also proposed an adaptive strategy , where interim analyses would be performed to update these probabilities as more data became available. This would allow the researchers to end the study early if the probability of finding a meaningful result was very low.

We ran these models using Markov chain Monte Carlo methods, implemented in Stan using our favourite open source programming language, R.

Using modern methods allowed us to extract maximum information from the available data, and be more explicit about the uncertainty in the effect estimates from prior research. Rather than providing a formulaic sample size or power estimate, we were able to provide a richer understanding of the uncertainty and likely scenarios. We were also able to update these results once data from the clinical trial began. This work is ongoing as the trial continues. Ultimately, we reduced the probability of the research institute conducting an underpowered study.

Small or fixed sample sizes for clinical trials are not ideal but represent the norm, and there are important ethical considerations in the design and analysis of clinical trials for rare diseases. For debilitating neurological diseases like the one in question, for which few treatment options exist, there is considerable value in providing clinically useful information with the number of patients available, even with substantial uncertainty.

We have undertaken other innovative approaches to clinical trial design and analysis, in order to help our clients address challenges in their disease domains. For example:

• Use of external control arms for phase II clinical trials, particularly in oncology
• Use of novel methods for analyzing single arm trials
• Adaptive designs, such as use of stopping rules

Clinical trial design and analysis strategies are changing rapidly, as regulatory agencies move towards greater acceptance of real world data and innovative methods, including Bayesian approaches.

At Precision Analytics, we are very excited to be part of the modern analytical era in clinical trials; please get in touch if you would like to discuss further.