Adaptive Study Design: How Many Subjects Do I Need For My Clinical Trial?

By Michelle Michela, Boston Biomedical Associates

In keeping with FDA’s commitment to and alignment with the CDRH mission and vision statement of ensuring timely access to safe, effective, and high-quality medical devices, FDA has released an important guidance document on adaptive trial design in medical device trials.

How Many Subjects Do I Need For My Clinical Trial?
This age old question is brought to the desk of biostatisticians everywhere everyday by clinicians and businessmen trying to advance science or get a product to market. “Well it depends…..”  And doesn’t that answer make you crazy?

Of course, the required number of subjects does depend on many factors. We all understand that inherently, yet how do we pin down all the variables, and feel confident enough in our assumptions to ensure our trial is efficiently and appropriately sized? In many cases, your best guess is really not good enough, and you end up oversizing your trial to be conservative, costing your company and investors more than required. Or worse yet, under sizing your trial because funding is scarce, and just missing your primary endpoint because you are underpowered (otherwise known in the FDA guidance as “Anticipated Regret”¹). Neither of these scenarios are tenable. A more successful approach is to consider an adaptive trial design. Adaptive trial designs are gaining attention and becoming more utilized in the medical device arena in particular. FDA’s release of the draft guidance on “Adaptive Designs for Medical Device Clinical Studies” gives us guidance on when and how to incorporate an adaptive design into your clinical trial for medical device regulatory approval.

Adaptive approaches mitigate the sample size dilemma by allowing a range of possible sample sizes for your trial with interim analyses that guide the ultimate trial size. Approaches include Bayesian or Frequentist Sample Size Adaptation, or a Group Sequential Design (with our without Sample Size Reassessment).

Adaptive trial designs can be used for many other adaptations as well. In fact, you can adapt any of the following with an adaptive design:

• Dropping a Treatment Arm
• Changing the Randomization Ratio
• Changing the Hypothesis (Claim)
• Adaptive Enrichment (modify the inclusion/exclusion criteria)
• Planning to adapt based on the Total information (using variance on the endpoint)
• Adaptation of the Device or Endpoint
• Seamless Studies (feasibility transitions to a pivotal)
• Covariate adaptive randomization (using accumulating baseline data in an attempt to provide better balance between the two groups)

Adaptive designs may not always be the right choice. For example, studies with short enrollment timelines or endpoints that occur early in the study may not lend themselves to an adaptive design approach easily. The timing of the interim analysis and the time required to execute and review are critical factors to consider when planning implementation of an adaptive study design. Studies with shorter endpoints but longer recruitment times lend themselves better to adaptation. Studies in which the time to the primary endpoint evaluation is long but the accrual is even longer may also benefit from an adaptive design.

Of course, as with any approach there are Pros and Cons, and Watch-outs! A brief summary of these are provided below.

Advantages Of An Adaptive Design
Adaptive designs help support timelier device development decision-making, more efficient investment in resources in a clinical study, optimize the treatment of subjects enrolled in the study, and safeguard their welfare from ineffective or unsafe treatments. Specifically, adaptive designs can:

• Be more efficient, saving time, money, and resources
• Improve the chance of trial success
• Yield an improved understanding of the effect of the investigational treatment and a better understanding of benefit and risk
• Facilitate transition from premarket to post-market follow-up (FDA is willing to defer some data collection to a post market setting as a way to develop additional information regarding benefits or risks for certain devices)
• Allow planned modifications at no cost in either sample size increase or false positive error inflation provided there is a strong blind to outcomes by treatment groups
• Enhance patient protection by increasing the probability that a patient is allocated to the treatment most likely to result in a better outcome for that patient
• Identify patients more likely to have a favorable benefit-risk profile from the use of a device
• Allow modification of the patient population during the study, converting what would otherwise be a failed study to one with a more targeted indication for which there are data to support both safety and effectiveness
• Improve decision-making at milestones during product development
• Increase the chance of a successful study with the potential to improve time-to-market
• In some cases, an adaptive design can obviate the need for a feasibility study (or a second feasibility study)

Disadvantages Of An Adaptive Design
Adaptive designs do require some additional attention and careful logistical management. Most importantly, adaptive designs:

• Require more effort at the design stage
• Can be difficult to plan, requiring support of statisticians and others experienced in adaptive trial design
• Can add cost, at start-up as well as for planned interim analyses
• Can be logistically difficult to carry out, to ensure data at interim analyses remains blinded to those who should not have knowledge of interim treatment effects, and to ensure data at interim analysis points is available and accurate
• Can introduce bias if not done correctly, making it difficult to characterize the true effect of the investigational device
• Can confound the interpretation of the study results if a change to the study due to an adaptation leads to results before the adaptation that are not sufficiently similar to those after the adaptation
• In some cases, when studies enroll subjects rapidly, there may not be time to make changes to the study design. In such cases sponsors may consider slowing down enrollment to allow time to learn from the accumulating data and make preplanned adaptations.

Watch Outs And Challenges!

• Sponsors should provide to FDA sufficient evidence of a “firewall” and documented policies and information in advance that will assure personnel are appropriately blinded/masked during the conduct of the adaptive study.
• Take care to ensure you minimize operational bias! (Decision-makers, investigators, subjects or third party evaluators may behave differently if they have any knowledge of outcome, including the size of the sample size increase). Safeguards must be in place to ensure that those with legitimate access to unblinded data do not share information about these data with others.

Note: Operational bias cannot be overcome with statistical adjustments!

• Ensure careful attention to challenges in data analysis of adaptive design trials, including
• Bias Control in the Estimates (upwardly biased point estimate)
• Homogeneity of Results after a Modification (before and after modification) – may be an indicator of study operational bias
• Ensure careful Selection and Management of Data Monitoring Committees (ensure members are well selected, there is a firewall for managing data and interactions, investigators and subjects are shielded from knowledge of adaptive changes, there are SOPs relating to implementation of the adaptive design protocol).

Note: Once the DMC has access to unblinded outcomes, recommendations to change the design or study type can imperil the scientific integrity of the study!

• Clearly describe the adaptive nature to IRBs
• Don’t announce to sites when you are conducting interim analyses
• Have a pre-determined monitoring plan in place to ensure adequate monitoring if the pre-planned changes do occur. Conduct pre-planned site visits to ensure validity and accuracy of interim data, and to verify adequate documentation and execution of blinding procedures in order to ensure blinding was appropriately maintained.
• Address Logistical Challenges (Ensure efficient and reliable data management, device availability, SOPs to ensure outcome results remain blinded)

Even unplanned midstream changes to your trial may be possible, BUT the FDA will expect sponsors to be able to both justify the scientific rationale why such an approach is appropriate and preferable, and demonstrate that they have not had access to any unblinded data (either by coded treatment groups or completely unblinded) and that the data has been scrupulously safeguarded.

Reference:

1. Anticipate particular study outcomes that could lead to failure so as to ask what one might be regretted in the planning.  For example, if a study just barely missed its objective but still had a clinically important effect and in retrospect would have likely succeeded if the sample size had been 15% larger, that might suggest that one should have planned for an adaptive sample size design in which the sample size could be reassessed partway through the study.