Understanding FDA Recommendations for Covariate Adjustment in Randomized Clinical Trials

In May 2023, the US Food and Drug Administration (FDA) issued final guidance on adjusting for covariates in randomized clinical trials (RCTs), titled "Adjusting for Covariates in Randomized Clinical Trials for Drugs and Biological Products". This guidance provides both general considerations and specific recommendations for covariate adjustment in linear and nonlinear models. It is a valuable resource for industry statisticians, offering insights into the various statistical issues related to covariate adjustment in RCTs. In this post, I will share my comments on each item (bold text) from the "General Considerations" section of the guidance document. If possible, I'll address some of these issues in more detail in future posts.

• An unadjusted analysis is acceptable for the primary analysis of an efficacy endpoint.

  
  

  Comments: An unadjusted analysis is valid in RCTs, though it's often less efficient than an adjusted analysis. It’s like choosing to buy your lunch without a company subsidy - it’s up to you whether you want to take advantage of available resources.

  
  

• Sponsors can adjust for baseline covariates in the analyses of efficacy endpoints in randomized clinical trials. Doing so will generally reduce the variability of estimation of treatment effects and thus lead to narrower confidence intervals and more powerful hypothesis testing.

  
  

  Comments: An adjusted analysis is also acceptable. The purpose of covariate adjustment in RCTs is to increase the efficiency of statistical analyses - covariate adjustment is simply a tool to achieve this goal, not an objective in itself. Terms like "correct or adjust for covariate imbalance" or "correct bias" are misleading in the context of RCTs, as an unadjusted analysis is also valid. In other words, even if baseline covariates are perfectly balanced and an unadjusted analysis yields unbiased results, an adjusted analysis would still be preferable for its potential efficiency gains.

  
  

• Sponsors should prospectively specify the detailed procedures for executing covariate-adjusted analysis before any unblinding of comparative data. FDA review will emphasize the prespecified primary analysis rather than post-hoc analyses using different models or covariates.

  
  

  Comments: All key analyses must be prespecified to avoid ambiguity - this has nothing to do with the validity of covariate adjustment.

• Covariate adjustment leads to efficiency gains when the covariates are prognostic for the outcome of interest in the trial. Therefore, FDA recommends that sponsors adjust for covariates that are anticipated to be most strongly associated with the outcome of interest. In some circumstances these covariates may be known from the scientific literature. In other cases, it may be useful to use previous studies (e.g., a Phase 2 trial) to select prognostic covariates or form prognostic indices.

  
  

  Comments: Only choose prognostic covariates (those strongly correlated with the response variable) for adjustment. After all, you wouldn't want to include something like patients' shoe size or political party affiliation as covariates.

  
  

• Covariate adjustment can still be performed with covariates that are not prognostic, but there may not be any gain in precision (or may be a loss in precision) compared with an unadjusted analysis.

  
  

  Comments: Your analysis would still be valid even if you included patients' shoe size or political party affiliation as covariates (seriously). The concern is purely about efficiency - including such covariates is more likely to result in a loss rather than a gain in efficiency due to the reduced residual degrees of freedom.

  
  

• Covariate adjustment is acceptable even if baseline covariates are strongly associated with each other (e.g., body weight and body mass index). However, adjusting for less correlated baseline covariates generally provides greater efficiency gains.

  
  

  Comments: If an unadjusted analysis is valid, then statistical inferences about the treatment effect remain valid in an adjusted analysis with correlated baseline covariates. This is because the issue of collinearity only applies to the correlated covariates themselves. However, the efficiency gain from including strongly correlated covariates is expected to be limited.

  
  

• Randomization is often stratified by baseline covariates. A covariate adjustment model should generally include strata variables and can also include covariates not used for stratifying randomization. In some cases, incorrect stratification may occur and result in actual and as-randomized baseline strata variables. A covariate adjustment model can use either strata variable definition as long as this is prespecified.

  
  

  Comments: Study design and analysis should be consistent: stratification variables used for randomization should generally (but not always) be accounted for in the analysis. However, covariate adjustment is not necessarily limited to stratification variables only. Mis-stratification can be addressed using either the actual strata or the as-randomized strata in the analysis, provided this approach is specified in advance.

  
  

• Sponsors can conduct randomization/permutation tests with covariate adjustment (Rosenbaum 2002).

  
  

  Comments: It appears that the FDA is open to using permutation-based tests with covariate adjustment, which may be applicable if there is significant uncertainty about the distribution of the response variable. One advantage of permutation tests is their ability to avoid strong distributional assumptions. For example, permutation tests may be used for linear models without assuming a normal distribution for the error term.

  
  

• In a trial that uses covariate adjustment, the sample size and power calculations can be based on adjusted or unadjusted methods. The latter will often lead to a more conservative sample size.

  
  

  Comments: Sample size and power calculations can be based on either adjusted or unadjusted analyses if an adjusted analysis is planned. The key point is that the study must be adequately powered; therefore, using a conservative approach for sample size and power calculations is not considered a regulatory issue.

  
  

• Clinical trials often record a baseline measurement of a defined characteristic and record a later measurement of the characteristic to be used as an outcome. Adjusting for the baseline value rather than (or in addition to) defining the primary endpoint as a change from baseline is generally acceptable. Sponsors proposing to define the outcome as a percentage change rather than an absolute change from baseline should discuss the outcome definition and use of covariate adjustment with the relevant review division. Sponsors proposing to perform noninferiority testing on ratios of treatment group means rather than differences of treatment group means should also discuss change from baseline outcome definitions and use of covariate adjustment with the relevant review division.

  
  

  Comments: Analysis of a raw outcome or change from baseline can be performed with the baseline outcome value as a covariate. If the primary endpoint is defined as a percent change, the sponsor should consult with the agency regarding the endpoint definition and the use of covariate adjustment. It's likely that the FDA has concern about the distribution of such an endpoint (percent change), which could be highly skewed or complicated. Noninferiority (NI) testing is often conducted using the confidence interval approach, but deriving confidence intervals for ratios of means can be challenging and may require nontrivial approximations. For NI testing, it seems that the FDA may explore the option of using a change from baseline endpoint with covariate adjustment to assess treatment group mean differences.

  
  

• Sponsors should discuss proposals for complex covariate-adaptive randomization, data-adaptive covariate selection, or use of covariate adjustment in an adaptive design with the relevant review division.

  
  

  Comments: Type I error control in adaptive designs with covariate adjustment will likely face increased scrutiny from the FDA due to the complexity of the issue. It is advisable to seek input from the agency prospectively.

  
  

• The statistical properties of covariate adjustment are best understood when the number of covariates adjusted for in the study is small relative to the sample size (Tsiatis et al. 2008). Therefore, sponsors should discuss their proposal with the relevant review division if the number of covariates is large relative to the sample size or if proposing to adjust for a covariate with many levels (e.g., study site in a trial with many sites).

  
  

  Comments: Although large trials can generally accommodate more covariates than smaller ones, justification for including a large number of covariates should be provided, especially in small trials. When a large number of covariates are included, it is highly likely that: (1) some may be uncorrelated or only weakly correlated with the response variable; and/or (2) some may be correlated with each other. In either case, the efficiency gain from these additional covariates, if any, is expected to be limited. Moreover, strongly correlated covariates may lead to misleading inferences about the covariates themselves.