Understanding ICH Guidelines: Be Pragmatic, Not Dogmatic

This post is motivated by a disagreement I recently had with a colleague regarding a statistical issue related to the ICH (International Council for Harmonization) E9 guideline (Statistical Principles for Clinical Trials, 1998). This issue was raised by a statistical reviewer from the European Medicines Agency (EMA) during the review of a phase 3 trial protocol. The trial was designed with a planned interim analysis of a binary surrogate endpoint and a final analysis of a clinical outcome endpoint, utilizing a stratified randomization schedule. The stratification variables, as outlined in the trail protocol, are considered prognostic for the clinical outcome endpoint and thus were accounted for only in the final analysis. However, the EMA reviewer appeared to overlook this detail and commented that adjustment for these variables is also required for the interim analysis.

My colleague insisted that we should follow the reviewer's recommendation, claiming it aligns with the ICH E9 guideline. I argued that the stratification variables are not prognostic for the surrogate endpoint (at least, there is no evidence to suggest that they are) so adjustment for these variables in the interim analysis is not justified. This disagreement highlights a crucial issue in understanding and implementing ICH guidelines: should we always adhere strictly to the guidelines, or should we adopt a more pragmatic approach and follow them only when applicable?

The answer to this question is best illustrated in a general statement included in the FDA (Food and Drug Administration) version of the guidelines: "This guidance represents the current thinking of the Food and Drug Administration (FDA or Agency) on this topic. It does not establish any rights for any person and is not binding on FDA or the public. You can use an alternative approach if it satisfies the requirements of the applicable statutes and regulations." This statement implies that the guidance reflects the FDA's current views and recommendations on a specific topic (i.e., statistical principles for clinical trials). It’s also important to note that the guidance:

1. Doesn't Create Legal Rights: It does not provide legal entitlements or protections for anyone, including the public.

2. Is Not Mandatory: The FDA is not legally bound by the guidance, nor does it obligate others to follow it.

3. Allows Flexibility: If an alternative approach meets the relevant laws and regulations, it can be used instead of following the guidance.

In summary, ICH guidelines serve more as informative resources rather than strict rules.

The ICH E9 guidance provides the following specific recommendations regarding the adjustment of the randomization stratification variables in the analysis: "If one or more factors are used to stratify the design, it is appropriate to account for those factors in the analysis. When the potential value of an adjustment is in doubt, it is often advisable to nominate the unadjusted analysis as the one for primary attention, the adjusted analysis being supportive." This statement clearly implies that statisticians have the flexibility to determine whether an adjusted analysis is appropriate, which aligns with the general statement in the FDA guidance document.