The Frequentist Confidence Interval (CI)

I recently stumbled upon an interesting LinkedIn post by a biostatistics professor. He stated that "Bayesians should take the term 'confidence interval' for credible interval while frequentist confidence interval should be called 'frequency interval'.". While this "suggestion" is unlikely to be serious, the message behind it probably resonates with many statisticians. Simply put, the phrase "confidence interval" in its literal sense has a more Bayesian flavor than a frequentist one.

Let me use a simple example to show how a 95% CI is defined in the frequentist context. Consider a random sample Ⅹ₁, Ⅹ₂, ..., Ⅹₙ from the normal distribution 𝑁(𝜇, 𝜎²), where 𝜇 is the fixed but unknown population mean, and 𝜎² the known variance. Let X̅ be the sample mean, then we have the following (exact) coverage probability:

Please note that the interval (X̅ - 1.96𝜎/√𝑛, X̅ + 1.96𝜎/√𝑛) is a random interval. The above probability statement can be interpreted as: Prior to the random experiment, the probability is 0.95 for the random interval (X̅ - 1.96𝜎/√𝑛, X̅ + 1.96𝜎/√𝑛) to include the unknown fixed parameter 𝜇. Suppose the experiment yields Ⅹ₁ = 𝑥₁, Ⅹ₂ = 𝑥₂, ..., Ⅹₙ = 𝑥ₙ then the sample value of X̅ is x̅ = (𝑥₁+𝑥₂+...+𝑥ₙ)/𝑛, a known number. Now the observed interval (x̅ - 1.96𝜎/√𝑛, x̅ + 1.96𝜎/√𝑛) is no longer random since both the lower and upper limits of this interval are fixed numbers. As a result, this interval either does or does not include 𝜇. Obviously, we cannot say that 0.95 is the probability that this particular interval includes the parameter 𝜇. However, we do have some confidence in this interval since, prior to the experiment, there is a high probability that the random interval (X̅ - 1.96𝜎/√𝑛, X̅ + 1.96𝜎/√𝑛) includes the parameter 𝜇. This confidence is then reflected by calling the observed interval (x̅ - 1.96𝜎/√𝑛, x̅ + 1.96𝜎/√𝑛) a 95% confidence interval for 𝜇.

A few important points to note:

• A 95% CI means that if we independently repeat the same sampling process many times and compute the 95% CI for each sample, then approximately 95% of the CIs will contain the true parameter value (𝜇) - see the graph above. For any particular sample, however, we will not know if the CI produced contains the true parameter value.

• A 95% CI does not mean that there is 95% chance that the particular interval contains the true parameter value - a Bayesian type of probability statement but apparently incorrect.

• The interval limits represent a plausible range of the true parameter value, not the probability distribution within.

A fundamental difference between the frequentist and Bayesian inferences is their assumptions about the mechanism of randomness (or uncertainty). Frequentist probabilities always describe the random behavior of the statistics (constructed from random samples), not of the parameters. In contrast, the Bayesian approach mainly concerns the uncertainty in the parameters and also embraces the idea that probabilities change based on new information.