A Taste of Optimal Designs

Suppose we want to use a two-pan balance (no bias) to weigh four different fruits: an apple, a pearl, an orange, and a banana (see below). For simplicity, standard weights are always placed in one pan, while the fruits can be placed in either pan. The goal is to determine (estimate) the weights of these fruits as accurately as possible using exact four weighings.

-----------------------------------------------------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------------------------------------------------

The following are three different methods for weighing these fruits. For each method, we present the corresponding statistical model and analysis results. As the notation used here is standard, further clarification is omitted.

----------------------------------------------------------- Method 1 ------------------------------------------------------------

----------------------------------------------------------- Method 2 ------------------------------------------------------------

----------------------------------------------------------- Method 3 ------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------------------

Method 3 clearly yields the best estimates among these three methods. Intuitively, it also appears to be the best among all possible methods, as the minimum achievable variance for each estimate (â, b̂, ĉ, d̂) is likely to be σ²/4 - for example, when the same fruit is weighed four times. We will see that Method 3 is indeed an optimal design for this weighing experiment.

Consider a more general scenario where 𝑛 objects are to be weighted in 𝑛 weighings with a two-pan balance (no bias). Let

where 𝑖, 𝑗 = 1, 2, ..., 𝑛, then the 𝑛×𝑛 matrix 𝐗 = (𝑥ᵢⱼ) completely characterizes the weighing experiment.

Let 𝐘 and 𝞫 denote the column vectors of the readings and true weights, respectively, then the readings can be represented by the linear model

where 𝝴 is the error vector with mean zero and covariance matrix σ²𝑰 (𝑰 is the identity matrix). Assume that 𝐗 is non-singular, then the best linear unbiased estimator of 𝞫 is

with covariance matrix

Hotelling (1944) proved that for any weighing design, the variance of the estimated weight cannot be smaller than σ²/𝑛. Therefore, a design 𝐗 is considered optimal if it estimates each weight with this minimum variance, σ²/𝑛. It can be shown that 𝐗 is optimal if and only if the (Fisher) information matrix satisfies 𝐗ᵀ𝐗 = 𝑛𝑰. For 𝑛 = 4, it is straightforward to verify that 𝐗ᵀ𝐗 equals 𝑰, 2𝑰, and 4𝑰 for Methods 1, 2 and 3, respectively. This confirms that Method 3 constitutes an optimal design for the weighing problem described.

An optimal 𝑛×𝑛 weighing design 𝐗 is simultaneously A-, D-, and E-optimal in the following sense.

• A-optimal: the trace of (𝐗ᵀ𝐗)⁻¹ is minimum among all 𝑛×𝑛 weighing designs.

• D-optimal: the determinant of (𝐗ᵀ𝐗)⁻¹ is minimum among all 𝑛×𝑛 weighing designs.

• E-optimal: the maximum eigenvalue of (𝐗ᵀ𝐗)⁻¹ is minimum among all 𝑛×𝑛 weighing designs.

References

Hotelling, H. (1944). Some Improvement in Weighing and Other Experimental Techniques. Annals of Mathematical Statistics 15, 297-306.