Past event
Department of Economics Brown Bag with Professor David Jaeger Robustness? Range Tests for Equality and Equivalence Across Specifications
Abstract: Applied economists routinely compare the results from multiple specifications, observe that they are “similar,” and conclude that the results are “robust.” This common practice is an implicit inferential claim about the range of estimates, but it ignores the joint sampling distribution of estimates computed from the same data. I formalize robustness claims and bring to them the kind of inferential discipline that the credibility revolution has brought to point estimation and inference. I propose two statistics that characterize robustness claims about a set of estimates. The minimum equivalence bound, ω→(ε), is the smallest tolerance within which the estimates can be judged equivalent at level ε. The equality p-value, pR, asks whether the estimates are statistically distinguishable. The two statistics differ only in whether the bootstrap imposes a common estimand, and together they distinguish lack of detectable differences from genuine agreement. Because the bootstrap captures the joint sampling distribution of all the estimates, the framework avoids parametric covariance estimation, does not require a privileged baseline specification, and applies broadly whenever joint bootstrap is feasible. Simulations show approximately correct size and demonstrate that informal eyeballing often provides false comfort. Applications to five prominent papers validate some conventional robustness claims while revealing fragility in others that is not apparent from casual inspection. I suggest that ω→ and pR be reported whenever multiple specifications are used to support a claim of robustness.