How to construct regression models for observational studies (and how NOT to do it!)

Excerpt

This pithy aphorism, in various forms, is attributed to the eminent statistician, George Box. Despite its apparent simplicity, it conveys a number of deep issues. First, it reminds us that all statistical analyses are in some sense attempts to model real world phenomena. Second, it asserts that no statistical model is perfect and therefore must be wrong in some sense. Models can be imperfect for various reasons, such as incorrect assumptions, improper specification, and flawed model building strategies (among others). Finally, we are reminded that, despite the imperfect models we can build, some can be useful. Although the usefulness of a model is closely related to the primary goal of an analysis, for most of the discussion that follows, a useful model is defined as one that informs us about the population with minimal distortion and permits the researcher to generalize from a specific sample to the population with reasonable comfort. An important corollary is that some models are not useful. It should come as no surprise that good methodology is an essential ingredient when producing useful models. Unfortunately, much model building seen in the literature continues to employ flawed methodology. The remainder of this article discusses some of these flaws and presents alternatives. To provide context, we begin with a hypothetical scenario. …