Abstract
Incremental validity testing (i.e., testing whether a focal predictor is associated with an outcome above and beyond a covariate) is common (e.g., 57% of Personal Relationships articles in 2017), yet it is fraught with conceptual and statistical problems. First, researchers often use it to overemphasize the novelty or counterintuitiveness of findings, which hinders cumulative understanding. Second, incremental validity testing requires that the focal predictor and the covariate represent separate constructs; researchers risk committing the “jangle fallacy” without such evidence. Third, the most common approach to incremental validity testing (i.e., standard multiple regression, 88% of articles) inflates Type I error and can produce invalid conclusions. This article also discusses the relevance of these issues to dyadic/longitudinal designs and offers concrete solutions.