Bayesian hierarchical model approaches for disattenuating correlation in studies of individual differences

Recently, there has been a merger between experimental and differential Psychology where experimental tasks have been employed to probe individual differences. While this merger appears desirable, results have been problematic in two ways. First, correlations between tasks measuring the same construct are relatively low. For example, flanker and Stroop tasks are both assumed to measure the ability to inhibit the prepotent responses, yet performance on these tasks in the literature typically have correlations around .1 (Enkavi et al., 2019; Rey-Mermet, Gade, & Oberauer, 2018). Such low correlation values stand in contrast with findings in other domains where measures of abilities often have substantial positive correlations (Ritchie, 2015), a fact known as Spearman’s positive manifold. These low correlations undoubtedly reflect low reliability leading to the well-known problem of attenuation. Following from this, the second way the merger has been problematic is that latent variable analyses tend to be unstable and unreplicable (Karr et al., 2019). Although there are methods of disattenuation, their resulting correlations are often too variable to provide meaningful insights (Rouder, Haaf, & Kumar, in preparation). To address the current predicament, we propose a new method of disattenuation that leverages the positive manifold by assuming it as a prior in a Bayesian hierarchical model. With this constraint, correlations may be disattenuated with reasonable precision, even in low-reliability experimental settings. We compare the performance of this approach to relatively unconstrained Bayesian hierarchical models (such as those with LKJ and Wishart priors) and the more conventional Spearman correction for attenuation.