Decision, Vol 11(1), Jan 2024, 60-85; doi:10.1037/dec0000211
Previous research shows that variation in coherence (i.e., degrees of respect for axioms of probability calculus), when used as a basis for performance-weighted aggregation, can improve the accuracy of probability judgments. However, many aspects of coherence-weighted aggregation remain a mystery, including both prescriptive issues (e.g., how best to use coherence measures) and theoretical issues (e.g., why coherence-weighted aggregation is effective). Using data from six experiments in two earlier studies (N = 58, N = 2,858) employing either general-knowledge or statistical information integration tasks, we addressed many of these issues. Of prescriptive relevance, we examined the effectiveness of coherence-weighted aggregation as a function of judgment elicitation method, group size, weighting function, and the bias of the function’s tuning parameter. Of descriptive relevance, we propose that coherence-weighted aggregation can improve accuracy via two distinct, task-dependent routes: a causal route in which the bases for scoring accuracy depend on conformity to coherence principles (e.g., Bayesian information integration) and a diagnostic route in which coherence serves as a cue to correct knowledge. The findings provide support for the efficacy of both routes, but they also highlight why coherence weighting, especially the most biased forms, sometimes imposes costs to accuracy. We conclude by sketching a decision–theoretic approach to how aggregators can sensibly leverage the wisdom of the coherent within the crowd. (PsycInfo Database Record (c) 2024 APA, all rights reserved)