Social meta-inference and the evidentiary value of consensus

Reasoning beyond available data is a ubiquitous feature of human cognition. But while the availability of first-hand data typically diminishes as the concepts we reason about become more complex, our ability to draw inferences and reach conclusions seems not to. We may offset the sparsity of direct evidence by observing the statements and actions of others and inferring properties of evidence assumed to exist. But such social meta-inference comes with challenges of its own. For example, while we might infer the existence of evidence on the basis of a social consensus, its evidentiary strength is not immediately clear. Ideally it should be governed by the nature and extent of ground truth data from which the consensus was derived -- but this is the very thing that remains latent. Here, we present the results of an experiment aimed at examining people's perception of the evidentiary strength of social consensus in the context of social media posts. By systematically varying the degree of consensus along with the diversity of people and premises involved we are able to assess the contribution of each factor to evidentiary weight. Across a range of topics where reasoning from first-hand data is more or less difficult we find that while people were influenced by the number of people on each side of an argument, the number of posts was the dominant factor in determining how people updated their beliefs. However, in contrast to well established premise diversity effects, we find that people were largely insensitive to whether or not the posts represented distinct premises. We discuss the applied and theoretical implications of our findings.