Explaining away differences in face matching

Unfamiliar face processing is often studied in the context of face matching, where an observer judges whether two images depict the same individual. On matching trials, the two images depict the same person but differ by factors. On non-matching trials, the two images depict different people, chosen in part because of their resemblance to each other. Accurate performance benefits from a representation of identity that is invariant both to state-based changes (e.g., in viewpoint, pose, and illumination) and to structural or surface-level changes to the faces themselves — e.g., those caused by aging or body modification. Here, we cast the problem of face matching as one of causal inference where the observer infers whether the depicted person underwent a transformation or is a different person. We introduce a causal model of face matching in which the observer infers which factor best explains the observed differences between a pair of faces. Our model produces a classic phenomenon in causal inference — explaining away — whereby two independent causes become dependent conditioned on a common effect. We then provide support for the model in two experiments that asked participants to make face matching determinations and explain them. We find that observers have a rich understanding of the causal mechanisms that affect identity and appearance and can use that knowledge to make accurate inferences unattainable by approaches that rely only on feature detection and comparison.