Toward the adaptive design of General Recognition Theory experiments: A control study

General Recognition Theory (GRT) is a framework for characterizing perceptual independence and separability. Experiments employing GRT follow a factorial complete identification paradigm where participants must distinguish stimuli based on their combined features (e.g., shape and color; brightness and loudness). Inferences are based on the patterns of confusions, hence, stimuli must be similar enough to produce errors but not so similar that a participant cannot discriminate them. Traditionally, the feature levels are determined through pilot testing and fixed across all participants. However, this pilot testing is time-consuming and using the same stimulus levels for all participants leads to problems due to individual differences. Participants who are too sensitive make too few errors, and participants who are not sensitive enough end up guessing. Furthermore, hardware differences may cause confounds for online studies and hamper replication attempts. We previously introduced a method for adapting the design of GRT experiments to individual participants based on the Psi psychophysical method. Simulation results indicated the efficacy of our approach and its robustness to violations of the measurement model’s assumptions. As part of a validation study with human participants, we ran a control study using the traditional pilot testing approach as a baseline for comparison. In Experiment 1, ten participants performed a complete identification task with stimuli defined by their size and orientation (separable condition). In Experiment 2, another ten participants performed the task with stimuli defined by their saturation and brightness (integral condition). We observed more violations of marginal response invariance in the integral condition than in the separable condition, but the difference could have been larger. Many participants performed near the intended accuracy criterion, but some participants achieved near perfect accuracy and others were near chance on one or both dimensions. We discuss how these results leave room for improvement through adaptive experimental design.