Adaptive design for general recognition theory experiments
General recognition theory (GRT), a multivariate generalization of signal detection theory, is a powerful means for inferring the interaction of representations and decision processes when perceiving a multidimensional stimulus. In order for inferences to be made from a GRT experiment, stimuli must be sufficiently confusable so that subjects make identification errors. Stimulus intensities are typically chosen through repeated pilot testing, and the same stimuli are used for every subject in the experiment. This approach is time consuming on its own but can critically fail for some subjects due to individual differences. Here, we propose an algorithm to improve the effectiveness of GRT by adapting the design of the experiment to individual subjects. Our method leverages adaptive psychophysical methods (e.g., Psi, Quest+) to iteratively fit a highly constrained GRT model to a subject’s responses in real time. The algorithm converges rapidly on a rough approximation of the subject’s internal perceptual process by assuming perceptual independence, perceptual separability, and decisional separability. The user only needs to specify the intensity range of interest for each perceptual dimension of the stimulus for the adaptive process to generate reasonable stimuli to use in the main GRT experiment. When combined with the analysis code provided by existing R packages, our method permits a completely automated pipeline from hypothesis to data collection to statistical inference. We present the results from a simulation study assessing the recoverability and statistical properties of the algorithm and one human experiment comparing the adaptive process to the more traditional pilot testing approach.
Very clear talk. This is a good use of the psi methods to standardize stimuli for participants. How does it compare with staircase in this setup? Also, please elaborate on enhancing HCI, mentioned on the last slide. I could not make the connection.
I enjoyed this presentation. It was very clear and informative. What would be your guess about the scale of effectiveness of the adaptive method when compared to traditional approach (non adaptive)?