Single-trial estimates of sequential sampling models parameters are not just noisy but can also be biased
Traditionally, researchers estimate parameters of sequential sampling models (SSMs) from repeated choices across different conditions. Crucially, differences in parameters across conditions are interpreted as shifts in the underlying cognitive processes: For example, lower decision thresholds under high time pressure are interpreted as decreased cautiousness. Recent work has explored whether the parameters of SSMs can be estimated at a more detailed, single-trial level as well, to infer shifts in cognitive processes in subsequent trials. Such a more detailed window on decision-making processes has exciting applications. For example, by correlating single-trial estimates to neuroimaging data, we can relate specific brain areas to cognitive processes that may vary from trial to trial and not merely across conditions. The present work highlights some important limitations of such a powerful approach. First, we reproduce earlier work and show that single-trial estimates of SSM parameters are extremely noisy. We also show that single-trial SSM parameter estimates can be highly biased by the outcome of a choice. For example, single-trial estimates of the rate of evidence accumulation in incorrect choices are severely underestimated when compared to the generating single-trial parameter (and vice versa for correct choices). We will show how these problems can pollute the cognitive interpretation of single-trial parameters and can be exacerbated by correlations to process data. Finally, we offer a potential solution where SSMs that incorporate more information about trial-to-trial differences (e.g., stimulus or feedback properties) produce more reliable single-trial estimates.
Cool stuff! I was wondering whether your results depend on the fitting method: does it matter whether you use HDDM, the ML-fits, the chi-squared fits, etc? And then I was also wondering whether time on task is really a problem? I think the stationarity assumption of the DDM is the problem more than time on task, because it seems likely that atten...
Thank you for this talk Dr. Fontanesi! Do you mind clarifying some of the details of the model simulations? Are the models simulated with neural impulse functions or square-waves that were linearly related to a particular trial's drift rate or response time etc., and these neural impulses or square-waves gave rise to BOLD responses?