Using parameter contours to achieve more robust model estimation

Many current practices in parameter estimation and model evaluation rely on fit statistics, calculated on the basis of estimated parameterizations of competing models. However, the design of an experiment can influence the conclusions a modeler draws about the parameterizations and relative performance of these models (Broomell, Sloman, Blaha, and Chelen, 2019). We highlight the importance of mapping the model-stimulus space, i.e. understanding how the parameter-dependent predictions of a model change across different stimuli. To achieve this goal, we represent models as a topography across the stimulus space, in which adjacent contour lines are defined by adjacent parameter values. Using data simulated from models of decision-making, we show how our proposed techniques can identify conditions under which traditional parameter estimation techniques will lead to inconclusive and inconsistent results. We also discuss ways in which modelers could exploit these insights to develop experimental designs for more robust parameter estimation. In addition, we demonstrate how a better understanding of the model-stimulus space can help researchers design powerful experiments to diagnose data generated by hypothesized models. Finally, we explore the conceptual implications of representing cognitive models as a topography of the stimulus space.