Close
This site uses cookies

By using this site, you consent to our use of cookies. You can view our terms and conditions for more information.

Return to Session

Using parameter contours to achieve more robust model estimation

Authors
Ms. Sabina Johanna Sloman
Carnegie Mellon University ~ Department of Social and Decision Sciences
Stephen Broomell
Carnegie Mellon University ~ School of Decision Sciences
Daniel Oppenheimer
Carnegie Mellon University, United States of America
Abstract

Many current practices in parameter estimation and model evaluation rely on fit statistics, calculated on the basis of estimated parameterizations of competing models. However, the design of an experiment can influence the conclusions a modeler draws about the parameterizations and relative performance of these models (Broomell, Sloman, Blaha, and Chelen, 2019). We highlight the importance of mapping the model-stimulus space, i.e. understanding how the parameter-dependent predictions of a model change across different stimuli. To achieve this goal, we represent models as a topography across the stimulus space, in which adjacent contour lines are defined by adjacent parameter values. Using data simulated from models of decision-making, we show how our proposed techniques can identify conditions under which traditional parameter estimation techniques will lead to inconclusive and inconsistent results. We also discuss ways in which modelers could exploit these insights to develop experimental designs for more robust parameter estimation. In addition, we demonstrate how a better understanding of the model-stimulus space can help researchers design powerful experiments to diagnose data generated by hypothesized models. Finally, we explore the conceptual implications of representing cognitive models as a topography of the stimulus space.

Tags

Keywords

model comparison
experimental design
decision-making
prospect theory
parameter estimation

Topics

Cognitive Modeling
Decision Making
Probabilistic Models
Model Analysis and Comparison
Study design
Discussion
New
Use of TC Last updated 3 months ago

Great project. Topographical consistency could be used for multiple purposes, and thus possibly have multiple formal definitions depending on the application. It could help developers evaluate design decisions of models. As in your example, probing for consistency and contours can reveal new behaviors that should be explored with models.

Mark Pitt 3 comments
How to define contours? Last updated 3 months ago

One question: I have problems understanding what defines a parameter contour. Is it any line or region in a plot that follows from a model with fixed parameter values? In other words, can the axes be anything as long as a fixed set of parameter values produces more than one predicted data point? Thanks for the interesting presentation!

Dr. Henrik Singmann 2 comments