A robust Bayesian test for context effects in multi-attribute decision making

In the past decades, context effects have been crucial in the development of cognitive models of decisions between multi-attribute alternatives. Nevertheless, to this date, only few studies have discussed what the best practices to analyze context effects are. Context effects occur when participants prefer identical options more or less depending on the choice set they are embedded in. Context effects are measured using what is called the Relative choice Share of the Target (RST), i.e., the change in preference of a target option from one choice set to the next. In this talk, we discuss two ways of calculating the RST: one frequently used in the literature, and a novel one we propose. Through simulations, we show that our proposed RST analyses overcome shortcomings of the more traditional approach. In particular, it is resistant to biases due to unequal sample size across choice sets. Furthermore, we apply our model to four previously published context effect studies, and we show that some reported context effects can change substantially (from significant to non-significant and vice versa). Implications of these results for cognitive modeling and empirical research on context effects are further discussed.