Applying signal detection theory to evaluate bias in expert and novice predictions for NFL games

The standard signal detection theory (SDT) model often uses an unbiased optimal criterion based on the assumption that the signal and noise distributions have roughly equivalent frequencies of occurrence. However, in some situations, optimal decisions should exhibit some partiality toward one distribution over the other. A real-world example is choosing between the home and away team in a sporting contest, since home teams do have a greater probability of winning. We considered the context of experts and novices predicting the winning team for the 256 games in the 2017 National Football League (NFL) season. We applied hierarchical SDT models to expert predictions provided by nflpickwatch.com and novice predictions collected during the 2017 NFL season to evaluate different biases in their predictions. We were particularly interested in the following biases: (1) home team advantage, (2) the cumulative win-loss record of teams, (3) herding by making the same prediction as other experts, (4) selecting the team with an unexpected win from the previous week, and (5) selecting against the team with an unexpected loss from the previous week. We then investigated patterns in how experts and novices used the 5 biases with a latent trait extension to our hierarchical SDT model. Applying the SDT models provides a way to measure the under- or over-reliance that experts and novices have on these biases when making predictions, and the latent trait extension helps us evaluate differences between expert and novice use of the biases.