Type I error in diffusion models: A drift towards false positives?

Diffusion models are one of the most used tools to analyze reaction times (RT), and their relevance keeps growing overtime. According to these models, evidence is accumulated over time, until a threshold is reached, leading to a response. Contrary to simpler RT analysis approaches, these models are equipped with more parameters to be estimated, such as the drift rate, the threshold or non-decisional factors. This allows a more nuanced understanding of the process underlying the decision and response. Unfortunately, this higher number of parameters can also be problematic. We present a series of three simulations with Ratcliff's diffusion model. Simulation 1 used empirical data, Simulation 2 simulated data based on empirically estimated parameters and Simulation 3 was carried out with simulated data based on common distributions of the parameters. The three simulations show that commonly used statistical analyses in diffusion models can lead to an inflation of the Type I error rate. Different strategies to prevent this problem are discussed, including pre-registration of the analysis, model comparisons and Type I error corrections.