Close
This site uses cookies

By using this site, you consent to our use of cookies. You can view our terms and conditions for more information.

Return to Session

Flexibility in reaction time analysis: Many roads to a false positive?

Authors
Dr. Luis Morís Fernández
Universidad Autónoma de Madrid ~ Basic Psychology
Miguel A. Vadillo
Universidad Autónoma de Madrid, Spain
Abstract

In the present talk, we explore the influence of undisclosed flexibility in the analysis of reaction times (RTs). RTs entail some degrees of freedom of their own, due to their skewed distribution, the potential presence of outliers and the availability of different methods to deal with these issues. Moreover, these degrees of freedom are usually not considered part of the analysis itself, but preprocessing steps that are contingent on data. We analysed the impact of these degrees of freedom on the false-positive rate using Monte Carlo simulations over real and simulated data. When several preprocessing methods are used in combination, the false-positive rate can easily rise to 17%. This figure becomes more concerning if we consider that more degrees of freedom are awaiting down the analysis pipeline, potentially making the final false-positive rate much higher. We propose that pre-registering studies would ameloriate this problem by reducing the degrees of freedom when analyzing RT data.

Tags

Keywords

Reaction-time
questionable research practices
p-hacking
false- positives

Topics

Reaction Times
Hypothesis Testing
Discussion
New

There is nothing here, yet. Be the first to create a thread