What is more democratic, a stone or a feather? Predicting nonsensical choices using high-dimensional vector representations obtained from a semantic space model

Semantic space models are powerful tools in semantic memory research, which use the distributional structure of words in large natural language datasets to derive high dimensional vector representations for the words or concepts in a semantic space. In a recent line of research, these word vectors have been used to predict judgments of similarity, probability, or other quantities. If these spaces capture the structure of human conceptual representations, it should also be possible to predict comparative choices of concepts on nonsensical attributes as long as the concepts are spatially arranged at sufficiently distinct locations along the attribute dimension. In a first experiment, we presented n = 30 participants with k = 60 nonsensical comparisons, in order to investigate the ability of the semantic space model to predict the response of participants. Overall, the analysis using a Bayesian logistic hierarchical regression model showed that the model could predict the responses of participants above chance level, with an accordance rate of model-predicted and observed responses of θ = 57%. However, the results also showed that while there was only a small difference between participants (θ ranging from 53% to 56%), there were large differences between items in how good the model predicted the actual judgment of participants, with accordance rates ranging from θ = 36% to θ = 89%. Given that the observed responses of participants are similar and as predicted by the semantic space model, at least for some items, might indicate that the derived high dimensional vector representation of the semantic space to some extent incorporates some shared aspects of people’s semantic memory.