Cognitively and Linguistically Motivated Part of Speech Tagging: Quantitative Assessment of a Near Human-Scale Computational Cognitive Model

We provide a quantitative assessment of the part of speech tagging accuracy rate of the written word recognition subcomponent of the computational implementation of Double R Grammar. Double R Grammar is a cognitively and linguistically motivated near human-scale computational cognitive model of the grammatical analysis of written English which is focused on the grammatical encoding of two key dimensions of meaning: referential and relational meaning. Cognitively, the computational implementation is implemented in the ACT-R cognitive architecture. It contains a mental lexicon which encodes explicit declarative knowledge about lexical items and grammatical constructions, and a procedural memory which encodes implicit knowledge about how to grammatically analyze input expressions. With ~100,000 words and multi-word units, the size of the mental lexicon aligns with numerous estimates of the size of the human mental lexicon. The words were primarily borrowed from the COCA corpus and are assigned a part of speech specific base-level activation based on their frequency of use in that corpus. The retrieval of lexical items corresponding to input tokens depends on the spread of activation from the lexical, morphological, and grammatical context, and the base-level activation. Grammatical productions determine how to integrate retrieved lexical items and projected grammatical constructions into grammatical representations. There are ~2500 manually created productions that cover the common grammatical patterns of English. The basic processing mechanism is pseudo-deterministic in that it pursues the single best analysis, but is capable of non-monotonically adjusting to the evolving context. The processing mechanism adheres to two well established cognitive constraints on human language processing: incremental and interactive processing. Linguistically, Double R Grammar aligns with cognitive and construction grammar, and is strongly usage based. On a previously unseen sample corpus of book abstracts of spy novels and a few paragraphs of a Clive Cussler book, the computational implementation achieved a 98.48% part of speech tagging accuracy rate over 1838 tokens. On a second sample corpus of 8 abstracts of books on the topic of self-help and two political biographies, the computational implementation achieved a part of speech tagging accuracy rate of 98.56% over 766 tokens. Although this accuracy rate is not directly comparable to competing machine learning approaches trained over an annotated corpus, or deep learning approaches trained over big data, the current state of the art for part of speech tagging accuracy is in the neighborhood of 98% for systems trained on the annotated Penn Treebank corpus, using the Penn Treebank tagset which contains 36 atomic parts of speech organized into a flat listing with no internal structure. This compares to 56 non-atomic parts of speech with internal structure, organized into a multiple inheritance hierarchy in Double R Grammar.