Deriving critical tests of ACT-R using systems factorial technology with global model analysis

Cognitive architectures (CAs) are unified theories of cognition which describe invariant properties in the structure and function of cognition, and how sub-systems (e.g., memory, vision) interact as a coherent system. An important role of CAs is integrating findings across many domains into a unified theory and preventing research silos. One downside of CAs is that their breadth and complexity create challenges for deriving critical tests of core architectural assumptions. Consequentially, it is often unclear to what extent empirical tests of CAs are driven by core architectural vs. auxiliary assumptions. To address this issue, we developed a methodology for deriving critical tests of CAs which combines systems factorial technology (SFT; Townsend & Nozawa, 1995) and global model analysis (GMA), forming what we call SFT-GMA. In SFT-GMA, GMA is performed within an SFT model space of qualitative model classes spanning four dimensions: architecture, stopping rule, dependence, and workload capacity. Constraints on the model space are derived from core architectural assumptions which may provide a basis for critical tests. To demonstrate the utility of SFT-GMA, we applied it to the ACT-R cognitive architecture (Anderson et al., 2004). Despite many degrees of freedom in the specification of parameters values, production rules, and declarative memory representations, SFT-GMA revealed that ACT-R’s core architectural assumptions impose testable constraints on the SFT model space. In particular, ACT-R is incompatible with most parallel SFT models of perceptual processing. We believe that the use of theorem-based methods such as SFT-GMA have the potential to stimulate theoretical progress for CAs. The views expressed in this paper are those of the authors and do not reflect the official policy or position of the Department of Defense or the US Government. This work was supported by the Air Force Research Laboratory (FA8650-22-C-1046). Approved for public release; distribution unlimited. Cleared 12/21/2023; Case Number: AFRL-2023-6387.