Identifying cognitive skills in student data with an application in education

Cognitive tutors typically use a student model to track progress of the learner. This model can be used to give feedback to teachers and students, and to select new material and assignments. Student models are typically constructed by modelers and/or education specialists. However, it is hard to assess whether the constructed student model aligns with knowledge and skills students actually need to master the material.
Instead, we propose a hybrid approach, in which we use bottom-up machine learning methods to use individual differences in student performance to construct a knowledge graph, in which each node represents a possible knowledge state of the student.
As a pilot, we constructed a knowledge graph for an arithmetic course in the mid-level vocational education (MBO) in the Netherlands. The basis for this graph was an math entry test, which, according to the publisher, addressed several specific topics, such as length measurements, weight, clock time, etc. However, when we constructed a knowledge graph from data from 413 students, we found that students do not differ on mastery of those topics, but rather on more general underlying skills, such as general arithmetic skills, reading skills and multi-step reasoning. A pilot conducted in two schools using a dashboard representing the knowledge graph was judged to be insightful and helpful by both teachers and students, and can serve as a basis for the construction of a cognitive tutor.