LLM-Driven Cognitive Diagnosis Model Refines Student Assessment

Cognitive diagnosis, the backbone of online adaptive learning systems, has long relied on student answer logs and expert-built Q-matrices to quantify learners’ grasp of discrete knowledge concepts. However, conventional frameworks often focus solely on student-exercise interaction signals, neglecting parallel and synergistic connections that exist widely in educational knowledge systems.

The research team behind the HCGCDM model has addressed this limitation by introducing a graph-augmented cognitive diagnosis framework that leverages large language models to automatically map multi-type links between educational knowledge concepts. This approach enables more precise assessments of students’ learning proficiency and marks a significant improvement over existing methods.

Heterogeneous concept graphs, constructed through the HCGCDM’s two sequential functional modules, are at the heart of this innovation. The first module automates graph building without heavy manual annotation or large training datasets by embedding all knowledge concepts via BERT-base-nli-mean-tokens and filtering high-similarity concept pairs above a predefined cosine similarity threshold.

The framework then deploys RAG to pull domain-specific external knowledge bases, concatenating retrieved context with structured prompts to guide LLMs in generating relational triples covering three defined edge categories: prerequisite, parallel, and synergistic relations. A second RAG-powered LLM validation step screens unreliable triples to reduce graph noise, and validated triples form a unified heterogeneous graph split into three independent relation-specific subgraphs.

The second core module applies tailored graph neural network architectures to each subgraph to match relation characteristics. Two-layer graph attention networks process directed prerequisite subgraphs to capture asymmetric contribution weights of child concepts to parent nodes; two-layer graph convolutional networks encode parallel subgraphs to preserve hierarchical consistency among peer concepts; CoAttention-GNN handles bidirectional information flow within synergistic subgraphs to model mutual knowledge reinforcement.

The research team ran comparative tests against mainstream baseline CD models across multiple real-world educational datasets, confirming the effectiveness of HCGCDM in refining student cognitive diagnosis. Ablation experiments revealed that each core module delivers consistent performance gains, with the LLM-driven heterogeneous graph construction component producing the largest diagnostic metric shifts when removed.

Additional controlled trials swapped the base LLM between GPT-4o and Llama3-8B-Instruct, confirming that LLM capability correlates with triple construction accuracy and downstream diagnostic performance. The paper notes several existing limitations, including limited support for datasets lacking textual concept descriptions and reliance on predefined relation categories, and outlines future directions to expand automated relation extraction and incorporate multi-agent LLM pipelines for graph refinement.

The HCGCDM model has the potential to revolutionize cognitive diagnosis by providing a more comprehensive understanding of students’ learning proficiency. By leveraging large language models to automatically map complex relationships between educational knowledge concepts, this approach can help educators identify areas where students need additional support and tailor their instruction accordingly.