Data Analysis Tools Reveal Hidden Factors Behind AI Model Performance

Researchers at OpenAI have made an unexpected discovery that sheds new light on the performance of their large language models, specifically GPT-5.6 Sol and its predecessor GPT-5.5. When these models were tested on the ARC-AGI-3 benchmark, a 2D puzzle game evaluation tool designed to measure AI agents’ ability to learn and reason, they scored surprisingly low. The scores of 7.8% for GPT-5.6 Sol and an abysmal 0.4% for GPT-5.5 raised questions about the models’ capabilities in this specific domain.