OpenAI’s 38.3% ARC-AGI-3 Score Collapses to 7.8% When Tested with Official Harness
OpenAI’s GPT-5.6 Sol hit 38.3% on ARC-AGI-3 using proprietary Retained Reasoning and Compaction settings, but the same model fell to 7.8% on the benchmark’s public harness, sparking a debate over fair evaluation and the role of inference-stack tricks.