The Performance Gap Is Stark
The numbers tell the story. GPT-5.6 Terra achieved just 2.3% accuracy on the Public ARC-AGI-3 tasks and 0.80% on the Semi-Private ARC-AGI-3 tasks. For context, the model performs dramatically better on easier benchmarks within the same family. This sharp drop-off suggests the model hits a wall when tackling abstract reasoning challenges that the ARC-AGI-3 suite is designed to expose.
Reasoning Variants Show Consistent Weakness
OpenAI tested five different reasoning effort levels: Max, Extra High, High, Medium, and Low. The expectation would be that more computational resources yield better results. Instead, the data reveals something counterintuitive and troubling.
| Reasoning Level | ARC-AGI-1 | ARC-AGI-2 | ARC-AGI-3 |
|---|---|---|---|
| Max | 96.5% | 83.9% | 0.8% |
| Extra High | 94.0% | 74.2% | 0.7% |
| High | 92.0% | 67.1% | 0.5% |
| Medium | 77.0% | 37.5% | 0.1% |
| Low | 60.2% | 18.8% | 0.0% |
The pattern is clear: as you move down from Max to Extra High to High, performance on ARC-AGI-3 either stays flat or declines. Even the most basic reasoning levels (Medium and Low) are essentially useless for this benchmark, scoring 0.1% and 0.0% respectively.
More Thinking Time Doesn’t Help
The counterintuitive finding here is that allocating more computational resources for reasoning does not improve results. The Max configuration scored 0.8%, but Extra High dropped to 0.7%. This pattern repeats across specific tasks within the ARC-AGI-3 Public Demo environments.
Take task FT09 as an example. With Max reasoning, it achieved 28% success. Increase the reasoning effort to Extra High, and it drops to 5%. Bump it to High, and it stays at 5%. Other environments like AR25 and SC25 show 0% success across nearly every reasoning level tested.
This suggests the problem is not computational budget. The model is not simply running out of processing power. Something deeper is broken in how the model approaches these problems.
What This Reveals About Current AI
GPT-5.6 Terra’s struggles point to a fundamental architectural or algorithmic limitation. The model excels at interpolating patterns within familiar domains (hence the 96.5% on ARC-AGI-1), but it cannot generalize to abstract reasoning tasks that demand genuine problem-solving insight.
Throwing more compute at the problem does not fix this. A brute-force approach to reasoning appears insufficient for making progress toward artificial general intelligence. Future models will need to address how the system actually understands and plans solutions, not just how much processing time it gets.
While other models in the GPT-5.6 family like Sol and Luna may perform differently, GPT-5.6 Terra’s results expose where today’s LLMs truly struggle. Until those core comprehension gaps are solved, benchmark results like these will keep appearing.
Follow Hashlytics on Bluesky, LinkedIn, Telegram and X to Get Instant Updates
