Announcing ARC-AGI-3 - An benchmark that tests if AI can explore, learn, and adapt in unfamiliar situations. Humans score 100%. Frontier AI scores 0.26%. - Lemmy.ca

The ARC Prize organization designs benchmarks which are specifically crafted to demonstrate tasks that humans complete easily, but are difficult for AIs like LLMs, “Reasoning” models, and Agentic frameworks. > ARC-AGI-3 is the first fully interactive benchmark in the ARC-AGI series. ARC-AGI-3 represents hundreds of original turn-based environments, each handcrafted by a team of human game designers. There are no instructions, no rules, and no stated goals. To succeed, an AI agent must explore each environment on its own, figure out how it works, discover what winning looks like, and carry what it learns forward across increasingly difficult levels. > > Previous ARC-AGI benchmarks predicted and tracked major AI breakthroughs, from reasoning models to coding agents. ARC-AGI-3 points to what’s next: the gap between AI that can follow instructions and AI that can genuinely explore, learn, and adapt in unfamiliar situations. You can try the tasks yourself here: https://arcprize.org/arc-agi/3 [https://arcprize.org/arc-agi/3] Here is the current leaderboard for ARC-AGI 3, using state of the art models - OpenAI GPT-5.4 High - 0.3% success rate at $5.2K - Google Gemini 3.1 Pro - 0.2% success rate at $2.2K - Anthropic Opus 4.6 Max - 0.2% success rate at $8.9K - xAI Grok 4.20 Reasoning - 0.0% success rate $3.8K. ARC-AGI 3 Leaderboard [https://lemmy.ca/pictrs/image/c7521941-7eac-46f4-98de-876bcf99220c.png] (Logarithmic cost on the horizontal axis) https://arcprize.org/leaderboard [https://arcprize.org/leaderboard]