Will an LLM report >50% score on ARC in 2025?
Plus
18
Ṁ37342025
82%
chance
1D
1W
1M
ALL
ARC is a general-purpose AI eval designed to test intelligence as opposed to memorization. https://arcprize.org/arc
This market resolves to Yes if there are public demonstrations of general-purpose base models from OpenAI, Anthropic or Deepmind that solve at least 50% of the ARC questions before 2026.
(note that this is separate from winning the arc price, which requires only using open-source models)
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
Will GPT4/Opus report >50% score on ARC in 2024?
28% chance
LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?
60% chance
Will an LLM be able to solve the Self-Referential Aptitude Test before 2025?
19% chance
At EOY 2024, who will have the best LLM?
Which company will have the best LLM by the end of 2024?
Will I think that the top Chatbot Arena scores accurately reflect which LLMs are most capable and useful at EOY 2024?
41% chance
Will an LLM have a context window of 20M or more in 2024?
21% chance
Will LLMs mostly overcome the Reversal Curse by the end of 2025?
67% chance
Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?
66% chance
Who will be ahead in the AI/LLM war by the end of 2024?