Unfortunate name collision: you’re looking at numbers on the AI2 Reasoning Challenge, not Chollet’s Abstraction & Reasoning Corpus.
Thanks for clarifying! I just tried a few simple ones by prompting gpt-4o and gpt-4 and it does absolutely horrific job! Maybe trying actually good prompting could help solving it, but this is definitely already an update for me!
Unfortunate name collision: you’re looking at numbers on the AI2 Reasoning Challenge, not Chollet’s Abstraction & Reasoning Corpus.
Thanks for clarifying! I just tried a few simple ones by prompting gpt-4o and gpt-4 and it does absolutely horrific job! Maybe trying actually good prompting could help solving it, but this is definitely already an update for me!