I feels as if OpenAI acted as a client of Epoch AI ordering a math benchmark and that Epoch AI would not be working on it without this funding.
I also suspect that computer use/task execution benchmark EpochAI is hiring for currently (Technical Lead, Benchmarks) has the same arrangement with OpenAI.
William,
not exactly an answer to your question but BOTEC estimate for FrontierMath costs: $400k — $2M
EpochAI shared two job postings on their LinkedIn few months ago that seem relevant to FrontierMath:
Project lead compensation is $200k/year with expected contract length of 7 months: ~$100k.
Benchmark consists of about 1k problems, payment per problem is between $300 and $1k based on quality. Full-time contributors also have base rate of $2k/month with 2-4 months expected duration. Over 60 mathematicians have contributed.
Low amount assuming just project lead, minimally valued problems ($300), no full-time contractors, no overhead (e.g. taxes):
Medium amount assuming project lead, medium valued problems ($500), 10 full-time contractors working for 3 months, 30% of overhead:
High amount assuming project lead, exceptional problems ($1k), 60 full-time contractors working for 3 months, 50% of overhead:
I came to similar conclusions after another geopolitical event few years ago.