I think this would be more informative for the community if we had answers to the following questions here:
What are the AI states of the art on these problems?
How have the SoTAs changed over time?
What is human performance on these problems (top human performance, average, any other statistics, the whole distribution, etc., whichever seems most useful)?
(Anyone can answer, and feel free to provide only partial information. I’m guessing the authors have a lot of this info handy already.)
I think this would be more informative for the community if we had answers to the following questions here:
What are the AI states of the art on these problems?
How have the SoTAs changed over time?
What is human performance on these problems (top human performance, average, any other statistics, the whole distribution, etc., whichever seems most useful)?
(Anyone can answer, and feel free to provide only partial information. I’m guessing the authors have a lot of this info handy already.)