Why would AGI research be anything other than recursion.
We make a large benchmark of automatically gradeable cognitive tasks. Things like “solve all these multiple choice tests” from some enormous set of every test given in every program at an institution willing to share.
“Control this simulated robot and diagnose and repair these simulated machines”
“Control this simulated robot and beat Minecraft”
“Control this simulated robot and wash all the dishes”
And so on and so forth.
Anyways, some tasks would be “complete all the auto gradeable coursework for this program of study in AI” and “using this table of information about prior attempts, design a better AGI to pass this test”.
We want the machine to have generality—use information it learned from one task on others—and to perform well on all the tasks, and to make efficient use of compute.
So the scoring heuristic would reflect that.
The “efficient use of compute” would select for models that don’t have time to deceive, so it might in fact be safe.
Why would AGI research be anything other than recursion.
We make a large benchmark of automatically gradeable cognitive tasks. Things like “solve all these multiple choice tests” from some enormous set of every test given in every program at an institution willing to share.
“Control this simulated robot and diagnose and repair these simulated machines”
“Control this simulated robot and beat Minecraft”
“Control this simulated robot and wash all the dishes”
And so on and so forth.
Anyways, some tasks would be “complete all the auto gradeable coursework for this program of study in AI” and “using this table of information about prior attempts, design a better AGI to pass this test”.
We want the machine to have generality—use information it learned from one task on others—and to perform well on all the tasks, and to make efficient use of compute.
So the scoring heuristic would reflect that.
The “efficient use of compute” would select for models that don’t have time to deceive, so it might in fact be safe.