Is the claim here that the 2^200 “persuasive ideas” would actually pass the scrutiny of top human researchers (for example, Paul Christiano studies one of them for a week and concludes that it is probably a full solution)? Or do you just mean that they would look promising in a shorter evaluation done for training purposes?
Assuming humans can always find the truth eventually, the number of persuasive ideas probably shrinks as humans have more time—maybe 2^300 in a training loop, 2^250 for Paul thinking for a day, 2^200 for Paul thinking for a month, 2^150 for Paul thinking for 5 years… I think the core point still applies.
Is the claim here that the 2^200 “persuasive ideas” would actually pass the scrutiny of top human researchers (for example, Paul Christiano studies one of them for a week and concludes that it is probably a full solution)? Or do you just mean that they would look promising in a shorter evaluation done for training purposes?
Assuming humans can always find the truth eventually, the number of persuasive ideas probably shrinks as humans have more time—maybe 2^300 in a training loop, 2^250 for Paul thinking for a day, 2^200 for Paul thinking for a month, 2^150 for Paul thinking for 5 years… I think the core point still applies.