By “predict sufficiently well” do you mean “predict such that we can’t distinguish their output”?
Unless the noise is of a special form, can’t we distinguish $f$ and $tilde{f}$ by how well they do on $f$’s goals? It seems like for this not to be the case, the noise would have to be of the form “occasionally do something weak which looks strong to weaker agents”. But then we could get this distribution by using a weak (or intermediate) agent directly, which would probably need less compute.
Suppose “predict well” means “guess the output with sufficiently high probability,” and the noise is just to replace the output with something random 5% of the time.
Yeah, I had something along the lines of what Paul said in mind. I wanted not to require that the circuit implement exactly a given function, so that we could see if daemons show up in the output. It seems easier to define daemons if we can just look at input-output behaviour.
By “predict sufficiently well” do you mean “predict such that we can’t distinguish their output”?
Unless the noise is of a special form, can’t we distinguish $f$ and $tilde{f}$ by how well they do on $f$’s goals? It seems like for this not to be the case, the noise would have to be of the form “occasionally do something weak which looks strong to weaker agents”. But then we could get this distribution by using a weak (or intermediate) agent directly, which would probably need less compute.
Suppose “predict well” means “guess the output with sufficiently high probability,” and the noise is just to replace the output with something random 5% of the time.
Yeah, I had something along the lines of what Paul said in mind. I wanted not to require that the circuit implement exactly a given function, so that we could see if daemons show up in the output. It seems easier to define daemons if we can just look at input-output behaviour.