That seems fine though. If the model behaves badly on any input we can test that. If the model wants to behave well on every input, then we’re happy. If it wants to behave badly on some input, we’ll catch it.
Are you concerned that we can’t test whether the model is behaving badly on a particular input? I think if you have that problem you are also in trouble for outer alignment.
When you say “test” do you mean testing by writing a single program that outputs whether the model performs badly on a given input (for any input)?
If so, I’m concerned that we won’t be able to write such a program.
If not (i.e. if we only assume that human researchers can safely figure out whether the model behaves badly on a given input), then I don’t understand how we can use Opt to find an input that the model behaves badly on (in a way that would work even if deceptive alignment occurs).
When you say “test” do you mean testing by writing a single program that outputs whether the model performs badly on a given input (for any input)?
If so, I’m concerned that we won’t be able to write such a program.
That’s the hope. (Though I assume we mostly get it by an application of Opt, or more efficiently by modifying our original invocation of Opt to return a program with some useful auxiliary functions, rather than by writing it by hand.)
That seems fine though. If the model behaves badly on any input we can test that. If the model wants to behave well on every input, then we’re happy. If it wants to behave badly on some input, we’ll catch it.
Are you concerned that we can’t test whether the model is behaving badly on a particular input? I think if you have that problem you are also in trouble for outer alignment.
When you say “test” do you mean testing by writing a single program that outputs whether the model performs badly on a given input (for any input)?
If so, I’m concerned that we won’t be able to write such a program.
If not (i.e. if we only assume that human researchers can safely figure out whether the model behaves badly on a given input), then I don’t understand how we can use Opt to find an input that the model behaves badly on (in a way that would work even if deceptive alignment occurs).
That’s the hope. (Though I assume we mostly get it by an application of Opt, or more efficiently by modifying our original invocation of Opt to return a program with some useful auxiliary functions, rather than by writing it by hand.)