I won’t deny probably misunderstanding parts of LDA but if the point is to learn corrigibility from H couldn’t you just say that corrigibility is a value that H has? Then use the same argument with “corrigibility” in place of “value”? (This assumes that corrigiblity is entirely defined with reference to H. If not, replace with the subset that is defined entirely from H, if that is empty then remove H).
If A[*] has H-derived-corrigibility then so must A[1] so distillation must preserve H-derived-corrigibility so we could instead directly distill H-derived-corrigibility from H which can be used to directly train a powerful agent with that property, which can then be trained from some other user.
so we could instead directly distill H-derived-corrigibility from H which can be used to directly train a powerful agent with that property
I’m imagining the problem statement for distillation being: we have a powerful aligned/corrigible agent. Now we want to train a faster agent which is also aligned/corrigible.
If there is a way to do this without starting from a more powerful agent, then I agree that we can skip the amplification process and jump straight to the goal.
I won’t deny probably misunderstanding parts of LDA but if the point is to learn corrigibility from H couldn’t you just say that corrigibility is a value that H has? Then use the same argument with “corrigibility” in place of “value”? (This assumes that corrigiblity is entirely defined with reference to H. If not, replace with the subset that is defined entirely from H, if that is empty then remove H).
If A[*] has H-derived-corrigibility then so must A[1] so distillation must preserve H-derived-corrigibility so we could instead directly distill H-derived-corrigibility from H which can be used to directly train a powerful agent with that property, which can then be trained from some other user.
I’m imagining the problem statement for distillation being: we have a powerful aligned/corrigible agent. Now we want to train a faster agent which is also aligned/corrigible.
If there is a way to do this without starting from a more powerful agent, then I agree that we can skip the amplification process and jump straight to the goal.