philip_b comments on Iterated Distillation and Amplification

philip_b 21 Jun 2020 19:00 UTC
3 points
I think there are 2 mistakes in the pseudocode.

First mistake

what rmoehn said.

Second mistake

In the personal assistant example you say

In the next iteration of training, the Amplify(H, A[0]) system takes over the role of H as the overseer.

which implies that we do
```
H <- Amplify(H, A)
```
But in the pseudocode the original human overseer acts as the overseer all the time.

Suggested change of the pseudocode, which fixes both mistakes
```
def IDA(H):
   repeat:
      A ← Distill(H)
      H ← Amplify(H, A)
```
- Rafael Harth 1 Aug 2020 9:17 UTC
  6 points
  Parent
  I think H is always the same. In fact, H is a human, so it doesn’t make any sense to have code of the form $H \leftarrow x$ . In every step, a new system $A^{(t + 1)}$ is trained by letting a regular human oversee it, where the human has access to the system $A^{(t)}$ .
  Conversely, your code would imply that the human itself is replaced with something, and that thing then uses the system $A^{(t)}$ . This does not happen.
  (Unless my understanding is widely off; I’m only reading this sequence for the second time.)