Once it observes that covert tampering did in fact occur, it stops assuming that the human will be correct. (Since the most likely explanation is either that the human messed up, or that the model underestimated human abilities.) It seems like it won’t end up assuming that both tampering occurred to show a diamond and that the diamond was actually present.
But the neat thing is that there is no advantage, either to the size of the computational graph or in predictive accuracy, to doing that. In the training set the human is always right. Regular reporters make mistakes because what is seen on camera is a non-robust feature that generalizes poorly, here we have no such problems.
But I might have misunderstood, pseudocode would be useful to check that we can’t just remove “function calls” and get a better system.
But the neat thing is that there is no advantage, either to the size of the computational graph or in predictive accuracy, to doing that. In the training set the human is always right. Regular reporters make mistakes because what is seen on camera is a non-robust feature that generalizes poorly, here we have no such problems.
But I might have misunderstood, pseudocode would be useful to check that we can’t just remove “function calls” and get a better system.