The BlueDot Impact write-up for scalable oversight seems to suggest that people have given up on iterated distillation and amplification (IDA) working. I haven’t really seen much research here, but is that actually the case?
I know that Ought is now pretty much inactive because their attempts at factored cognition failed and this has made many people pessimistic about the factored cognition hypothesis. However, Ought seemed to be really pushing the angle that we could break things into sub-problems each of which could be completed quickly, whilst I don’t think that’s a required part of IDA. Therefore it isn’t clear to me that the failure of factored cognition indicates the failure of IDA.
[Question] Have people given up on iterated distillation and amplification?
The BlueDot Impact write-up for scalable oversight seems to suggest that people have given up on iterated distillation and amplification (IDA) working. I haven’t really seen much research here, but is that actually the case?
I know that Ought is now pretty much inactive because their attempts at factored cognition failed and this has made many people pessimistic about the factored cognition hypothesis. However, Ought seemed to be really pushing the angle that we could break things into sub-problems each of which could be completed quickly, whilst I don’t think that’s a required part of IDA. Therefore it isn’t clear to me that the failure of factored cognition indicates the failure of IDA.