On Quintin’s secondly’s concrete example 1 from above:
I think the core disagreement here is that Quintin thinks that you need very close parallels in order for the evolutionary example to be meaningful, and I don’t think that at all. And neither of us can fully comprehend why the other person is going with as extreme a position as we are on that question?
Thus, he says, yes of course you do not need all those extra things to get misalignment, I wasn’t claiming that, all I was saying was this would break the parallel. And I’m saying both (1) that misalignment could happen these other ways which he agrees with in at least some cases (but perhaps not all cases) and (2) also I do not think that these extra clauses are necessary for the parallel to matter.
And also (3) yes, I’ll totally cop to, because I don’t see why the parallel is in danger with these changes, I didn’t fully have in my head the distinction Quintin is trying to draw here, when I was writing that.
But I will say that, now that I do have it in my head, that I am at least confused why those extra distinctions are necessary for the parallel to hold, here? Our models of what is required here are so different that I’m pretty confused about it, and I don’t have a good model of why e.g. it matters that there are 9 OOMs of difference, or that the creation of the inner optimizer is deliberate (especially given that nothing evolution did was in a similar sense deliberate, as I understand these things at least—my model is that evolution doesn’t do deliberate things at all). And in some cases, to the extent that we want a tighter parallel, Quintin’s requirements seem to move away from that? Anyway, I notice I am confused.
Concrete example 4: Am I wrong here that you’re arguing that this path still exhibits key differences from cultural development and thus evolution does not apply? And then you also argue that this path does not cause additional severe alignment difficulties beyond those above. So I’m not sure where the misreading is here. After that, I discuss a disagreement with a particular claim.
On Quintin’s secondly’s concrete example 1 from above:
I think the core disagreement here is that Quintin thinks that you need very close parallels in order for the evolutionary example to be meaningful, and I don’t think that at all. And neither of us can fully comprehend why the other person is going with as extreme a position as we are on that question?
Thus, he says, yes of course you do not need all those extra things to get misalignment, I wasn’t claiming that, all I was saying was this would break the parallel. And I’m saying both (1) that misalignment could happen these other ways which he agrees with in at least some cases (but perhaps not all cases) and (2) also I do not think that these extra clauses are necessary for the parallel to matter.
And also (3) yes, I’ll totally cop to, because I don’t see why the parallel is in danger with these changes, I didn’t fully have in my head the distinction Quintin is trying to draw here, when I was writing that.
But I will say that, now that I do have it in my head, that I am at least confused why those extra distinctions are necessary for the parallel to hold, here? Our models of what is required here are so different that I’m pretty confused about it, and I don’t have a good model of why e.g. it matters that there are 9 OOMs of difference, or that the creation of the inner optimizer is deliberate (especially given that nothing evolution did was in a similar sense deliberate, as I understand these things at least—my model is that evolution doesn’t do deliberate things at all). And in some cases, to the extent that we want a tighter parallel, Quintin’s requirements seem to move away from that? Anyway, I notice I am confused.
Concrete example 4: Am I wrong here that you’re arguing that this path still exhibits key differences from cultural development and thus evolution does not apply? And then you also argue that this path does not cause additional severe alignment difficulties beyond those above. So I’m not sure where the misreading is here. After that, I discuss a disagreement with a particular claim.