Note that P1 in this argument can be weakened to simply say that SAI is a non-trivial possibility but, in response, P2 and P3 need to be strengthened (that is, if SAI is less likely, it needs to be more dangerous or FAI research needs to be more effective in order for FAI research to have the same expected value).
This is arguing from the bottom line. If P1 is weakened, it in no way implies that P2 or P3 become stronger, instead it argues that expected value of FAI research is lower. If you are targeting a particular power of an argument, then it’s OK, but you also seem to be making assertions about the true values of these parameters, and the true values don’t work like that.
I read this section of the post as saying, “if one argument is weakened, the others need to be stronger [or else SI’s conclusions are no longer supported.]”
But if you don’t read that into it, it does sound very sketchy.
This is definitely what I meant—I apologise that it was unclear. So to clarify, I meant that the premises can still support the conclusion if P1 is weakened, as long as P2 and P3 are strong enough that the expected value of FAI research is still adequately high.
In some ways, the strength of P1, P2 and P3 can be traded off against one another. That means that the argument might be more convincing to more people—you can think SAI is unlikely but still think the conclusion is correct (though I’m certainly not arguing that if P1 gets weakened, P2 and P3 must be stronger because the bottom line is undeniable, or any such thing).
Perhaps I should have said: Note that P1 in this argument can be weakened to simply say that SAI is a non-trivial possibility but, in response, a stronger version of P2 and P3 are required if the conclusion is still to be viable (that is, if SAI is less likely, it needs to be more dangerous or FAI research needs to be more effective in order for FAI research to have the same expected value).
Unless anyone comments negatively regarding this new version of the paragraph in the next few hours, I’ll update the original post.
What’s wrong with the parent comment, what am I missing?
Edit: So this comment summoned some upvotes to the parent, but didn’t clarify the problem; staying at 0 Karma for several hours indicates that I’m probably missing something, and in this case I don’t have any plausible hypotheses to give more weight to based on that observation (besides “the idea that expected value of FAI research can be overestimated makes people flinch”, which seems weak)...
This is arguing from the bottom line. If P1 is weakened, it in no way implies that P2 or P3 become stronger, instead it argues that expected value of FAI research is lower. If you are targeting a particular power of an argument, then it’s OK, but you also seem to be making assertions about the true values of these parameters, and the true values don’t work like that.
I read this section of the post as saying, “if one argument is weakened, the others need to be stronger [or else SI’s conclusions are no longer supported.]”
But if you don’t read that into it, it does sound very sketchy.
This is definitely what I meant—I apologise that it was unclear. So to clarify, I meant that the premises can still support the conclusion if P1 is weakened, as long as P2 and P3 are strong enough that the expected value of FAI research is still adequately high.
In some ways, the strength of P1, P2 and P3 can be traded off against one another. That means that the argument might be more convincing to more people—you can think SAI is unlikely but still think the conclusion is correct (though I’m certainly not arguing that if P1 gets weakened, P2 and P3 must be stronger because the bottom line is undeniable, or any such thing).
Perhaps I should have said: Note that P1 in this argument can be weakened to simply say that SAI is a non-trivial possibility but, in response, a stronger version of P2 and P3 are required if the conclusion is still to be viable (that is, if SAI is less likely, it needs to be more dangerous or FAI research needs to be more effective in order for FAI research to have the same expected value).
Unless anyone comments negatively regarding this new version of the paragraph in the next few hours, I’ll update the original post.
What’s wrong with the parent comment, what am I missing?
Edit: So this comment summoned some upvotes to the parent, but didn’t clarify the problem; staying at 0 Karma for several hours indicates that I’m probably missing something, and in this case I don’t have any plausible hypotheses to give more weight to based on that observation (besides “the idea that expected value of FAI research can be overestimated makes people flinch”, which seems weak)...