Yeah I also have the sense that we mostly agree here.
I have the sense that CEV stands for, very roughly, “what such-and-such a person would do if they became extremely wise”, and the hope (which I think is a reasonable hope) is that there is a direction called “wisdom” such that if you move a person far enough in that direction then they become both intelligent and benevolent, and that this eventually doesn’t depend super much on where you started.
The tricky part is that we are in this time where we have the option of making some moves that might be quite disruptive, and we don’t yet have direct access to the wisdom that we would ideally use to guide our most significant decisions.
And the key question is really: what do you do if you come into a position of really significant influence, at a time when you don’t yet have the tools to access the CEV-level wisdom that you might later get? And some people say it’s flat-out antisocial to even contemplate taking any disruptive actions, while others say that given the particular configuration of the world right now and the particular problems we face, it actually seems plausible that a person in such a position of influence ought to seriously consider disruptive actions.
I really agree with the latter, and I also contend that it’s the more epistemically humble position, because you’re not saying that it’s for sure that a pivotal act should be performed, but just that it’s quite plausible given the specifics of the current world situation. The other side of the argument seems to be saying that no no no it’s definitely better not to do anything like that in anything like the current world situation.
I also contend that it’s the more epistemically humble position, because you’re not saying that it’s for sure that a pivotal act should be performed, but just that it’s quite plausible given the specifics of the current world situation
The thing I’d say in favor of this position is that I think it better fits the evidence. I think the problem with the opposing view is that it’s wrong, not that it’s more confident. E.g., if I learned that Nate assigns probability .9 to “a pivotal act is necessary” (for some operationalization of “necessary”) while Critch assigns probability .2 to “a pivotal act is necessary”, I wouldn’t go “ah, Critch is being more reasonable, since his probability is closer to .5″.
I agree with the rest of what you said, and I think this is a good way of framing the issue.
I’d add that I think discussion of this topic gets somewhat distorted by the fact that many people naturally track social consensus, and try to say the words they think will have the best influence on this consensus, rather than blurting out their relevant beliefs.
Many people are looking for a signal that stuff like this is OK to say in polite society, or many others are staking out a position “the case for this makes sense intellectually but there’s no way it will ever attract enough support, so I’ll preemptively oppose it in order to make my other arguments more politically acceptable”. (The latter, unfortunately, being a strategy that can serve as a self-fulfilling prophecy.)
Yeah I also have the sense that we mostly agree here.
I have the sense that CEV stands for, very roughly, “what such-and-such a person would do if they became extremely wise”, and the hope (which I think is a reasonable hope) is that there is a direction called “wisdom” such that if you move a person far enough in that direction then they become both intelligent and benevolent, and that this eventually doesn’t depend super much on where you started.
The tricky part is that we are in this time where we have the option of making some moves that might be quite disruptive, and we don’t yet have direct access to the wisdom that we would ideally use to guide our most significant decisions.
And the key question is really: what do you do if you come into a position of really significant influence, at a time when you don’t yet have the tools to access the CEV-level wisdom that you might later get? And some people say it’s flat-out antisocial to even contemplate taking any disruptive actions, while others say that given the particular configuration of the world right now and the particular problems we face, it actually seems plausible that a person in such a position of influence ought to seriously consider disruptive actions.
I really agree with the latter, and I also contend that it’s the more epistemically humble position, because you’re not saying that it’s for sure that a pivotal act should be performed, but just that it’s quite plausible given the specifics of the current world situation. The other side of the argument seems to be saying that no no no it’s definitely better not to do anything like that in anything like the current world situation.
The thing I’d say in favor of this position is that I think it better fits the evidence. I think the problem with the opposing view is that it’s wrong, not that it’s more confident. E.g., if I learned that Nate assigns probability .9 to “a pivotal act is necessary” (for some operationalization of “necessary”) while Critch assigns probability .2 to “a pivotal act is necessary”, I wouldn’t go “ah, Critch is being more reasonable, since his probability is closer to .5″.
I agree with the rest of what you said, and I think this is a good way of framing the issue.
I’d add that I think discussion of this topic gets somewhat distorted by the fact that many people naturally track social consensus, and try to say the words they think will have the best influence on this consensus, rather than blurting out their relevant beliefs.
Many people are looking for a signal that stuff like this is OK to say in polite society, or many others are staking out a position “the case for this makes sense intellectually but there’s no way it will ever attract enough support, so I’ll preemptively oppose it in order to make my other arguments more politically acceptable”. (The latter, unfortunately, being a strategy that can serve as a self-fulfilling prophecy.)