I’m not an expert on decision theory, but my understanding (of FDT) is that there is no reason for the AI to cooperate with the paperclip maximizer (cooperate how?) because there is no scenario in which the paperclip maximizer treats the friendly AI differently based on it cooperating in counter-factual worlds. For it to be a question at all, it would require that
1) the paperclip maximizer is not a paperclip maximizer but a different kind of unfriendly AI
2) this unfriendly AI is actually launched (but may be in an inferior position)
I think there could be situations where it should cooperate. As I understand it, updateless/functional may say yes, causal and evidental would say no.
“1) the paperclip maximizer is not a paperclip maximizer but a different kind of unfriendly AI”
Being a paperclip maximizer is about values, not about decision theory. You can want to maximize paperclips but still use some of acausal decison theory that will cooperate with decision makers that would cooperate with paperclippers, as in cousin_it’s response.
I’m not an expert on decision theory, but my understanding (of FDT) is that there is no reason for the AI to cooperate with the paperclip maximizer (cooperate how?) because there is no scenario in which the paperclip maximizer treats the friendly AI differently based on it cooperating in counter-factual worlds. For it to be a question at all, it would require that
1) the paperclip maximizer is not a paperclip maximizer but a different kind of unfriendly AI
2) this unfriendly AI is actually launched (but may be in an inferior position)
I think there could be situations where it should cooperate. As I understand it, updateless/functional may say yes, causal and evidental would say no.
“1) the paperclip maximizer is not a paperclip maximizer but a different kind of unfriendly AI”
Being a paperclip maximizer is about values, not about decision theory. You can want to maximize paperclips but still use some of acausal decison theory that will cooperate with decision makers that would cooperate with paperclippers, as in cousin_it’s response.
That seems true, thanks for the correction.