Yeah why not? Once when I asked if the SIAI would consider the possibility of paying AGI researchers not to do AGI research, or kill an AGI researcher who is just days away from launching an uFAI, Yudkowsky said something along the lines that it is OK to just downvote me to −10 rather −10000. Talk about taking ideas seriously?
Never mind the above, I can’t tell you why it would be wrong but I have a feeling that it is. It would lead to all kinds of bad behavior based on probabilities and expected utility calculations. I don’t feel like taking that route right now...
...if you gave me a button and convinced me that pressing it would save many people from suffering a century from now...
Can I conclude that you would give in to a Pascal’s Mugging scenario? If not, where do you draw the line and why? If an important part of your calculation, the part that sets the upper and lower bounds, is necessarily based on ‘instinct’ then why don’t you disregard those calculations completely and do what you feel is right and don’t harm anyone?
To answer your questions: No, I don’t think you can fairly conclude that I’m subject to Pascal’s Mugging, and I draw the line based on what calculations I can do and what calculations I can’t do.
That is, my inability to come up with reliable estimates of the probability that Pascal’s Mugger really can (and will) kill 3^^^3 people is not a good reason for me to disregard my ability to come up with reliable estimates of the probability that dropping poison in a well will kill people; I can reasonably refuse to do the latter (regardless of what I feel) on the grounds that I don’t choose to kill people, regardless of what I say or don’t say about Pascal’s Mugging.
And if there’s a connection between any of this and the initial question I asked, I don’t see it.
Yeah why not? Once when I asked if the SIAI would consider the possibility of paying AGI researchers not to do AGI research, or kill an AGI researcher who is just days away from launching an uFAI, Yudkowsky said something along the lines that it is OK to just downvote me to −10 rather −10000. Talk about taking ideas seriously?
Never mind the above, I can’t tell you why it would be wrong but I have a feeling that it is. It would lead to all kinds of bad behavior based on probabilities and expected utility calculations. I don’t feel like taking that route right now...
Can I conclude that you would give in to a Pascal’s Mugging scenario? If not, where do you draw the line and why? If an important part of your calculation, the part that sets the upper and lower bounds, is necessarily based on ‘instinct’ then why don’t you disregard those calculations completely and do what you feel is right and don’t harm anyone?
To answer your questions: No, I don’t think you can fairly conclude that I’m subject to Pascal’s Mugging, and I draw the line based on what calculations I can do and what calculations I can’t do.
That is, my inability to come up with reliable estimates of the probability that Pascal’s Mugger really can (and will) kill 3^^^3 people is not a good reason for me to disregard my ability to come up with reliable estimates of the probability that dropping poison in a well will kill people; I can reasonably refuse to do the latter (regardless of what I feel) on the grounds that I don’t choose to kill people, regardless of what I say or don’t say about Pascal’s Mugging.
And if there’s a connection between any of this and the initial question I asked, I don’t see it.