It sounds like we draw the line differently on the continuum between persuasion and brain hacking. I’d like to hear more about why you think some or all parts of this are is hacking so I can calibrate my “I’m probably not a sociopath” prior.
Or maybe we are diverging on what things one can legitimately claim are the purposes of Petrov Day. If the purpose of an in-person celebration is “Provide a space for community members to get together, break bread, and contemplate, and the button’s raises the tension high enough that it feels like something is at stake but there’s no serious risk that the button will be pressed,” then I’m wrong, I’m being cruel, and some other forum is more appropriate for my Ruiner to raise zir concern. But I don’t get the sense that there’s a community consensus on this.
On the contrary, the fact that all attendees are supposed to leave in silence if the button is pressed suggests that the risk is meant to be real and to have consequences.
I’d like to hear more about why you think some or all parts of this are hacking so I can calibrate my “I’m probably not a sociopath” prior.
I don’t think that you doing this would be “brain hacking”. But your plan to press the button in order to make me be more cautious and paranoid works roughly like this: you would decide to press the button, so as to cause me to believe that the world is full of untrustworthy people, so that I make different decisions. Here’s my attempt to break down my complaint about this:
You are trying to manipulate my beliefs to change my actions, without checking if this will make my beliefs more accurate.
Your manipulation of my beliefs will cause them to be inaccurate: I will probably believe that the world is contains careless people, or people with a reckless disregard for the website staying up. But actually what’s going on is the world contains people who want me to be paranoid.
To the extent that I do figure out what’s going on and do have true beliefs, then you’re just choosing whether I can have accurate beliefs in the world where things are bad, vs having accurate beliefs in the world where things are good. But it’s better for things to be good than bad.
If I have wrong beliefs about the distribution of people’s trustworthiness (or the ways in which people are untrustworthy), I will actually make worse decisions about which things to prioritize in AI security. You seem to believe the converse, but I doubt you have good reasons to think that.
On the contrary, the fact that all attendees are supposed to leave in silence if the button is pressed suggests that the risk is meant to be real and to have consequences.
Yes. Pressing the button makes life worse for your companions, which is the basic reason that you shouldn’t do it.
I genuinely, sincerely appreciate that you took the time to make this all explicit, and I think you assumed a more-than-reasonable amount of good faith on my part given how lathered up I was and how hard it is to read tone on the Internet.
I think the space we are talking across is “without checking if this will make my beliefs more accurate.” Accuracy entails “what do I think is true” but also “how confident am I that it’s true”. Persuasion entails that plus “will this persuasion strategy actually make their beliefs more accurate”. In hindsight, I should have communicated why I thought what I proposed would make people’s beliefs about humanity more accurate.
However, the response to my comments made me less confident that the intervention would be effective at making those beliefs more accurate. Plus, given the context, you had little reason to assume that my truth+confidence calculation was well-calibrated.
There’s also the question of whether the expected value of button-pressing exceeds the expected life-worsening, and how confident a potential button-presser is in their answer and the magnitude of the exceeding. I do think that’s a fair challenge to your final thought.
It sounds like we draw the line differently on the continuum between persuasion and brain hacking. I’d like to hear more about why you think some or all parts of this are
ishacking so I can calibrate my “I’m probably not a sociopath” prior.Or maybe we are diverging on what things one can legitimately claim are the purposes of Petrov Day. If the purpose of an in-person celebration is “Provide a space for community members to get together, break bread, and contemplate, and the button’s raises the tension high enough that it feels like something is at stake but there’s no serious risk that the button will be pressed,” then I’m wrong, I’m being cruel, and some other forum is more appropriate for my Ruiner to raise zir concern. But I don’t get the sense that there’s a community consensus on this.
On the contrary, the fact that all attendees are supposed to leave in silence if the button is pressed suggests that the risk is meant to be real and to have consequences.
I don’t think that you doing this would be “brain hacking”. But your plan to press the button in order to make me be more cautious and paranoid works roughly like this: you would decide to press the button, so as to cause me to believe that the world is full of untrustworthy people, so that I make different decisions. Here’s my attempt to break down my complaint about this:
You are trying to manipulate my beliefs to change my actions, without checking if this will make my beliefs more accurate.
Your manipulation of my beliefs will cause them to be inaccurate: I will probably believe that the world is contains careless people, or people with a reckless disregard for the website staying up. But actually what’s going on is the world contains people who want me to be paranoid.
To the extent that I do figure out what’s going on and do have true beliefs, then you’re just choosing whether I can have accurate beliefs in the world where things are bad, vs having accurate beliefs in the world where things are good. But it’s better for things to be good than bad.
If I have wrong beliefs about the distribution of people’s trustworthiness (or the ways in which people are untrustworthy), I will actually make worse decisions about which things to prioritize in AI security. You seem to believe the converse, but I doubt you have good reasons to think that.
Yes. Pressing the button makes life worse for your companions, which is the basic reason that you shouldn’t do it.
I genuinely, sincerely appreciate that you took the time to make this all explicit, and I think you assumed a more-than-reasonable amount of good faith on my part given how lathered up I was and how hard it is to read tone on the Internet.
I think the space we are talking across is “without checking if this will make my beliefs more accurate.” Accuracy entails “what do I think is true” but also “how confident am I that it’s true”. Persuasion entails that plus “will this persuasion strategy actually make their beliefs more accurate”. In hindsight, I should have communicated why I thought what I proposed would make people’s beliefs about humanity more accurate.
However, the response to my comments made me less confident that the intervention would be effective at making those beliefs more accurate. Plus, given the context, you had little reason to assume that my truth+confidence calculation was well-calibrated.
There’s also the question of whether the expected value of button-pressing exceeds the expected life-worsening, and how confident a potential button-presser is in their answer and the magnitude of the exceeding. I do think that’s a fair challenge to your final thought.
Thanks again.