The framing effect & aversion to losses generally cause us to execute more cautious plans. I’m realizing this is another reason to reframe my x-risk motivation from “I won’t let the world be destroyed” to “there’s so much fun we could have, and I want to make sure that happens”. I think we need more exploratory thinking in alignment research right now.
(Also, the former motivation style led to me crashing and burning a bit when my hands were injured and I was no longer able to do much.)
ETA: actually, i’m realizing I had the effect backwards. Framing via losses actually encourages more risk-taking plans. Oops. I’d like to think about this more, since I notice my model didn’t protest when I argued the opposite of the experimental conclusions.
I’m realizing how much more risk-neutral I should be:
Paul Samuelson… offered a colleague a coin-toss gamble. If the colleague won the coin toss, he would receive $200, but if he lost, he would lose $100. Samuelson was offering his colleague a positive expected value with risk. The colleague, being risk-averse, refused the single bet, but said that he would be happy to toss the coin 100 times! The colleague understood that the bet had a positive expected value and that across lots of bets, the odds virtually guaranteed a profit. Yet with only one trial, he had a 50% chance of regretting taking the bet.
Notably, Samuelson‘s colleague doubtless faced many gambles in life… He would have fared better in the long run by maximizing his expected value on each decision… all of us encounter such “small gambles” in life, and we should try to follow the same strategy. Risk aversion is likely to tempt us to turn down each individual opportunity for gain. Yet the aggregated risk of all of the positive expected value gambles that we come across would eventually become infinitesimal, and potential profit quite large.
For what it’s worth, I tried something like the “I won’t let the world be destroyed”->”I want to make sure the world keeps doing awesome stuff” reframing back in the day and it broadly didn’t work. This had less to do with cautious/uncautious behavior and more to do with status quo bias. Saying “I won’t let the world be destroyed” treats “the world being destroyed” as an event that deviates from the status quo of the world existing. In contrast, saying “There’s so much fun we could have” treats “having more fun” as the event that deviates from the status quo of us not continuing to have fun.
When I saw the world being destroyed as status quo, I cared a lot less about the world getting destroyed.
The framing effect & aversion to losses generally cause us to execute more cautious plans. I’m realizing this is another reason to reframe my x-risk motivation from “I won’t let the world be destroyed” to “there’s so much fun we could have, and I want to make sure that happens”. I think we need more exploratory thinking in alignment research right now.
(Also, the former motivation style led to me crashing and burning a bit when my hands were injured and I was no longer able to do much.)
ETA: actually, i’m realizing I had the effect backwards. Framing via losses actually encourages more risk-taking plans. Oops. I’d like to think about this more, since I notice my model didn’t protest when I argued the opposite of the experimental conclusions.
I’m realizing how much more risk-neutral I should be:
For what it’s worth, I tried something like the “I won’t let the world be destroyed”->”I want to make sure the world keeps doing awesome stuff” reframing back in the day and it broadly didn’t work. This had less to do with cautious/uncautious behavior and more to do with status quo bias. Saying “I won’t let the world be destroyed” treats “the world being destroyed” as an event that deviates from the status quo of the world existing. In contrast, saying “There’s so much fun we could have” treats “having more fun” as the event that deviates from the status quo of us not continuing to have fun.
When I saw the world being destroyed as status quo, I cared a lot less about the world getting destroyed.