I’m the one who picked the particular virtues we displayed, and it seemed good to explain a bit more about where I was coming from. We’ve had a lot of interesting discussion each year about the True Meaning of Petrov Day.
Today is September 26th, Petrov Day, celebrated to honor the deed of Stanislav Yevgrafovich Petrov on September 26th, 1983.
Wherever you are, whatever you’re doing, take a minute to not destroy the world.
The theme of “Reflect on the fact that the world could have been destroyed, and be the sort of people who don’t destroy the world” was a central element most Petrov Day celebrations for years going forward. In 2019, when the LW team first built The Button experiment for LessWrong (where some users could press a button that took down the LessWrong frontpage for a day), we leaned heavily into this framing.
But throughout the years, some people have objected to this being the primary (or at least only) framing of the event. We’ve had a lot of interesting discussion each year about the True Meaning of Petrov Day.
Some people argued that it’s actually kind of confused to celebrate Petrov for the sake of “not taking dangerous unilateral actions”, when in fact it’s quite important that Petrov did act unilaterally in some sense – he violated the rules he had agreed to in favor of his own judgment. He was non-unilateralist with respect to the World At Large, but quite unilateralist with respect to his role in the Soviet military.
Some people argued that one of the most important aspects of Petrov is that he resisted social pressure, and did what he thought was right. In this frame, there was something ironic about everyone dutifully not-clicking-a-button as part of a tradition, especially if they were doing so mostly/entirely because they expected to be socially shamed if they clicked it.
Some people argued about whether it made sense to think of the thing more as a sacred ritual or as a fun game. Some people countered with “Look man, it doesn’t matter whether it’s a ritual or a game. The important thing is to notice that your action is going to destroy a bunch of value for no reason, and decide not to do that, regardless of what social role you think you’re supposed to be playing.”
In the last couple years, I’ve personally come to believe that the skill of orienting is quite important – noticing when you’re in a new situation (or a different situation that you thought you were in), and you need to reprocess your background information in light of that information. Decide what your actual goals are, and then figure out what to do in light of those goals. This feels like one of the important things that Petrov did to me.
Finally, just last week, Jimrandomh pointed out to me that one of the noteworthy things about Petrov is that, while he didn’t relay-the-information according to what the machine said and what the rules dictated… he also was accurately representing his epistemic state to his superiors. Petrov had helped build the missile-detection-system, and he had an idea of what it’s limitations were. The indicator was weird (I haven’t looked up the details right now, but my recollection is that it was showing only a small number of incoming missiles, which wasn’t what you’d actually expect if the US was attacking).
We don’t know what would have happened if it had looked to Petrov like a full scale attack was inbound. But in our current timeline, when he reported “false alarm”, it’s noteworthy that that was his actual epistemic state.
...
I’m sure there are many more frames to look at Petrov Day through. The Petrov Incident was not a movie, written with a theme in mind to communicate a particular message. It was a real thing, with lots of messy details. We can choose how to look at it and think about it, and take what lessons we want.
I was interested, this year, in leaning somewhat into the fact that Petrov actually had to decide what was right. I wanted LessWrong’s Petrov Day commemoration to present users with something more ambiguous that they had to reason about themselves.
(Also, separately, the Lightcone team was very busy last week and we scrambled to put Petrov Day together at the last minute. I’m fairly happy with what we came up with at the last minute but, like, don’t wanna oversell it as particularly well thought out)
A couple people have said “man none of these virtues carve up virtue-space the way I care about.” I’ve heard “‘not taking actions that might destroy the world’ isn’t a virtue even though it’s the most imprtant thing”, and “the most important virtue was ‘having agency in the face of systemic pressure to follow orders’”, which the person conceptualized as sort of a mix of the “Orient quickly” and “Resist Social Pressure” virtue. (or rather, I said “well, I think I basically split that into these two other virtues, so, uh, you gotta pick one?”, which they did grudgingly).
Also, after shipping the initial code and initial wave of notifications, it took us a while to realize I had mispelled “Petrov” as “Petroy”. We thought about trying to abort the notification volley, but decided “well, figuring out whether the system is malfunctioning is just a kinda reasonable part of the game.”
I’m the one who picked the particular virtues we displayed, and it seemed good to explain a bit more about where I was coming from. We’ve had a lot of interesting discussion each year about the True Meaning of Petrov Day.
Eliezer’s original post on Petrov Day begins:
The theme of “Reflect on the fact that the world could have been destroyed, and be the sort of people who don’t destroy the world” was a central element most Petrov Day celebrations for years going forward. In 2019, when the LW team first built The Button experiment for LessWrong (where some users could press a button that took down the LessWrong frontpage for a day), we leaned heavily into this framing.
But throughout the years, some people have objected to this being the primary (or at least only) framing of the event. We’ve had a lot of interesting discussion each year about the True Meaning of Petrov Day.
Some people argued that it’s actually kind of confused to celebrate Petrov for the sake of “not taking dangerous unilateral actions”, when in fact it’s quite important that Petrov did act unilaterally in some sense – he violated the rules he had agreed to in favor of his own judgment. He was non-unilateralist with respect to the World At Large, but quite unilateralist with respect to his role in the Soviet military.
Some people argued that one of the most important aspects of Petrov is that he resisted social pressure, and did what he thought was right. In this frame, there was something ironic about everyone dutifully not-clicking-a-button as part of a tradition, especially if they were doing so mostly/entirely because they expected to be socially shamed if they clicked it.
Some people argued about whether it made sense to think of the thing more as a sacred ritual or as a fun game. Some people countered with “Look man, it doesn’t matter whether it’s a ritual or a game. The important thing is to notice that your action is going to destroy a bunch of value for no reason, and decide not to do that, regardless of what social role you think you’re supposed to be playing.”
In the last couple years, I’ve personally come to believe that the skill of orienting is quite important – noticing when you’re in a new situation (or a different situation that you thought you were in), and you need to reprocess your background information in light of that information. Decide what your actual goals are, and then figure out what to do in light of those goals. This feels like one of the important things that Petrov did to me.
Finally, just last week, Jimrandomh pointed out to me that one of the noteworthy things about Petrov is that, while he didn’t relay-the-information according to what the machine said and what the rules dictated… he also was accurately representing his epistemic state to his superiors. Petrov had helped build the missile-detection-system, and he had an idea of what it’s limitations were. The indicator was weird (I haven’t looked up the details right now, but my recollection is that it was showing only a small number of incoming missiles, which wasn’t what you’d actually expect if the US was attacking).
We don’t know what would have happened if it had looked to Petrov like a full scale attack was inbound. But in our current timeline, when he reported “false alarm”, it’s noteworthy that that was his actual epistemic state.
...
I’m sure there are many more frames to look at Petrov Day through. The Petrov Incident was not a movie, written with a theme in mind to communicate a particular message. It was a real thing, with lots of messy details. We can choose how to look at it and think about it, and take what lessons we want.
I was interested, this year, in leaning somewhat into the fact that Petrov actually had to decide what was right. I wanted LessWrong’s Petrov Day commemoration to present users with something more ambiguous that they had to reason about themselves.
(Also, separately, the Lightcone team was very busy last week and we scrambled to put Petrov Day together at the last minute. I’m fairly happy with what we came up with at the last minute but, like, don’t wanna oversell it as particularly well thought out)
A couple people have said “man none of these virtues carve up virtue-space the way I care about.” I’ve heard “‘not taking actions that might destroy the world’ isn’t a virtue even though it’s the most imprtant thing”, and “the most important virtue was ‘having agency in the face of systemic pressure to follow orders’”, which the person conceptualized as sort of a mix of the “Orient quickly” and “Resist Social Pressure” virtue. (or rather, I said “well, I think I basically split that into these two other virtues, so, uh, you gotta pick one?”, which they did grudgingly).
Also, after shipping the initial code and initial wave of notifications, it took us a while to realize I had mispelled “Petrov” as “Petroy”. We thought about trying to abort the notification volley, but decided “well, figuring out whether the system is malfunctioning is just a kinda reasonable part of the game.”