I think accepting or rejecting the moratorium has nothing to do with game theory at all. It’s purely a question of understanding.
Think of it this way. Imagine you’re pushing a bus full of children, including your own child, toward a precipice. And you’re paid for each step. Why on Earth would you say “oh no, I’ll keep pushing, because otherwise other people will get money and power instead of me”? It’s not like other people will profit by that money and power! If they keep pushing, their kids will die too, along with everyone else’s! The only thing that keeps you pushing the bus is your lack of understanding, not any game theory considerations. Anyone with a clear understanding should just stop pushing the frigging bus.
Every time you move the bus 1cm further forward you get paid $10000. The precipice isn’t actually visible, it’s behind a bank of fog; you think it’s probably real but don’t know for sure. There are 20 other people helping you push the bus, and they also get paid. All appearances suggest that most of the other bus-pushers believe there is no precipice. One person is enough to keep the bus moving; even if 20 people stop pushing and only one continues, if the precipice is real the bus still falls, just a bit later. It’s probably possible to pretend you’ve stopped pushing while in fact continuing to push, without everyone else knowing you’re doing that.
I’m not sure whether this is exactly “game theory”, but it’s certainly a situation where you need to take into account the other people, and it’s certainly not as simple as “you have to stop pushing the bus because otherwise the children will die” since plausibly those children are going to die anyway.
It’s a simplification certainly. But the metaphor kinda holds up—if you know the precipice is real, the right thing is still to stop and try to explain to others that the precipice is real, maybe using your stopping as a costly signal. Right now the big players can send such a signal, if top researchers say they’ve paused working, no new products are released publicly and so on. And maybe if enough players get on board with this, they can drag the rest along by social pressure, divestment or legislation. The important thing is to start, I just made a post about this.
I think accepting or rejecting the moratorium has nothing to do with game theory at all. It’s purely a question of understanding.
Think of it this way. Imagine you’re pushing a bus full of children, including your own child, toward a precipice. And you’re paid for each step. Why on Earth would you say “oh no, I’ll keep pushing, because otherwise other people will get money and power instead of me”? It’s not like other people will profit by that money and power! If they keep pushing, their kids will die too, along with everyone else’s! The only thing that keeps you pushing the bus is your lack of understanding, not any game theory considerations. Anyone with a clear understanding should just stop pushing the frigging bus.
Every time you move the bus 1cm further forward you get paid $10000. The precipice isn’t actually visible, it’s behind a bank of fog; you think it’s probably real but don’t know for sure. There are 20 other people helping you push the bus, and they also get paid. All appearances suggest that most of the other bus-pushers believe there is no precipice. One person is enough to keep the bus moving; even if 20 people stop pushing and only one continues, if the precipice is real the bus still falls, just a bit later. It’s probably possible to pretend you’ve stopped pushing while in fact continuing to push, without everyone else knowing you’re doing that.
I’m not sure whether this is exactly “game theory”, but it’s certainly a situation where you need to take into account the other people, and it’s certainly not as simple as “you have to stop pushing the bus because otherwise the children will die” since plausibly those children are going to die anyway.
Those of them who believe it based on vibes may switch beliefs when a moratorium goes into effect.
Yes, I think one of the biggest benefits of stopping pushing the bus is in fact the signalling effect.
It’s a simplification certainly. But the metaphor kinda holds up—if you know the precipice is real, the right thing is still to stop and try to explain to others that the precipice is real, maybe using your stopping as a costly signal. Right now the big players can send such a signal, if top researchers say they’ve paused working, no new products are released publicly and so on. And maybe if enough players get on board with this, they can drag the rest along by social pressure, divestment or legislation. The important thing is to start, I just made a post about this.