Say you disagree with someone, e.g. they have low pdoom and you have high pdoom. You might be interested in finding cruxes with them.
You can keep imagining narrower and narrower scenarios in which your beliefs still diverge. Then you can back out properties of the final scenario to identify cruxes.
For example, you start by conditioning on AGI being achieved—both of your pdooms tick up a bit. Then you also condition on that AGI being misaligned, and again your pdooms increase a bit (if the beliefs move in opposite directions, that might be worth exploring!). Then you condition on the AGI self-exfiltrating, and your pdooms nudge up again.
Now you’ve found a very narrow scenario in which you still disagree! You think it’s obvious that a misaligned AGI proliferating around the world is an endgame, they don’t see what the big deal is. From there, you’re in a good position to find cruxes.
(Note that you’re not necessarily finding the condition of maximum disagreement, you’re just trying to get information about where you disagree.)
Conditioning as a Crux Finding Device
Say you disagree with someone, e.g. they have low pdoom and you have high pdoom. You might be interested in finding cruxes with them.
You can keep imagining narrower and narrower scenarios in which your beliefs still diverge. Then you can back out properties of the final scenario to identify cruxes.
For example, you start by conditioning on AGI being achieved—both of your pdooms tick up a bit. Then you also condition on that AGI being misaligned, and again your pdooms increase a bit (if the beliefs move in opposite directions, that might be worth exploring!). Then you condition on the AGI self-exfiltrating, and your pdooms nudge up again.
Now you’ve found a very narrow scenario in which you still disagree! You think it’s obvious that a misaligned AGI proliferating around the world is an endgame, they don’t see what the big deal is. From there, you’re in a good position to find cruxes.
(Note that you’re not necessarily finding the condition of maximum disagreement, you’re just trying to get information about where you disagree.)