If I didn’t care about myself in particular, only the aesthetics of the universe, and I was in charge of all humans, and alien species were making decisions similar to me, then I would shut down humanity rather than running an AI with a 20% chance of turning the lightcone into boring dreck and an 80% chance of turning it into a flourishing civilization. (Not because I think the boring drek is 5x as bad as flourishing civilization is good—there’s extra caution because this is playing “cooperate” with aliens who are faced with similar choices.)
Since I do care about myself in particular, I will intuitively accept much worse odds of civilization vs. dreck. But I do respect arguments that maybe we should still play “cooperate” and not destroy the lightcone. I think my intuition just expects other humans not to cooperate, let alone aliens.
If I didn’t care about myself in particular, only the aesthetics of the universe, and I was in charge of all humans, and alien species were making decisions similar to me, then I would shut down humanity rather than running an AI with a 20% chance of turning the lightcone into boring dreck and an 80% chance of turning it into a flourishing civilization. (Not because I think the boring drek is 5x as bad as flourishing civilization is good—there’s extra caution because this is playing “cooperate” with aliens who are faced with similar choices.)
Since I do care about myself in particular, I will intuitively accept much worse odds of civilization vs. dreck. But I do respect arguments that maybe we should still play “cooperate” and not destroy the lightcone. I think my intuition just expects other humans not to cooperate, let alone aliens.