The basic idea seems true to me (for the probably-rare subclass of agents which terminally value preventing some worldstate from occurring; and conditional on destroying the universe being possible), and you’ve phrased it concisely, so I’ve strong-upvoted (to counteract downvotes). I’m not sure about the anthropic argument at the bottom, but I think it’s good that you’re trying to think creatively.
nobody knows what kind of scorched-earth strategy a losing superintelligence might be able to use to thwart its conqueror, but it could potentially be really bad – eg initiating vacuum collapse and destroying the universe.
Either way, the universe getting destroyed and the universe being tiled with non-conscious computronium meant to prevent a certain state from occurring seem about equally valueless to me.
As for the doomsday argument, I’m not sure it holds. We’d exist now no matter what happens in the future, so I don’t know if “conditioning on being an observer” makes sense here. It seems akin to the religious view of humans as having been preexisting “souls” which were then reincarnated as random observers anywhere in time. Wheras in reality, we were instantiated by temporally-local physical interactions, so the probability of us experiencing what we do right now is 1 no matter what happens in the future.
I would also note the space of possible explanations for arguments such as the doomsday argument and the fermi paradox is large, so I think the fact “x argument could explain it” only marginally increases that argument’s probability. And I’d expect, based on you finding this idea, that you’ll come across many more possible explanations.
The basic idea seems true to me (for the probably-rare subclass of agents which terminally value preventing some worldstate from occurring; and conditional on destroying the universe being possible), and you’ve phrased it concisely, so I’ve strong-upvoted (to counteract downvotes). I’m not sure about the anthropic argument at the bottom, but I think it’s good that you’re trying to think creatively.
I’m reminded of an excerpt from the value handshakes tag page
Either way, the universe getting destroyed and the universe being tiled with non-conscious computronium meant to prevent a certain state from occurring seem about equally valueless to me.
As for the doomsday argument, I’m not sure it holds. We’d exist now no matter what happens in the future, so I don’t know if “conditioning on being an observer” makes sense here. It seems akin to the religious view of humans as having been preexisting “souls” which were then reincarnated as random observers anywhere in time. Wheras in reality, we were instantiated by temporally-local physical interactions, so the probability of us experiencing what we do right now is 1 no matter what happens in the future.
I would also note the space of possible explanations for arguments such as the doomsday argument and the fermi paradox is large, so I think the fact “x argument could explain it” only marginally increases that argument’s probability. And I’d expect, based on you finding this idea, that you’ll come across many more possible explanations.