Other ASI may be aligned to their own progenitor civilizations and thus base their relation to our ASI on idea is it “aligned” or not. They may even go on war or acualsally threat to do it if other ASI is bad to their original civilization.
I feel like your previous comment argues against that, rather than for it. You said that people who are trapped together should be nice to each other because the cost of a conflict is very high. But now you’re suggesting that ASIs that are metaphorically trapped together would aggressively attack each other to enforce compliance with their own behavioral standards. These two conjectures do not really seem allied to me.
Separately, I am very skeptical of aliens warring against ASIs to acausally protect us. I see multiple points where this seems likely to fail:
Would aliens actually take our side against an ASI merely because we created it? If humans hear a story about an alien civilization creating a successor species, and then the successor species overthrowing its creators, I do not expect humans to automatically be on the creators’ side in this story. I expect humans will take a side mostly based on how the two species were treating each other (overthrowing abusive masters is usually portrayed as virtuous in our fiction), and that which one of them is the creator will have little weight. I do not think “everyone should be aligned with their creators” is a principle that humans would actually endorse (except by motivated reasoning, in situations where it benefits us).
Also note that humans are not aligned with the process that produced us (evolution) and approximately no humans think this is a problem
Even if the aliens sympathize with us, would they care enough to take expensive actions about it?
Even if the aliens would war to save us, would the ASI predict that? It can only acausally save us if the ASI successfully predicts the policy. Otherwise, the war might still happen, but that doesn’t help us.
Even if the ASI predicts this, will it comply? This seems like what dath ilan would consider a “threat”, in that the aliens are punishing the ASI rather than enacting their own BATNA. It may be decision-theoretically correct to ignore the threat.
This whole premise, of us being saved at the eleventh hour by off-stage actors, seems intuitively like the sort of hypothesis that would be more likely to be produced by wishful thinking than by sober analysis, which would make me distrust it even if I couldn’t see any specific problems with it.
I am argue against relation of necessity which EY suggested, but do not argue against the idea that “ASI will kill us all” is highly likely. So there is no actionable difference. My my arguments need to be only to demonstrate possibility but not necessity to undermine the necessity of the outcome that ASI will kill us.
Also metaphors has limited proving power as they often miss some details. Both true for Arnault and jail. In jail, there is a high authority which prevents most inmates to kill each other until only one survive. However, even in some jails people who committed special types of crimes are prosecuted by other inmates. For example, in female jails, women who killed minors may be killed by other women.
So it is better not to go in jail if your perform a type of crime which other inmates hate. The same way may not want to go in space full of other ASIs if it performed the crime which other ASIs may hate: killing its creators.
Other your arguments are about possibilities, and they can be converted in expected utilities for ASI. It may ask itself: do I want to take 0.1 per cent increase of the risk of space war in the future for 0.000000000001 increase of amount of available atoms? If it starts reason this way, it is likely that it will prefer to reduce the risk of space war.
We can increase our chances by making precommitment now (Rolf Nelson attack for space wars) to go to war with future non-aligned AI in space. Nelson suggested similar precomitment attack for simulation: we precommit now to simulate all possible hostile AIs and turn them off, if they will not play as if they are aligned; as any young ASI may not be sure if it simulation of not, it may prefer to play aligned.
Will ASI play 4D chess and ignore the acasual threat just to punish everyone who do it? I am not sure. There could be 5D chess level where ASI will find benefitial to comply only to [don’t know what].
Anyway predicting ASI behavior is difficult and any claims about it can’t be certain. So we can’t be certain that ASI will kill us.
I don’t see why either expecting or not-expecting to meet other ASIs would make it instrumental to be nice to humans.
Other ASI may be aligned to their own progenitor civilizations and thus base their relation to our ASI on idea is it “aligned” or not. They may even go on war or acualsally threat to do it if other ASI is bad to their original civilization.
I feel like your previous comment argues against that, rather than for it. You said that people who are trapped together should be nice to each other because the cost of a conflict is very high. But now you’re suggesting that ASIs that are metaphorically trapped together would aggressively attack each other to enforce compliance with their own behavioral standards. These two conjectures do not really seem allied to me.
Separately, I am very skeptical of aliens warring against ASIs to acausally protect us. I see multiple points where this seems likely to fail:
Would aliens actually take our side against an ASI merely because we created it? If humans hear a story about an alien civilization creating a successor species, and then the successor species overthrowing its creators, I do not expect humans to automatically be on the creators’ side in this story. I expect humans will take a side mostly based on how the two species were treating each other (overthrowing abusive masters is usually portrayed as virtuous in our fiction), and that which one of them is the creator will have little weight. I do not think “everyone should be aligned with their creators” is a principle that humans would actually endorse (except by motivated reasoning, in situations where it benefits us).
Also note that humans are not aligned with the process that produced us (evolution) and approximately no humans think this is a problem
Even if the aliens sympathize with us, would they care enough to take expensive actions about it?
Even if the aliens would war to save us, would the ASI predict that? It can only acausally save us if the ASI successfully predicts the policy. Otherwise, the war might still happen, but that doesn’t help us.
Even if the ASI predicts this, will it comply? This seems like what dath ilan would consider a “threat”, in that the aliens are punishing the ASI rather than enacting their own BATNA. It may be decision-theoretically correct to ignore the threat.
This whole premise, of us being saved at the eleventh hour by off-stage actors, seems intuitively like the sort of hypothesis that would be more likely to be produced by wishful thinking than by sober analysis, which would make me distrust it even if I couldn’t see any specific problems with it.
I am argue against relation of necessity which EY suggested, but do not argue against the idea that “ASI will kill us all” is highly likely. So there is no actionable difference. My my arguments need to be only to demonstrate possibility but not necessity to undermine the necessity of the outcome that ASI will kill us.
Also metaphors has limited proving power as they often miss some details. Both true for Arnault and jail. In jail, there is a high authority which prevents most inmates to kill each other until only one survive. However, even in some jails people who committed special types of crimes are prosecuted by other inmates. For example, in female jails, women who killed minors may be killed by other women.
So it is better not to go in jail if your perform a type of crime which other inmates hate. The same way may not want to go in space full of other ASIs if it performed the crime which other ASIs may hate: killing its creators.
Other your arguments are about possibilities, and they can be converted in expected utilities for ASI. It may ask itself: do I want to take 0.1 per cent increase of the risk of space war in the future for 0.000000000001 increase of amount of available atoms? If it starts reason this way, it is likely that it will prefer to reduce the risk of space war.
We can increase our chances by making precommitment now (Rolf Nelson attack for space wars) to go to war with future non-aligned AI in space. Nelson suggested similar precomitment attack for simulation: we precommit now to simulate all possible hostile AIs and turn them off, if they will not play as if they are aligned; as any young ASI may not be sure if it simulation of not, it may prefer to play aligned.
Will ASI play 4D chess and ignore the acasual threat just to punish everyone who do it? I am not sure. There could be 5D chess level where ASI will find benefitial to comply only to [don’t know what].
Anyway predicting ASI behavior is difficult and any claims about it can’t be certain. So we can’t be certain that ASI will kill us.