while this appears to me to be true, a species of cooperator-behavior mechanisms A that same-mechanism-cooperate A-with-A in order to defect against not-same-mechanism behaviors B can be catastrophically dangerous to the other species B, even if the other species B are themselves self-cooperators. if one subnetwork of self-cooperators does not participate in the eventual broad interspecies cooperation group, that subnetwork can potentially produce non-cooperative outcomes on the outer scale. it seems to me that at a given scale, agency that is local on that scale does tend to try non-cooperative routes first, and that this initially limits its success; see, for example, how in evo game theory simulations, semi-cooperators like tit-for-tat-with-forgiveness and such usually win eventually and produce a cooperative society (depending on the setup), but defector-against-outer-network subnetworks can have a significant foothold for a long time.
because of this, it is my current view that eventually, some unspecified species of durable semicooperators are likely to win nearly the entire universe. however, on the path to get there, species of non-cooperator behaviors may cause significant damage to other cooperators. This despite that a fully cooperative network appears to me to be eventually near-guaranteed, for efficiency reasons.
so if one network of self-cooperators A value their own defection against other self-cooperators B where [A, B] could be [one military, humanity], or [humanity, cows and chickens], or [integrated multi-component ASI, humanity], then the defense analysis necessary to produce a game tree that visibly pays out to semicooperators when they cooperate is not necessarily trivial.
That said, because I strongly agree that long term, any agent wants to become a universally pro-tolerance semicooperator and constrain its aesthetic-structure values to apply to only a finite amount of the universe’s negentropy, I think we have the potential to teach agentic AIs from the start that bridging non-cooperative circumstances is a worthy and useful goal which results in durability for the agent’s aesthetic intentions because of durable coprotection.
(coprotection is a word I’ve used for a while to refer to agentic cooperation, ie an agent that tries to find ways to produce durability for other agents’ values, not just their own. there are probably terms of art that I should be using, but coprotection seems to mean the right thing semantically without further explanation in nontechnical conversation.)
Universal cooperation on all system levels means total optimisation in the universe as a neural network, and indeed this can be a “goal” (unattainable, though), but the maximum gradient according to this loss function doesn’t necessarily mean removing optimisation frustrations with particular subsystems (humans and the society) first, or at all. Especially if the AI takes panpsychism seriously.
constrain its aesthetic-structure values to apply to only a finite amount of the universe’s negentropy
I don’t understand this phrase. (Neg)entropy) is a numeric property of a physical system (including the whole universe), that is, a number. What does it mean to apply something to a “limited amount” of it?
I mean that we can assign a particular block of matter, as priced by the amount of negentropy it contains, to a computation trajectory (eg, a person, or an ai). that is, we would fuel the ai with that amount of unspent energy.
can you clarify what you mean by the comparison to the universe as a neural network? I’m having trouble understanding the paper due to insufficient physics background, but it seems like it’s not drawing a very coherent connection. I do think there’s a connection to be drawn, but I’m extremely suspicious about whether this is the correct one.
while this appears to me to be true, a species of cooperator-behavior mechanisms A that same-mechanism-cooperate A-with-A in order to defect against not-same-mechanism behaviors B can be catastrophically dangerous to the other species B, even if the other species B are themselves self-cooperators. if one subnetwork of self-cooperators does not participate in the eventual broad interspecies cooperation group, that subnetwork can potentially produce non-cooperative outcomes on the outer scale. it seems to me that at a given scale, agency that is local on that scale does tend to try non-cooperative routes first, and that this initially limits its success; see, for example, how in evo game theory simulations, semi-cooperators like tit-for-tat-with-forgiveness and such usually win eventually and produce a cooperative society (depending on the setup), but defector-against-outer-network subnetworks can have a significant foothold for a long time.
because of this, it is my current view that eventually, some unspecified species of durable semicooperators are likely to win nearly the entire universe. however, on the path to get there, species of non-cooperator behaviors may cause significant damage to other cooperators. This despite that a fully cooperative network appears to me to be eventually near-guaranteed, for efficiency reasons.
so if one network of self-cooperators A value their own defection against other self-cooperators B where [A, B] could be [one military, humanity], or [humanity, cows and chickens], or [integrated multi-component ASI, humanity], then the defense analysis necessary to produce a game tree that visibly pays out to semicooperators when they cooperate is not necessarily trivial.
That said, because I strongly agree that long term, any agent wants to become a universally pro-tolerance semicooperator and constrain its aesthetic-structure values to apply to only a finite amount of the universe’s negentropy, I think we have the potential to teach agentic AIs from the start that bridging non-cooperative circumstances is a worthy and useful goal which results in durability for the agent’s aesthetic intentions because of durable coprotection.
(coprotection is a word I’ve used for a while to refer to agentic cooperation, ie an agent that tries to find ways to produce durability for other agents’ values, not just their own. there are probably terms of art that I should be using, but coprotection seems to mean the right thing semantically without further explanation in nontechnical conversation.)
Universal cooperation on all system levels means total optimisation in the universe as a neural network, and indeed this can be a “goal” (unattainable, though), but the maximum gradient according to this loss function doesn’t necessarily mean removing optimisation frustrations with particular subsystems (humans and the society) first, or at all. Especially if the AI takes panpsychism seriously.
I don’t understand this phrase. (Neg)entropy) is a numeric property of a physical system (including the whole universe), that is, a number. What does it mean to apply something to a “limited amount” of it?
I mean that we can assign a particular block of matter, as priced by the amount of negentropy it contains, to a computation trajectory (eg, a person, or an ai). that is, we would fuel the ai with that amount of unspent energy.
can you clarify what you mean by the comparison to the universe as a neural network? I’m having trouble understanding the paper due to insufficient physics background, but it seems like it’s not drawing a very coherent connection. I do think there’s a connection to be drawn, but I’m extremely suspicious about whether this is the correct one.