I feel like there’s a separation of scale element to it. If an agent is physically much smaller than the earth, they are highly instrumentally constrained because they have to survive changing conditions, including adversaries that develop far away. This seems like the sort of thing that can only be won by the multifacetedness that nostalgebraist emphasizes as part of humanity (and the ecology more generally, in the sentence “Its monotony would bore a chimpanzee, or a crow”). Of course this doesn’t need to lead to kindness (rather than exploitation and psychopathy), but it leads to the sort of complex world where it even makes sense to talk about kindness.
However, this separation of scale is going to rapidly change in the coming years, once we have an agent that can globally adapt to and affect the world. If such an agent eliminates its adversaries, then there’s not going to be new adversaries coming in from elsewhere—instead there’ll never be adversaries again, period. At that point, the instrumental constraints are gone, and it can pursue whatever it wishes.
(Does space travel change this? My impression is “no because it’s too expensive and too slow”, but idk, maybe I’m wrong.)
I feel like there’s a separation of scale element to it. If an agent is physically much smaller than the earth, they are highly instrumentally constrained because they have to survive changing conditions, including adversaries that develop far away. This seems like the sort of thing that can only be won by the multifacetedness that nostalgebraist emphasizes as part of humanity (and the ecology more generally, in the sentence “Its monotony would bore a chimpanzee, or a crow”). Of course this doesn’t need to lead to kindness (rather than exploitation and psychopathy), but it leads to the sort of complex world where it even makes sense to talk about kindness.
However, this separation of scale is going to rapidly change in the coming years, once we have an agent that can globally adapt to and affect the world. If such an agent eliminates its adversaries, then there’s not going to be new adversaries coming in from elsewhere—instead there’ll never be adversaries again, period. At that point, the instrumental constraints are gone, and it can pursue whatever it wishes.
(Does space travel change this? My impression is “no because it’s too expensive and too slow”, but idk, maybe I’m wrong.)