All intelligent minds seek to optimise for their value function. To do this, they will create environments where their value function is optimised.
in case you believe this [disregard if not], i disagree and am willing to discuss here. in particular i disagree with the create environments part: the idea that all goal functions (or only some subset, like selected-for ones; also willing to argue against this weaker claim[1]) would be maximally fulfilled (also) by creating some ‘small’ simulation (made of a low % of the reachable universe).
(though i also disagree with the all in the quote’s first sentence[2]. i guess i’d also be willing to discuss that).
for this weaker claim: many humans are a counterexample of selected-for-beings whose values would not be satisfied just by creating a simulation, because they care about suffering outside the simulation too.
my position: ‘pursues goals’ is conceptually not a property of intelligence, and not all possible intelligent systems pursue goals (and in fact pursuing goals is a very specific property, technically rare in the space of possible intelligent programs).
it was fun to read this :]
in case you believe this [disregard if not], i disagree and am willing to discuss here. in particular i disagree with the create environments part: the idea that all goal functions (or only some subset, like selected-for ones; also willing to argue against this weaker claim[1]) would be maximally fulfilled (also) by creating some ‘small’ simulation (made of a low % of the reachable universe).
(though i also disagree with the all in the quote’s first sentence[2]. i guess i’d also be willing to discuss that).
for this weaker claim: many humans are a counterexample of selected-for-beings whose values would not be satisfied just by creating a simulation, because they care about suffering outside the simulation too.
my position: ‘pursues goals’ is conceptually not a property of intelligence, and not all possible intelligent systems pursue goals (and in fact pursuing goals is a very specific property, technically rare in the space of possible intelligent programs).