I do remember a bunch of content around that, yeah. And I would agree that terminal goals are arbitrary in the sense that they could be anything. But, for any given agent/organism/”thing that wants stuff”, there will be a fact-of-the-matter of what terminals goals got instantiated inside that thing.
There are also a few separate but related and possibly confusing facts;
The process of evolution will tend to produce organisms that have certain kinds of terminal goals instantiated inside them.
Empirically, humans happen to have a huge overlap in their terminal goals (including the terminal goal that other beings have their terminal goals satisfied).
If there are a bunch of roughly equally-capable agents around, then it maximizes your own utility (= terminal goals) to do a lot of game-theoretic cooperation with them.
I do remember a bunch of content around that, yeah. And I would agree that terminal goals are arbitrary in the sense that they could be anything.
I see. Thanks for clarifying.
But, for any given agent/organism/”thing that wants stuff”, there will be a fact-of-the-matter of what terminals goals got instantiated inside that thing.
Yeah I agree and think that those are important points.
I do remember a bunch of content around that, yeah. And I would agree that terminal goals are arbitrary in the sense that they could be anything. But, for any given agent/organism/”thing that wants stuff”, there will be a fact-of-the-matter of what terminals goals got instantiated inside that thing.
There are also a few separate but related and possibly confusing facts;
The process of evolution will tend to produce organisms that have certain kinds of terminal goals instantiated inside them.
Empirically, humans happen to have a huge overlap in their terminal goals (including the terminal goal that other beings have their terminal goals satisfied).
If there are a bunch of roughly equally-capable agents around, then it maximizes your own utility (= terminal goals) to do a lot of game-theoretic cooperation with them.
I see. Thanks for clarifying.
Yeah I agree and think that those are important points.