Agreed, but, quite separately from this, we can still clarify our own values by inquiring into the kind of counterfactuals that Nate gave, within the privacy of our own minds.
We can introspect without sharing the results of our introspection, but then the title for this post should not be “Humans aren’t fitness maximizers”. That’s a general claim about humans that implies that we are sharing the results of our introspections to come to a consensus. The IGF-optimization hypothesis predicts that we will all share that we are not fitness maximizers and that will be our consensus.
In any case, people are not perfect liars, so the IGF-optimization hypothesis also predicts that most people will answer NO in the privacy of their own minds. This isn’t as strong a prediction, it depends on your model of the effectiveness of lying vs the costs of self-deception. It also predicts that anyone who models themselves at having a high likelihood of getting a socially unacceptable result from introspection will choose not to do the introspection.
This isn’t specific to IGF-optimization. Saying and thinking socially acceptable things is instrumentally convergent, and any theory of human values that is reality-adjacent predicts that mostly everyone says and thinks socially acceptable things, and indeed that is what we mostly observe.
We can introspect without sharing the results of our introspection, but then the title for this post should not be “Humans aren’t fitness maximizers”. That’s a general claim about humans that implies that we are sharing the results of our introspections to come to a consensus.
That is an interesting point. We could, in principle, be exhibiting behavior perfectly consistent with IGF-optimization, and yet, if we were to look carefully, find out that what we really truly care about is something different.
In any case, people are not perfect liars, so the IGF-optimization hypothesis also predicts that most people will answer NO in the privacy of their own minds.
Indeed
Saying and thinking socially acceptable things is instrumentally convergent, and any theory of human values that is reality-adjacent predicts that mostly everyone says and thinks socially acceptable things, and indeed that is what we mostly observe.
Indeed, and yet even in the presence of powerful coercian that acts on both our outward behavior and our inward reflections, it is still possible to find the courage to look directly at our own true nature.
Agreed, but, quite separately from this, we can still clarify our own values by inquiring into the kind of counterfactuals that Nate gave, within the privacy of our own minds.
We can introspect without sharing the results of our introspection, but then the title for this post should not be “Humans aren’t fitness maximizers”. That’s a general claim about humans that implies that we are sharing the results of our introspections to come to a consensus. The IGF-optimization hypothesis predicts that we will all share that we are not fitness maximizers and that will be our consensus.
In any case, people are not perfect liars, so the IGF-optimization hypothesis also predicts that most people will answer NO in the privacy of their own minds. This isn’t as strong a prediction, it depends on your model of the effectiveness of lying vs the costs of self-deception. It also predicts that anyone who models themselves at having a high likelihood of getting a socially unacceptable result from introspection will choose not to do the introspection.
This isn’t specific to IGF-optimization. Saying and thinking socially acceptable things is instrumentally convergent, and any theory of human values that is reality-adjacent predicts that mostly everyone says and thinks socially acceptable things, and indeed that is what we mostly observe.
That is an interesting point. We could, in principle, be exhibiting behavior perfectly consistent with IGF-optimization, and yet, if we were to look carefully, find out that what we really truly care about is something different.
Indeed
Indeed, and yet even in the presence of powerful coercian that acts on both our outward behavior and our inward reflections, it is still possible to find the courage to look directly at our own true nature.