Actual answer: Because the entire field of experimental psychology that’s why.
This excerpt isn’t specific so it’s hard to respond, but I do think there’s a lot of garbage in experimental psychology (like every other field), and more specifically I believe that Eliezer has cited some papers in his old blog posts that are bad papers. (Also, even when experimental results are trustworthy, their interpretation can be wrong.) I have some general thoughts on the field of evolutionary psychology in Section 1 here.
Eliezer’s reasoning is surprisingly weak here. It doesn’t really interact with the strong mechanistic claims he’s making (“Motivated reasoning is definitely built-in, but it’s built-in in a way that very strongly bears the signature of ‘What would be the easiest way to build this out of these parts we handily had lying around already’”).
He just flatly states a lot of his beliefs as true:
the conventional explanation of which is that we have built-in cheater detectors. This is a case in point of how humans aren’t blank slates and there’s no reason to pretend we are.
Conventional explanations are often bogus, and in particular I expect this one to be bogus.
Here, Eliezer states his dubious-to-me stances as obviously True, without explaining how they actually distinguish between mechanistic hypotheses, or e.g. why he thinks he can get so many bits about human learning process hyperparameters from results like Wason (I thought it’s hard to go from superficial behavioral results to statements about messy internals? & inferring “hard-coding” is extremely hard even for obvious-seeming candidates).
Similarly, in the summer (consulting my notes + best recollections here), he claimed ~”Evolution was able to make the (internal physiological reward schedule) ↦ (learned human values) mapping predictable because it spent lots of generations selecting for alignability on caring about proximate real-world quantities like conspecifics or food” and I asked “why do you think evolution had to tailor the reward system specifically to make this possible? what evidence has located this hypothesis?” and he said “I read a neuroscience textbook when I was 11?”, and stared at me with raised eyebrows.
I just stared at him with a shocked face. I thought, surely we’re talking about different things. How could that data have been strong evidence for that hypothesis? I didn’t understand how could possibly neuroscience textbooks provide huge evidence for evolution having to select the reward->value mapping into its current properties.
I also wrote in my journal at the time:
EY said that we might be able to find a learning process with consistent inner / outer values relations if we spent huge amounts of compute to evolve such a learning process in silicon
Eliezer seems to attach some strange importance to the learning process being found by evolution, even though the learning initial conditions screen off evolution’s influence.
I still don’t understand that interaction. But I’ve had a few interactions like this with him, where he confidently states things, and then I ask him why he thinks that, and offers some unrelated-seeming evidence which doesn’t—AFAICT—actually discriminate between hypotheses.
Eliezer’s reasoning is surprisingly weak here. It doesn’t really interact with the strong mechanistic claims he’s making (“Motivated reasoning is definitely built-in, but it’s built-in in a way that very strongly bears the signature of ‘What would be the easiest way to build this out of these parts we handily had lying around already’”).
He just flatly states a lot of his beliefs as true:
Conventional explanations are often bogus, and in particular I expect this one to be bogus.
Here, Eliezer states his dubious-to-me stances as obviously True, without explaining how they actually distinguish between mechanistic hypotheses, or e.g. why he thinks he can get so many bits about human learning process hyperparameters from results like Wason (I thought it’s hard to go from superficial behavioral results to statements about messy internals? & inferring “hard-coding” is extremely hard even for obvious-seeming candidates).
Similarly, in the summer (consulting my notes + best recollections here), he claimed ~”Evolution was able to make the (internal physiological reward schedule) ↦ (learned human values) mapping predictable because it spent lots of generations selecting for alignability on caring about proximate real-world quantities like conspecifics or food” and I asked “why do you think evolution had to tailor the reward system specifically to make this possible? what evidence has located this hypothesis?” and he said “I read a neuroscience textbook when I was 11?”, and stared at me with raised eyebrows.
I just stared at him with a shocked face. I thought, surely we’re talking about different things. How could that data have been strong evidence for that hypothesis? I didn’t understand how could possibly neuroscience textbooks provide huge evidence for evolution having to select the reward->value mapping into its current properties.
I also wrote in my journal at the time:
Eliezer seems to attach some strange importance to the learning process being found by evolution, even though the learning initial conditions screen off evolution’s influence.
I still don’t understand that interaction. But I’ve had a few interactions like this with him, where he confidently states things, and then I ask him why he thinks that, and offers some unrelated-seeming evidence which doesn’t—AFAICT—actually discriminate between hypotheses.