I think it’s misleading to describe this as finding that intelligence and coherence are actually inversely correlated. Rather, survey respondents’ ratings of intelligence and ratings of coherence were inversely correlated.
An AI could also be misaligned because it acts in ways that don’t pursue any consistent goal (incoherence).
It’s worth noting that this definition of incoherence seems inconsistent with VNM. Eg. A rock might satisfy the folk definition of “pursuing a consistent goal,” but fail to satisfy VNM due to lacking completeness (and by corollary due to not performing expected utility optimization over the outcome space).
I think it’s misleading to describe this as finding that intelligence and coherence are actually inversely correlated. Rather, survey respondents’ ratings of intelligence and ratings of coherence were inversely correlated.
Sure, and that’s why I said “coherence” instead of coherence
Epistemic status: clumsy
It’s worth noting that this definition of incoherence seems inconsistent with VNM. Eg. A rock might satisfy the folk definition of “pursuing a consistent goal,” but fail to satisfy VNM due to lacking completeness (and by corollary due to not performing expected utility optimization over the outcome space).