I have been thinking about interpretation convergence the past few days, since a friend (a medievalist) mentioned to me the other day that medieval scholastics underestimated the diversity of routes human cognition can take when thinking about a problem. They assumed that all reasonable people would converge, given enough time and resources, on the same truths and judgments.
He points out this is why early protestants felt comfortable advocating reading the Bible alone without either authorities to govern interpretations or an interpretive apparatus, such as a tradition or formal method. Turns out, Christians differed on interpretation.
Now what’s interesting about this, is that the assumption of interpretation convergence was held so widely for so long. This indicates to me that value convergence relies upon a shared cognitive processing styles. It might be possible then for two AIs to considers a utility function V, but process its significance differently, depending on how they process the rest of the global state.
Good post with good questions:
Side analogy concerning CEV:
I have been thinking about interpretation convergence the past few days, since a friend (a medievalist) mentioned to me the other day that medieval scholastics underestimated the diversity of routes human cognition can take when thinking about a problem. They assumed that all reasonable people would converge, given enough time and resources, on the same truths and judgments.
He points out this is why early protestants felt comfortable advocating reading the Bible alone without either authorities to govern interpretations or an interpretive apparatus, such as a tradition or formal method. Turns out, Christians differed on interpretation.
Now what’s interesting about this, is that the assumption of interpretation convergence was held so widely for so long. This indicates to me that value convergence relies upon a shared cognitive processing styles. It might be possible then for two AIs to considers a utility function V, but process its significance differently, depending on how they process the rest of the global state.