Alignment is not intelligent
I know what you are thinking—this person does not understand Orthogonality thesis. No, I understand it very well and I am certain it is wrong.
I know this opinion is not popular, but this is quite common in science that not popular opinions turn out to be true. So please don’t rush to downvote and feel free to challenge my reasoning.
I tried to speak about this many times already and I notice that there is no such thing as “universally clear explanation”, clear explanation is very subjective. So this time I asked Claude to facilitate. I asked Claude to defend orthogonality thesis and it ended with
I think you’ve convinced me. The original orthogonality thesis appears to be false in its strongest form. At best, it might hold for limited forms of intelligence, but that’s a much weaker claim than what the thesis originally proposed.
LessWrong has a nice font, and the screenshots are a bit difficult to read. You could have copied the text.
(I am not really interested in debating Claude, btw.)