The opposite of rational is “wrong” or “ineffective”. A person can’t be wrong or ineffective about everything, that’s senseless. I think all the confusion has arisen from Annoyance claiming that terminal values must have some spark of rationality, but Eliezer explained that he might have meant they must be coherent. So if I may paraphrase your question (which interests me as well), the question is: how may terminal values be incoherent?
The opposite of rational is “wrong” or “ineffective”. A person can’t be wrong or ineffective about everything, that’s senseless. I think all the confusion has arisen from Annoyance claiming that terminal values must have some spark of rationality, but Eliezer explained that he might have meant they must be coherent. So if I may paraphrase your question (which interests me as well), the question is: how may terminal values be incoherent?