If before you open the book, you believe that the book will provide incredibly compelling evidence of Zoroastrianism whether or not Zoroastrianism is true, and upon opening the book you find incredibly compelling evidence of Zoroastrianism, your probability of Zoroastrianism should not change, since you didn’t observe any evidence which is more likely to exist if Zoroastrianism were true than if it were not true.
This presumes that your mind can continue to obey the rules of Bayesian updating in the face of an optimization process that’s deliberately trying to make it break those rules. We can’t do that very well.
OP argued that self-deception occurs even if your brain remains unbroken. I would characterize “not breaking my brain” as allowing my prior belief about the book’s biasedness to make a difference in my posterior confidence of the book’s thesis. In that case the book might be arbitrarily convincing; but I might start with an arbitrarily high confidence that the book is biased, and then it boils down to an ordinary Bayesian tug o’ war, and Yvain’s comment applies.
On the other hand, I’d view a brain-breaking book as a “press X to self-modify to devout Y-believer” button. If I know the book is such, I decide not to read it. If I’m ignorant of the book’s nature, and I read it, then I’m screwed.
True. So in the process of deceiving yourself, you must first become irrational. The problem is then protesting that you are “still a reasonable person.”
True. So in the process of deceiving yourself, you must first become irrational. The problem is then protesting that you are “still a reasonable person.”
Not quite; you might choose to deceive yourself for decision-theoretic reasons. For example, the Zoroastrian Inquisition might be going around with very good lie detectors and punishing anyone who doesn’t believe. We usually equate rationality with true beliefs, but this is only an approximation; decision theory is more fundamental than truth.
This presumes that your mind can continue to obey the rules of Bayesian updating in the face of an optimization process that’s deliberately trying to make it break those rules. We can’t do that very well.
OP argued that self-deception occurs even if your brain remains unbroken. I would characterize “not breaking my brain” as allowing my prior belief about the book’s biasedness to make a difference in my posterior confidence of the book’s thesis. In that case the book might be arbitrarily convincing; but I might start with an arbitrarily high confidence that the book is biased, and then it boils down to an ordinary Bayesian tug o’ war, and Yvain’s comment applies.
On the other hand, I’d view a brain-breaking book as a “press X to self-modify to devout Y-believer” button. If I know the book is such, I decide not to read it. If I’m ignorant of the book’s nature, and I read it, then I’m screwed.
True. So in the process of deceiving yourself, you must first become irrational. The problem is then protesting that you are “still a reasonable person.”
Not quite; you might choose to deceive yourself for decision-theoretic reasons. For example, the Zoroastrian Inquisition might be going around with very good lie detectors and punishing anyone who doesn’t believe. We usually equate rationality with true beliefs, but this is only an approximation; decision theory is more fundamental than truth.
That only seems to mean that you were a reasonable person.