Suppose we carefully examine an agent who systematically becomes rich [that is, who systematically “wins” on decision problems], and try hard to make ourselves sympathize with the internal rhyme and reason of his algorithm. We try to adopt this strange, foreign viewpoint as though it were our own. And then, after enough work, it all starts to make sense — to visibly reflect new principles appealing in their own right. Would this not be the best of all possible worlds? We could become rich and have a coherent viewpoint on decision theory. If such a happy outcome is possible, it may require we go along with prescriptions that at first seem absurd and counterintuitive (but nonetheless make agents rich); and, rather than reject such prescriptions out of hand, look for underlying coherence — seek a revealed way of thinking that is not an absurd distortion of our intuitions, but rather, a way that is principled though different. The objective is not just to adopt a foreign-seeming algorithm in the expectation of becoming rich, but to alter our intuitions and find a new view of the world — to not only see the light, but also absorb it into ourselves.
Yudkowsky, Timeless Decision Theory