Measuring Epistemic Rationality

Collecting some previous discussions on the matter, trying to reach a conclusion that would benefit me in my path to be more rational. Written with haste to serve as a reference for myself, possibly useful for you, and to be more accountable to be methodological in my rationality self-improvement.

1. The Rationality Checklist from CFAR has many rationality habits listed. I can go through it once in a while to feel progress, or focus on one that I feel I need to improve more and incorporate to my daily routine. (say- ” I notice my mind flinching away from a thought; and when I notice, I flag that area as requiring more deliberate exploration. ”). I’m not sure that I trust myself to reflect accurately. This is also “Not meant to be a way to get a ‘how rational are you?’ score, but, rather, to help you notice specific habits you might want to develop. ”

2. Aliveness in Training—Gives the analogy of a Martial Artist, that improves by working against an opponent. I have three thoughts -

i. Work with a mentor, seriously consider criticism from friends, write down more of my beliefs publicly.

ii. Measure my research success. (Correlated with lots of other things, and unclear, but that’s actually what I care about)

iii. Make lot’s of predictions

3. In Swimming Upstream, TurnTrout gives his personal experience with instrumental rationality. Parts of this story makes me think that I can measure how seriously I take my beliefs by collecting my own decisions and counting how many of them were due to newly generated beliefs. Not sure about that, because I may be biased in the wrong way here. I think that a fine idea here is to track all decisions made, and their reasons, and afterwards I’ll understand how to analyze them.

4. It turns out that there is a Rationality Quotient, by Keith Stanovich. I did not find the test, but I feel aversion toward it as a method to improve rationality. I think it’s because I visualize myself Goodharting to it. It might be very useful as a one-time diagnostic tool.

In total, I think that I should

be systematic in analyzing my shortcomings
work on rationality habits like any other skill/habit
use my intuition to guide me to the most important skills I need to develop
be more clear about my reasoning, and seek criticism and help when communicating
track predictions and world view better